Search
Obie Fernandez
Patterns of Application Development Using AI
Edit
#artificial-intelligence-ai
#behavior-driven-development-bdd
#ruby-on-rails
#design-patterns
#large-language-models-llm
#generative-ai
#application-development
Summarized using AI

Patterns of Application Development Using AI

Obie Fernandez • September 13, 2024 • Sarajevo, Bosnia and Herzegovina • Talk

Summary of the Presentation: Patterns of Application Development Using AI

In this engaging talk at EuRuKo 2024 by Obie Fernandez, the main focus is on utilizing AI components within application development, based on the presenter's recent experiences with the Olympia.chat platform. Drawing from his extensive background in software development and Ruby on Rails, Fernandez explores the integration of AI in a way that enhances developer creativity and innovation while avoiding disruptive changes to existing workflows.

Key Points Discussed:

  • Introduction to AI in Development: The speaker reflects on the overwhelming nature of integrating AI and how it can transform developers' approaches.
  • Practical Application and Experience: Over the past year, the presenter has worked intensively with AI, leading to practical insights and the creation of the book "Patterns of Application Development Using AI".
  • Abstract Patterns: Fernandez introduces abstract programming concepts for AI, such as Narrowing the Path, which emphasizes the importance of precise input to the AI to avoid ambiguity and ensure desirable output, and Multitude of Workers, where AI is envisioned as discrete components functioning like human workers within an application, aiding in problem-solving and task automation.
  • Concrete Patterns: Specific coding methodologies discussed include Structured IO, which involves presenting data in XML or JSON formats to facilitate better understanding by AI, and the Ventriloquist pattern, where AI generates responses based on structured prompts and prior interactions.
  • Example Case: The presenter shares a case involving clients in the reinsurance industry who sought to enhance their actuary capabilities using AI, where mission-critical tasks could be managed more effectively through AI components.

Conclusion and Takeaways:

  • The integration of AI should not be seen as a complete overhaul but rather as a means to enhance existing applications through incremental changes and discrete AI components, thereby improving functionalities like account management, customer support, and data processing.
  • Fernandez invites attendees to explore AI's current capabilities while being mindful of its limitations, advocating for a balanced and informed implementation strategy.
  • Finally, he encourages developers to leverage the free and open-source resources he has created, coupled with insights from his book, to navigate their integration of AI effectively into their projects.

This presentation culminates in a vision of building more sophisticated applications by structuring AI interactions in manageable, meaningful ways that speak to both technical and non-technical stakeholders.

Patterns of Application Development Using AI
Obie Fernandez • Sarajevo, Bosnia and Herzegovina • Talk

Date: September 13, 2024
Published: January 13, 2025
Announced: unknown

This presentation draws on over a year of practical real-world experience and explores patterns ranging from code generation to real-time data analysis and automated customer support. Attendees will learn how to integrate AI tools and techniques: both potential pitfalls and emerging best practices.

Taking advantage of AI components can transform an application developer's workflow into an incredibly potent force, capable of competing at an unprecedented scale. This presentation draws on practical real-world experience gained by the presenter over the last 12 months of developing Olympia.chat. We will explore practical approaches ranging from code generation to real-time data analysis and automated customer support. The emphasis is on new tools that let application developers focus more than ever on innovation and creativity.

Attendees will leave with a roadmap for integrating AI tools and techniques into their projects, insights into the potential pitfalls and best practices, and inspiration to explore the boundaries of what a single developer or a small team can achieve with the right tools. The presenter is a well-known Ruby on Rails expert and his latest book, “Patterns of Application Development Using AI” is the definitive tome on the subject.

EuRuKo 2024

00:00:10.040 I like using this one because I like to pretend I'm a stand-up comic I did standup comedy
00:00:16.680 once and um I thought it was
00:00:22.119 funny it's debatable with the audience that I don't I hate speaking in big
00:00:28.640 rooms like this because uh um you see what you might not realize in the audience is that there's a big delay
00:00:36.320 Echo and uh everything that I'm saying I'm hearing like 500 milliseconds
00:00:43.000 later so I have a thread that's going right now that's controlling what's coming out
00:00:49.399 of my mouth but I also have several background threads that's receiving the audio that's coming back into my
00:00:56.719 ears and the first time I spoke in in a room this big I had that problem it was
00:01:02.199 actually worse because it didn't have uh this great sound mitigation here so I
00:01:09.720 had about a full second between what I was saying and what was coming back and at one point I noticed this
00:01:16.840 phenomenon it was about 10 minutes into my talk
00:01:22.840 and the background thread in my head was listening to what I was saying that's not a good situation to be in when
00:01:29.159 you're giving a talk because it was kind of like dominating my
00:01:34.840 attention and I guess I had a buffer of words that were going to come
00:01:40.439 out of my mouth and that buffer went empty while I was paying attention to
00:01:46.000 the Echo and all of a sudden I had no idea what to say next it was just very
00:01:52.159 uncomfortable I was like this
00:02:03.880 wow okay so anyway thanks for joining me today uh my name's Obie
00:02:21.200 along with some of you yesterday uh at the party I'm not only the DJ I'm also
00:02:26.920 presenting as you can tell um thanks for those of you that stayed the duration thanks to the guy whose shirt
00:02:34.280 was completely soaked and sweat nothing nothing gets a DJ off like seeing one of
00:02:40.319 the dancers completely soaking wet I don't know if someone dumped a bucket of water on him or he's just like no he was
00:02:47.080 dancing that hard right and thanks to everyone who tore it up in karaoke and
00:02:52.680 then came over and finish the night yeah you know what I'm talking
00:02:58.360 about you guys are crazy crazy don't you know this is Bosnia you're not supposed to get that crazy
00:03:04.720 here jeez all right so let's start my talk uh
00:03:11.159 my talk is called patterns of application development using AI rolls right off the
00:03:17.040 tongue um the reason I called it that is because I'm a fan of the old patterns
00:03:22.799 books who who has read the gang of four design patterns book show of hands Okay so crowd I've
00:03:31.159 spoken to some young developers these days and they don't even really know what a patterns book is any of you
00:03:37.439 noticed that it's um it makes me feel old geez but um anyway I want to tell
00:03:45.840 you the story of how this concept of using AI components in application code
00:03:52.519 came about and I want to give you an introduction to some of these patterns so that you have an idea of what uh you
00:03:59.239 might be able to experiment with in in your own applications so really quick bio because
00:04:06.560 I know some of you don't really know who I am I wrote this book The Rails way uh before writing this book um I had been
00:04:13.920 involved in software for for a while uh I'm coming up on 30 years of
00:04:19.000 professional software development experience although you wouldn't tell from this gorgeous body
00:04:25.800 um I started as a full stack Java developer uh Circa
00:04:32.000 1995 um I remember sitting on the first uh T1 connection of
00:04:39.080 anyone that I knew in 1995 and downloading the first
00:04:44.160 jdk and it took something like 15 20 seconds and I was amazed I was like wow
00:04:51.000 this is so fast um but those of you that have been around that long you know probably
00:04:56.560 understand that that that was literally some of the first job Java uh stuff in the world so I was
00:05:02.560 really really lucky uh with timing and then I got really lucky again because in
00:05:09.280 2004 my friend AAC helis soy at thought Works introduced me to Ruby and very quickly thereafter I became acquainted
00:05:16.880 with Ruby on Rails and I've been doing Ruby on Rails since early 2005 since pre1 uh and I was probably one of the
00:05:23.800 first people in the world to do uh rubyan rails commercially at thought works for John Deere a very very big
00:05:31.080 company um in 2007 the rails way was published and I
00:05:36.520 started hashrocket uh which became a very well-known consultancy I later sold
00:05:41.639 it it's still going on great company I um did a brief AI related
00:05:47.039 project in 2019 so I'm not completely completely new to this space um and then
00:05:53.319 when the GPT API came out the open AI uh apis became available one of the first
00:06:00.120 things I did was to write a chat jiki clone is you could call it that called magma chat and I presented it at reals
00:06:06.680 comp in 2023 and open sourced it and quickly abandoned it because open source
00:06:11.800 is not sustainable um but there was a cool thing that
00:06:17.199 happened as a result of that which was that um I was dating this lovely lady in
00:06:22.639 the first row here Victoria and um she was complaining about her
00:06:28.880 copywriter work and I said hey let me write you an AI copywriter using this open source
00:06:35.400 software that I had come up with and uh I gave it to her and she started using
00:06:41.000 it and it was very obvious that there were commercial possibilities here so we got together she quit her job and became
00:06:46.720 CEO of a company called Olympia and that's where I serve as a chief scientist which is a really glorified
00:06:52.840 way of saying CTO um if you've done 30 years you get to say that but so this is Olympia it's
00:07:00.360 olympia. chat if you want to check it out uh AI powered Consultants it's really like a very very powerful chat
00:07:07.360 gpts sort of thing and includes a lot of stuff I'm not here necessarily to
00:07:12.720 to you know plug that but this is me looking cool uh in Las Vegas there a
00:07:19.800 little conference that I highly highly recommend called Sin City Ruby I know some of you that were there um so I gave a talk which I
00:07:28.080 thought was a really cool talk called called the the the biggest wave of all and I was talking about how AI is I see
00:07:34.759 it as so revolutionary wow what a novel thought but the U the reason I saw it so
00:07:41.000 revolutionary was because of what I was doing in Olympia using AI inside of what's otherwise normal code yes Olympia
00:07:48.000 is an AI app quote unquote
00:07:57.360 but hello hello okay it's
00:08:02.960 a but it's a it's a Ruby on Rails app and it's a SAS application that is otherwise a normal rails app let's call
00:08:10.199 it and it happens to use a lot of AI components inside of it to do things that would have to be programmed in
00:08:16.840 otherwise normal ways and after discussing this someone in the audience said Obie this seems really great but
00:08:23.319 how do I get started and there was like a moment of insight
00:08:30.479 in my head where I realized okay I've been immersing myself in this for over a year at this
00:08:35.599 point and to a regular developer who has not been immersed in it it can seem
00:08:43.000 overwhelming and raise your hand if you have felt overwhelmed by the amount of
00:08:48.480 things going on in in generative Ai and llms and I I I would be surprised if most of you didn't like raise your hands
00:08:54.880 here uh you know oh back to standup comic mode
00:09:02.760 great hello okay so um on the plane back from Las Vegas
00:09:12.079 um I said to Victoria I'm going to start writing a book on this so I liter literally started writing it uh on the
00:09:18.000 plane and luckily I had collected notes uh throughout the year as I was writing
00:09:23.279 Olympia on patterns that I thought would someday be useful to share and the key
00:09:29.120 question is how can I apply AI in my day-to-day
00:09:34.160 programming without changing everything because it AI does seem and a lot of people talk
00:09:41.440 about it uh as something that is like a revolutionary change it's a see change
00:09:47.640 Chang everything about how you do everything and those those kinds of changes are intimidating to say the
00:09:53.720 least uh they're also very difficult to put into effect especially if you don't
00:09:58.880 just work for yourself on your own project uh so how can you incrementally
00:10:04.000 take advantage of uh some of these revolutionary things and that's what the
00:10:09.440 book uh talks about when I get to the end I I'll I'll share with you how to get the book because it's
00:10:15.440 finished uh in the book I have a section on abstract patterns and also on
00:10:21.560 concrete patterns and the book itself is modeled after some of the classic uh patterns books like uh Martin Fowler's
00:10:28.720 patterns Enterprise application architecture stuff like that um as I
00:10:34.240 mentioned I think I mentioned um I'm going to cover a couple of abstract
00:10:39.360 patterns and I'm Depending on time I'm going to try to cover a couple of concrete patterns
00:10:47.399 but the first of the abstract patterns that I want to talk about is called narrowing the path keep in mind that
00:10:53.920 this is that these abstract patterns represent ways of thinking of about
00:10:59.800 using AI in your code and at the most abstract level the
00:11:05.519 first thing you need to do to use AI in your code is to understand kind of what what can it do what can you expect it to
00:11:11.639 do and the novice that's starting out and using AI in their code in some way
00:11:17.440 and maybe some of you relate to this um Can very quickly get bogged down in the
00:11:23.320 AI not doing what you want it to do I asked it to do something it didn't do
00:11:28.360 the right thing so either it's stupid um or there's limitations to
00:11:34.440 generative AI um or the problems between the what do they say the keyboard and
00:11:40.600 the computer or the keyboard and the seat a lot of the times the AI is not
00:11:45.920 doing what we wanted to do because we have not constrained the latent space
00:11:52.360 enough the latent space is a vastly multi-dimensional space
00:11:59.519 uh where the knowledge of these um state-of-the-art models is kept it's the
00:12:06.320 brain of the AI when I say unimaginably vast I mean you have to start getting
00:12:12.360 into pretty deep mathematical Concepts to even start to to think about how big
00:12:19.079 uh this space is right and given that these Frontier models as they call them
00:12:25.160 are trained now with like trillions of of data points the the problem when you start prompting
00:12:33.160 one and trying to get some sort of usable output is that there is too much there so for
00:12:43.279 instance let's say that you're asking for a particular kind of information well keep in mind it probably has
00:12:50.720 hundreds to thousands or more different kinds of representations of the same information or related information as
00:12:57.360 what you're asking it to to use or communicate back to you there's also
00:13:02.880 contextual ambigu ambiguity that's introduced by that overload right so unless you're really
00:13:09.240 really precise with what you're talking about it's going to make an statistical
00:13:14.519 assumption as to what what you mean what you're talking about there's also
00:13:19.600 temporal inconsistencies because like for instance the you know the frontier models have been fed with information
00:13:25.120 from the internet and not all all of that information in fact probably most of it
00:13:31.760 is missing some indication of when that information is
00:13:38.079 valid uh so unless there's a preponderance of information that is current represented in that latent space
00:13:46.240 and you don't specify there's very good chances that it might be using outdated
00:13:51.480 information to to actually uh produce output and then of course there's domain
00:13:57.440 specific nuances so if you're talk if you're using a a large state-of-the-art
00:14:02.519 model for some sort of domain specific purpose unless you constrain the space
00:14:08.920 which you call narrowing the path it may not think you know of the output
00:14:15.839 in the way that a domain specific uh expert would think of producing an output some of these things may seem
00:14:22.360 obvious to you if you've started and again I'm going down kind of in specificity and abstraction
00:14:31.399 here's another abstract pattern that is useful I think uh is called multitude of
00:14:39.880 workers the reason that I think this is useful is
00:14:45.240 because um especially if you're working with clients and they you want to you
00:14:50.839 know or your boss or product manager whoever and that they want to use AI okay so now you need to think about how
00:14:57.000 to use AI or even in your own stuff it seems very magical and we'll
00:15:02.480 get to some of that magic aspect later but thinking of
00:15:09.519 AI as uh little human workers that I can insert into my code base helps me
00:15:17.240 to go from Magic to something that is not like Arcane you know wizard
00:15:24.480 undefined magic to like a magic trick let's say you know like it it
00:15:30.880 collapses the way that I think about the AI into something that is actually understandable okay um and let me give
00:15:38.680 you an example I was working with a Consulting client I still do a little bit of Consulting by the way if anyone
00:15:44.959 wants to talk to me about that um and they wanted to use AI saying
00:15:51.440 scare quotes to improve um the results of some of their actuary analysis for re
00:15:59.079 insurance and without getting into any specifics reinsurance is a fairly complicated domain it's the world of uh
00:16:06.680 specialized accountant analysts that uh do these kinds of figuring out what is
00:16:11.959 your insurance going to cost right and
00:16:17.120 um the client uh you know investor in this Venture had been reading up on you
00:16:22.720 know all the advances in Ai and he wanted AI to help fix or improve on existing things so I said okay well one
00:16:31.240 way to think about this is what you know if you have a
00:16:37.519 particular problem you're trying to solve I said what is the problem anyway
00:16:42.560 at that point and they he said well our actuaries don't scale I said okay I can
00:16:49.040 kind of understand that uh so I said do you you want to like provide more
00:16:54.440 actuary service yeah kind of um at a scale that hasn't been done before so
00:17:00.199 that we can price Insurance dynamically at a scale that hasn't been done before I said okay well what would you do if
00:17:07.039 you had a an army of like actuaries that were
00:17:12.240 like little Almost Human but like little robots and things like that and then we kind of talked through decomposing the
00:17:18.640 problem into um almost a microser service approach
00:17:24.760 but I mean the thing is if you think of them as little humans that are constrained than what they're able to do
00:17:30.520 AKA like agent based thinking it it just helps you in terms of visualizing the
00:17:36.360 architecture so I was able to take this client from really just not understanding anything other than this
00:17:42.760 magical AI thing will help fix you know or make my vision a reality to
00:17:48.640 specifically okay if we have these actuaries that perfectly follow instructions that are not limited to
00:17:54.080 their personal set of um Excel spreadsheets uh that can be spun up a
00:17:59.919 million at a time you know these kinds of things help to clarify the the
00:18:05.000 architecture of what a solution might be in this space and at this point I need to
00:18:12.280 clarify in case it's not obvious that the use of AI in this way I call
00:18:19.400 discrete AI components so I'm not talking about an application that you
00:18:24.480 put into a prompt and expect to just get like the The Prompt is the application that's not what I'm talking about I'm
00:18:31.280 talking about incorporating AI logic in discrete
00:18:36.840 components or in little services that function as blackboxes inside of your current
00:18:43.600 code these intelligent behaviors um Can Implement parts of your business
00:18:51.280 logic and this is where this whole thing starts to become pretty fun uh and profitable I think there are parts of
00:18:59.320 any given application that we have to write out um and some of it like
00:19:06.320 let's say authentication or a lot of many many different things we've kind of done a million times already and there's
00:19:12.960 other classes uh of problems that we haven't and many times the reason that we don't
00:19:19.600 do those things is because we don't get the ROI from being able to to implement them
00:19:26.720 it is too it's too difficult or too complex or maybe it's even too far-fetched you know some some of these
00:19:32.440 things that sometimes that your clients or product managers ask for are just simply not doable giving the
00:19:39.120 constraints if you have uh this kind of like little magic black box that's able
00:19:45.000 to encapsulate anywhere from dozens to millions of lines of code inside of it
00:19:51.760 and you just have to worry about how to interact with it in terms of inputs and outputs then that actually starts to
00:19:56.880 make certain kinds of things easier or just possible in the first
00:20:02.320 place so that's what I mean by disgree components and modularity is a key this is very very heavily influenced by oo
00:20:10.520 thinking uh proponents of O uh tell us to think about interactions between our
00:20:16.799 objects as messages and when you're using AI uh to power discreete
00:20:22.039 components you almost can use them as literal messages objects talking to each
00:20:27.640 other in the way that hum would talk to each other uh in case you can't read what's
00:20:33.320 in the chat bubbles here's an example user 742 just
00:20:38.480 to represent a user right hasn't logged in recently and another
00:20:44.760 component that that was said to internally in the app says let me look at their activity history so it goes and
00:20:52.039 plls their activity history it analyzes them and then it uh maybe the first
00:20:59.679 component or another component that it talks to says okay I'll send them an email about our latest new feature this
00:21:05.960 would be an example of let's say uh improving engagement so so these are building
00:21:11.440 blocks inside of your app that act like little workers uh you know doing stuff
00:21:17.440 for your app the inspiration for this style of
00:21:22.559 programming the one of the reasons that I find it really really interesting is because I was there at the birth of behavior driven development I worked
00:21:28.679 thought works and my colleagues Dan North and number of other notable people
00:21:34.720 including my old friend ASAC who wrote cucumber we were doing this and we were
00:21:40.520 doing this on client projects and we were doing it heavily I also did a lot of it at hashrocket and in case you're not familiar with
00:21:48.480 it uh what what you do with bdd and bdd testing Frameworks is you write out in
00:21:54.960 English or your chosen language anyway uh but in in plain language what you
00:22:00.480 want your application to do uh and then that drives tests that actually tests
00:22:05.960 whether it does what you wanted to do and this particular example says given I'm on the homepage when I click the
00:22:11.200 login button then I should see the login form this was all great and we got a lot
00:22:17.360 of mileage out of this but it was a pain in the neck to implement the what went behind this so
00:22:25.039 there was a bunch of let's call it Armature or you know like the the supporting code that made this work it's
00:22:33.679 kind of like the pseudo parser style matchers that match to the different plain language and it was it was very
00:22:40.919 difficult um and here I have to stop and say
00:22:46.360 there's a trigger warning I mean I I don't know that all of you like magic I mean I know if you like rails you're
00:22:51.799 generally okay with magic but some of the stuff I'm about to show you that is
00:22:56.960 kind of like this bdd on St start starts to feel pretty
00:23:02.360 magical so one of those types of workers uh that I talk about in the book as a
00:23:09.120 multitude of workers I'll give you an example the account manager in
00:23:15.440 Olympia this is the prompt that drives it so this is a discrete AI component in
00:23:21.320 Olympia and it's prompt I'll read to you you're an intelligent account manager
00:23:26.400 for Olympia the user will request change to their account and you will process those changes by invoking one or more of
00:23:32.679 the functions provided escalate to human support rep if you encounter any
00:23:38.000 problems do not allow the user to change their account or add a new AI assistant unless their account subscription status
00:23:44.039 is active make sure to notify the account owner of the result of the change request whether or not it's
00:23:50.080 successful always end by calling the Finish function so that we save the state of the change request as completed
00:23:56.679 this is the directive that is given to the discret AI component that is acting
00:24:02.120 as the account manager for a subset of the account management capability in
00:24:09.080 olympia.
00:24:14.200 chat I know this might be a little difficult to see if you're in the back of the room but it's this is the code that implements that account
00:24:22.039 manager and there you see from lines 12 to 21 the system directive and then
00:24:28.880 the function this is an example of a function that is given as a
00:24:35.520 tool to this discret AI component that it can use to do part of its job uh in
00:24:42.000 this case it's addbot config to account because one of one of the functions it can do is to add a new bot to your
00:24:51.360 Olympia account and even if you're in the back you can tell the meat of this
00:24:57.720 function is just normal Ruby application procedural code there's nothing AI about
00:25:03.240 the implementation of this tool but this tool is one of half a dozen that the
00:25:08.960 account manager has in addition to being able to uh add a bot config to a particular
00:25:15.520 account it can also uh talk to customer support it can
00:25:20.640 send an email to the customer um those emails are not predefined in templates or anything like
00:25:26.799 that it actually decides what to right to the customer um it can also close the ticket so
00:25:34.559 there's a workflow aspect to this there's a state machine and basically after it decides that it has worked on
00:25:39.679 it and I want to I hate that I have to go through this so quickly because I could do a whole workshop on it but I
00:25:46.279 want to talk about the power of this approach this part in green
00:25:52.120 here reads do not allow the user to change their account or add a new AI assistant unless their account
00:25:57.840 subscription status is active this was not in the first version of my
00:26:03.240 prompt so um someone someone knows what's
00:26:09.520 coming here so what what happened was in the first version this was missing and I rolled it out to
00:26:16.000 production and about a week later I got a message from our customer support guy and he said hey we just had a case where
00:26:23.159 this thing failed because it let someone add uh a new bot to their account even
00:26:28.799 though their account was uh scheduled to be cancelled and I said yeah that doesn't
00:26:34.240 seem right so he said great do you want me to add a story to notion and I said no I already fixed
00:26:40.240 it because I just opened this file and I added the part that's in
00:26:46.520 green and I went ahead and I tested it and it worked fine so like basically now
00:26:52.080 if your account is scheduled to be canceled and you try to add a new bu config it'll send you an email and
00:26:57.880 saying hey hey I'm sorry I was not able to complete your request now there's something really really important that I want to make sure you take away from
00:27:04.640 this this is internal code I'm not talking about the user
00:27:10.600 typing a request to an account manager bot
00:27:16.320 no the way that a user on Olympia adds a new bot is by going to a catalog of bots
00:27:23.399 and pushing a button so that's at the user interface layer
00:27:28.720 and then at the controller level that's where a message is produced that is sent
00:27:34.120 to the account manager that is insulated from the user that is constrained and it says something like hey user uid blah
00:27:41.480 blah blah wants to add bot config blah blah blah blah blah and packages it up
00:27:46.960 into a change request and sends it to the account manager for execution so this is just a tip of the iceberg in
00:27:54.519 terms of what you can do it's literally implementing business logic as prompts and I
00:28:03.679 think I mentioned that be uh on the slide but I didn't mention it before the
00:28:08.880 end goal kind of The Logical conclusion of this where we're going with it and I talk about in the book a little bit is
00:28:15.440 that eventually I think we can write entire applications just using props and that's not like small applications what
00:28:22.600 I mean is that eventually this behavior-driven development kind of style can be applied to our application
00:28:29.120 development but that's far off in the future I I want to get back to to reality even though this is
00:28:37.440 magical um briefly if you're saying how does this apply to me and what I work
00:28:45.679 on here's you know kind of something you know big big categories of domains so
00:28:52.880 account management almost any software has many of you probably work in e-commerce many of you probably work in
00:28:59.279 healthcare these are all areas where discrete AI components can make a big big difference um I recently have been
00:29:08.519 working a little bit with some clients that are uh medical doctors and they run
00:29:14.600 clinics and we're talking about using this kind of approach to handle things
00:29:21.200 related to Patient intake for instance uh collecting medical history system assessment Etc these are
00:29:30.880 all uh you know these bullets represent um what previously would take entire
00:29:37.840 companies it's not even like departments within uh development uh you know
00:29:43.760 there's probably entire companies that handle Insurance verification uh but it starts to become
00:29:49.640 achievable to kind of incorporate some of these things into uh your app as
00:29:54.760 components
00:30:01.039 okay I don't know how I'm doing on time I'm out of
00:30:06.320 time maybe uh briefly let's just do a little bit on concrete
00:30:11.480 patterns um I break them up into different categories one of the big categories is prompt engineering and the
00:30:18.679 example I have two examples but one I'll talk about is called structured
00:30:23.919 IO and this refers to structured input and structured output
00:30:28.960 again this is an implementation pattern that gets more concrete before we were talking about kind of like big architectural patterns abstract ways of
00:30:35.600 thinking about how to use AI in your code this is specifically how to code AI
00:30:41.000 into your code a concrete pattern structured input helps the llm better
00:30:46.360 understand what you're talking about by presenting it to the llm as uh XML or
00:30:52.519 Json or some sort of defined format and you ask the llm a I to give you
00:30:59.760 structured output back um after doing a lot of this stuff with
00:31:06.200 Json I made a wild ass prediction was that XML was going to make a comeback because XML is like infinitely easier to
00:31:13.760 parse in a lightweight manner than Json um you
00:31:20.000 know you can whip up you know matching for XML tags super super
00:31:26.039 easily uh here's an example of what I'm talking about uh the description for this
00:31:32.480 product this is uh part of some fictional e-commerce uh
00:31:37.840 application and uh you provide the description in an X wrapped in an XML
00:31:43.919 tag and you say extract name size price and color from this product description you don't have to say much more than
00:31:49.639 this the LM understands it and it gives you valid XML
00:31:55.320 back uh I'll give you one more more implementation pattern concrete one that
00:32:01.320 I call the ventriloquist um in this case you're not only expecting uh to give an
00:32:08.080 input to the AI and get an output you're actually providing some of the ai's output as well you're having a fake
00:32:14.919 conversation with the AI and how does that work well let me just show you uh
00:32:22.720 hopefully you can read this but this uses a framework an open source framework that I have published very
00:32:28.360 lightweight called Ray I'll show you how to get that later as well the transcript is being put
00:32:34.799 together in this method uh lines 11 12 13 and 14 the system prompt is your
00:32:40.399 powerful web search engine the user then asks a
00:32:47.799 question and then you provide the answer that the Assistant gave and the
00:32:53.279 assistant said searching no results found why did I do do this because what
00:32:59.919 I'm trying to do is up here specified in the prompt matching the original language of the question generate three
00:33:06.559 alternate keywords that might produce better results reply with just the list one per line if you if you do this
00:33:13.720 without the ventriloquist pattern sometimes it knows what you want and
00:33:19.159 sometimes it doesn't it's actually a really really difficult problem and I struggled it for
00:33:25.200 with it for a while and I had this component that didn't use Inus pattern to do this kind of
00:33:30.559 thing and then I said you know what let me narrow the path a bit let me
00:33:35.600 constrain it so that the the component has a little bit more context and
00:33:41.880 understanding of what I'm trying to do and one of the ways that you do that is by inserting this first
00:33:48.279 exchange I asked the question and there were no results this n Narrows the path that constrains the latent space I was
00:33:55.000 talking about before to where the The Prompt that I'm trying to accomplish
00:34:00.679 actually makes more sense oh there were no results found and now I'm trying to
00:34:06.399 generate different kinds of queries that might actually get a result so that's the alternate keywords
00:34:14.560 component there it is a little bit bigger um there are patterns for
00:34:20.599 discrete components but I think I am out of time can someone tell me if I'm out of time like for
00:34:25.720 real okay okay um I'm just going to skip these
00:34:31.119 last patterns I will make uh the presentation available if you're interested in the ray framework uh it is
00:34:37.200 available at this URL is uh one of the several uh open- Source uh libraries
00:34:44.280 I've made available that have been extracted from uh Olympia Ray gives you just drop dead
00:34:51.800 simple ways to include chat completions inside of your normal classes uh and
00:34:58.440 finally if you haven't already bought my book I know a lot of you have because you've told me um this is it's available
00:35:06.560 at leanpub just search obai and you you'll find links to it um you can get
00:35:12.400 it it's usually on sale for like 31 bucks or something like that it's a 500 page book it is a substantial book I put
00:35:19.440 a lot of time into it in the last few months um or the three months before that anyway uh it will be published in
00:35:26.640 hard cover uh available through Amazon hard cover soft cover Kindle Etc but you
00:35:31.880 can get the Complete Book right now uh in ebook format PDF Kindle uh Etc on
00:35:38.119 lean Pub and it goes into a lot of detail about how to do this and I
00:35:43.440 promise you it's not handwavy crap it is like
00:35:48.960 95% knowledge and code extracted from Olympia which is production software
00:35:54.319 that's running today and serving uh hundreds of custom customers every day
00:35:59.440 for critical applications there there are things that the AI is clearly not
00:36:05.359 suitable for like probably don't uh make a hospital patient dependent on this
00:36:11.480 stuff um but there are a lot of things that it is already very useful for so anyway thank you for letting me present
00:36:17.319 my crazy ideas and uh if you want to talk more about it uh you know please find me in the hallway
Obie Fernandez
@obie
Explore all talks recorded at EuRuKo 2024
+39