SF Bay Area Ruby Meetup

00:00:04.319 um everyone welcome welcome back to SF Ruby um

00:00:10.440 hey yeah uh Happy New Year uh and super super grateful to uh see all of you here

00:00:18.439 today and to kind of like begin this year together officially um and uh we have a pretty

00:00:27.720 cool um plan for today uh I'll begin with a

00:00:34.680 few a few updates okay uh I I will say the setup

00:00:42.120 today is a little bit tricky but we will figure it out

00:00:48.120 so the the biggest news is Jason SWAT arrived U to San Francisco and of course

00:00:54.800 he couldn't just come and not speak ATA Ruby yay

00:01:02.079 um so we'll hear from Jason in in a sec uh first is this

00:01:09.840 ah okay um uh I want to share something of course we had before between the last

00:01:16.640 Meetup and this one there were releases of rail Sid Ruby 3.4 and I I just saw

00:01:23.600 something I wanted to share yesterday um this is Maxim's uh post

00:01:29.680 about about uh uh Ruby 3.4 with white jet being not only faster more

00:01:37.079 productive but also less more memory efficient I think this is a cool new

00:01:43.159 update with the Y team is also now I think the team is now also optimizing for memory so I think it's it's pretty

00:01:50.560 cool um uh the other update is yeah we have this super cool page at SF Ruby oh

00:01:57.759 at um Ruby videos dode Ruby

00:02:03.560 video.de um and this is where you will find all the recordings including

00:02:10.039 including the recording of this one and we're also uploading the recording let's say every

00:02:16.599 time now um also there's this update um

00:02:22.360 there's this uh small rails shop gusta um and they reached out proposing

00:02:30.640 to sponsor our Meetup so I'm collecting

00:02:36.680 ideas what should we do yeah and one of the ideas for example one of the ideas

00:02:43.920 yeah just collecting ideas we could do for example we could do a a sruby scholarship for somebody uh so I'm just

00:02:51.319 thinking about something more meaningful because I think you all have a lot of merch already

00:02:57.519 um assuming um what way do they want to sponsor this in a way of giving us

00:03:07.879 money I mean yeah just just let's collect ideas as a as a commune

00:03:14.440 because swag what yeah um let's collect ideas um talk

00:03:22.840 to me after in the break or after this and uh also uh you can sh me an email or um

00:03:30.879 reach out on those social media um that's that um the next meet up

00:03:38.959 I just you didn't see it right no okay uh

00:03:44.720 what's your favorite venue for this Meetup product board is you can not no

00:03:52.319 no not product board except product Board second favorite second favorite

00:04:15.160 employer okay which one Center Center oh yeah but

00:04:22.079 yeah I I still want I still I I still I'm still looking forward to the hoko one you know the Salesforce yeah but we

00:04:28.919 we we're not the there yet yet yet um

00:04:34.080 yeah it's going to be at GitHub um yeah uh they they have a Prett

00:04:40.039 cool venue and uh we just we actually had uh notice

00:04:46.199 that the date is February 11 um two days ago if you went there to Luma SAU it was

00:04:53.240 February 13 kind of had to change a little bit um so that's going to be

00:04:58.280 February 11 uh please sign up on Lumar this is the only

00:05:03.680 way I can reach you um for free um

00:05:10.240 so yep hope hope to see you there and of course as as as always there is a link

00:05:15.400 to submit a cfp so please submit your talk proposals if I'm not responding to

00:05:21.000 your proposal reach out using this

00:05:26.720 email uh and yeah let's begin so um uh

00:05:33.080 now um clao from who's leading uh engineering at product board we'll tell

00:05:39.639 you a bit about where we are uh and uh we're super grateful for product to prod product board for hosting us for the

00:05:46.520 second time this year so let's give them a shout

00:05:54.600 out hey everyone so I'm gloud you I'm leading the San Francisco office

00:06:00.080 engineering and um how many know about prog board before

00:06:07.000 today okay it's a good it's a good amount so I to be honest I didn't know

00:06:12.840 much before I joined because what we do is actually we're trying to help

00:06:18.400 companies build better products right for their own customers that means we want to want to help them understand

00:06:24.639 their own customers and then take that information and act on it whether they're they're starting planning on and

00:06:30.000 thinking how Road mapping and putting everything in in motion right so this is

00:06:35.160 what we do we've been around for uh I think a company officially started 10 years ago but we launched around eight

00:06:42.080 years ago and uh we launched on ruon Rails um we've been uh it's still the

00:06:48.840 biggest part of our our code base I think and we got to be successful on

00:06:54.639 Rubi on Rails right so uh that's why we're hosting we're contining to hire

00:07:01.160 people not only Ruby on Rails but or Ruby but we're we're we're continuing

00:07:06.680 hiring right now we have open roll for our growth teams um and uh feel free you

00:07:14.160 know this Market I just read that meta is going to lay off on a bunch of people

00:07:19.680 so if you have friends if you're interested with hiring we have a great team here we have another great team in

00:07:26.160 Prague so we we travel between the the two offices is um with another perk so

00:07:32.199 feel free to say hi to us um say hi to me say hi to people that have Pro board

00:07:37.680 you met Jana and Sabrina at the entrance both downstairs and upstairs go and say

00:07:43.919 hi to them there are recruiters and uh yeah have fun today thank you thank you yeah cool

00:07:54.520 again yeah I think it's it's a pretty amazing front board just an amazing case of

00:08:00.120 growing on Rails being successful on Rails uh folks there there are few chairs there um please don't be shy to

00:08:08.360 pick a chair do we need more what do you think we might we might need more

00:08:15.759 chairs uh I think Marlin will help us with that yeah thank you um so uh with

00:08:22.039 that uh we get we have I know you're all cannot wait for Jason to talk to you

00:08:27.800 about uh writing B tests more tests that that are easier to

00:08:34.200 understand this is the criteria this is yeah this is let's talk about this after that we have Bart with pasis at uh how

00:08:43.800 to um like an intro to pass keys at ra applications in real applications and

00:08:49.040 then after the break we have camon uh who came up with something interesting with rubian wson h and then Alex um Alex

00:08:59.040 with puz which might might be something new for all of us to to learn today so

00:09:04.480 thank you so much uh again we have an open mic if you're here for the first time today you're not familiar with this

00:09:11.040 it's it's when it's when we all um kind of share short announcements or

00:09:17.040 questions or anything and so this is my call for you to mentally prepare and uh

00:09:24.760 prepare and share something during this Open Mic session um now uh Jason uh come

00:09:31.480 over and let's get Let's help you get set up uh Jason I don't know you want to

00:09:38.680 speak up or you want to try I can just speak up that's fine

00:09:44.440 okay but you you need to join Zoom this is the tricky part

00:09:50.720 um so you need to join zoom and you need to connect to

00:09:56.640 Wi-Fi folks sorry uh 30 second break uh

00:10:02.360 let board gu BB guest BB guest product board small letters

00:10:12.000 2020 just

00:10:56.040 turn turn

00:11:09.160 Shar theid like

00:11:15.399 this and then should just do entire screen no just do this do this

00:11:20.519 okay then okay do you want do you want the that' be

00:11:26.399 nice thank you

00:11:38.360 okay maybe not oh sorry

00:11:44.519 okay it doesn't work okay let I'll just do it manually

00:11:52.720 yeah set up so um folks how many of you U listens to one of the Jason's

00:12:04.600 podcast quite a lot so um Jason is U hosting a c City Ruby conference

00:12:12.320 um raise your hand if you're speaking at C City

00:12:17.639 Ruby this year it's going to be in May April when is it 11th and 12th 11

00:12:27.399 in um and Jon is the author of professional rails testing

00:12:32.880 book could you hold that up for me thank the first time I'm seeing

00:12:38.959 a physical form highly recommend it and also

00:12:45.560 the um what else what else so many things uh snail mail um yeah I have a

00:12:51.160 snail mail newsletter and I brought a bunch of copies to give out um it's I

00:12:56.279 I'll be talking about that during the talk but it's called nonsense

00:13:01.800 monthly so with that please welcome

00:13:16.040 Jason okay talk is about how to write tests that are

00:13:21.399 understandable um what is the biggest problem with most of the test Suites

00:13:27.160 that we work with let's have a little audience participation

00:13:34.399 Cameron yeah flaky flaky okay slow they

00:13:39.720 do exist but flaky they do exist but they're flaky okay slow uhhuh don't make sense that's a big

00:13:48.880 one yeah Okay so we've heard they're annoying is that what you

00:13:55.600 said yeah so there a problem speed flakiness lack of coverage over

00:14:02.560 mocking um I contend that the biggest problem with most of the test Suites

00:14:08.279 that I see is that can't tell what the is going

00:14:18.160 on okay so we're going to talk about that first a little bit about me I guess Arena already introduced me I didn't

00:14:24.720 know she was going to do that um author of professional rails testing host of the code with Jason podcast and I work

00:14:31.839 as a consultant and I'll talk more about that later um but I do Consulting on

00:14:37.440 topics including what I'm going to talk about today so why are our tests so hard

00:14:44.839 to understand does anybody have any idea why this is why does this

00:14:52.560 happen SK skill issue okay

00:14:59.399 um I think in order to answer that question we need to go a little bit deeper and ask ask ourselves some more

00:15:07.120 fundamental questions like what is application code and what are

00:15:14.639 tests might seem like a funny question to ask like what are tests we all know

00:15:19.680 what tests are but can you put it into words what is a test can anybody say

00:15:25.759 what a test is yeah you be testing something like that okay that's one kind

00:15:31.199 of test ABY testing yes go ahead definition of

00:15:38.160 functionality definition of functionality okay and yeah specification specifications all right I

00:15:44.759 wasn't expecting the correct answer so soon um but yeah a test the the way I

00:15:52.120 word it is a test is an executable specification took me many years from

00:15:58.600 when I started testing to when I realized that a test is an executable specification and I started putting it

00:16:05.800 in those terms um okay so that's what a test is and we can talk more about that but let's also talk about what is

00:16:13.000 application code so we have the test code and we have the application code what is

00:16:21.600 code the executable implementation of the specification the executable

00:16:27.639 implementation of the specification that's pretty

00:16:38.920 good oh a process that humans want to repeat over and over again by Machine

00:16:44.680 yeah can you say yours again me yeah yeah yeah an executable implementation of

00:16:53.519 specification an executable implementation of the specification

00:16:59.959 yeah that's interesting and Brad what did you say just a human process or a process

00:17:07.120 that humans want to have a machine repeat over and over yeah a process that humans want to repeat by machine over

00:17:15.039 and over yeah so okay so the conclusion that I arrived at is that code is

00:17:23.400 Machinery um application code is Machinery that creates a product

00:17:29.520 and so I mean product like in a really broad sense of the word like if an endpoint returns an HTTP HTTP response

00:17:38.559 for example that is the product that the code generates um and not to get too like

00:17:46.360 wacky but I realized after I thought about it that like it's not just an analogy like code literally is physical

00:17:55.640 machinery and it literally creates a physical product like code exists on a dis like

00:18:05.480 that's physical so the code even though we see it on the screen it is a physical

00:18:11.600 entity but we don't need to talk about that part um so application code is

00:18:16.640 Machinery that creates a product and tests are executable specifications for that machinery and you could flip that

00:18:24.080 the other way also and say tests are executable specifications and what was your definition

00:18:34.799 again for which one for for code for the application code executable implementation of the specification yes

00:18:42.440 yeah so I wasn't expecting such a thought-provoking answer so now I'm G to go have to like sit in my hotel room and

00:18:49.440 uh and ponder that um I have a talk that I've been

00:18:55.039 been working on the idea that you can write code without tests but you can't

00:19:02.120 write codes without code without specification and I actually checked

00:19:07.200 with the guy Creator rspec and confirmed stpc stands for Ruby

00:19:14.840 specification yeah all right so brief aside uh I I'll repeat some of that in case anybody didn't hear so you talked

00:19:21.760 with Stephen R Baker the creator of arpec and he actually told me the same

00:19:27.240 thing because I talked to him the other day and he said rspec stands for Ruby specification and that makes so much

00:19:34.159 sense he also told me that he doesn't use rspec the creator of rspec doesn't use rspec because it was never intended

00:19:41.760 to be used so we're all doing it

00:19:49.720 wrong um

00:19:56.159 right um so I would like to attempt to

00:20:01.240 persuade you if you're not already of this state of mind that in a system that

00:20:09.360 has tests and application code the tests are the main thing and the application

00:20:16.400 code is secondary um because the application code is subordinate to the

00:20:21.760 test it answers to the tests the tests only answer to your arbitrary desires

00:20:29.080 um which is not the way that most people think about it it's not the way that I thought about it for most of my

00:20:34.480 programming career um but after I worked with one of my Consulting clients for a

00:20:39.960 while for example he's like you know I used to think of the application code as

00:20:45.679 being the main thing and the test as being something secondary but now I think of the the tests as the main

00:20:54.440 thing um and I want to hopefully you buy this

00:20:59.919 statement also um if your tests are then your application code is

00:21:05.159 too uh I think that is necessarily true I'll also say I people usually find

00:21:13.120 this statement extremely controversial or so self-evident that it's not even worth stating that you can't have good

00:21:20.720 code without tests um does anybody not buy

00:21:26.840 that everybody buys that sorry everyone everyone

00:21:35.679 okay I work with people who will have a problem with that yeah yeah I and I

00:21:42.360 actually have interesting conversations there's it's interesting how some people

00:21:48.600 approach development in a completely different way that I ever could imagine

00:21:53.799 for example using like a dropping a the buger and like

00:21:59.640 looking around and developing your code as you are in the okay and then taking what you so sense you are kind

00:22:07.279 of but not right yeah I mean all code gets

00:22:15.000 tested one way or another yeah um okay so everybody here no nobody raised their

00:22:22.120 hand that they don't buy that statement I don't believe everybody yeah I just

00:22:28.039 feel it's and T signal mat so if I'm something my feedb is my

00:22:35.799 it's test so if you want to drive velocity unfortunately I've been I've

00:22:41.840 not WR a test because I'm like I don't know if this is going to be a strong signal of our customers and not spendo

00:22:47.919 much time on T but I actually don't like that so that's that's where I come from interesting I I've never been in a test

00:22:55.400 where maintaining it was worth can you please leave

00:23:03.480 um okay so for for any silent denters in the audience and in case you dear viewer

00:23:11.720 um don't buy my statement I have a rock solid um uh thing to to share with

00:23:21.080 you okay so my rock solid premise number one is you can't have good code without

00:23:27.640 periodic non-trivial refactoring I think that's pretty hard to argue against and

00:23:34.320 my second premise is that you can't do non-trivial refactoring without

00:23:40.320 tests and so if you buy both of those premises then what necessarily logically

00:23:47.919 follows is that you can't have good code without tests so if you disagree with me you're

00:23:55.000 just wrong

00:24:01.000 okay so if we want to have understandable tests there's a lot that goes into that but again first we need

00:24:08.679 to go down to the very fundamentals and we need to understand what tests are and

00:24:15.440 what they're for um so we've discussed it a little bit already but what is a

00:24:23.240 test spefication yes that's yes

00:24:29.000 um and what are tests for what's the purpose of

00:24:34.039 tests proof proof wait was that you who said that

00:24:41.480 the test okay that was you

00:24:48.559 okay okay okay does anybody have any additional answers documentation

00:24:56.600 documentation foot gun vention okay defect

00:25:03.440 prevention yeah protecting against regressions is maybe yeah shipping B production yeah

00:25:10.559 okay okay so here's here's what I came up with which I think is consistent with

00:25:15.840 all that um okay so I think that a test has two

00:25:26.399 purposes one is to describe the behavior of a system so you said

00:25:31.440 documentation I think this is basically a different way of saying the same thing um so it's like there's two audiences

00:25:38.640 for a test one is the human reader and the other is the system so it a test

00:25:44.679 exists to describe the behavior of a system and to enforce the described

00:25:54.279 behavior um and Bob Martin said this really interesting thing tests are about

00:25:59.760 specification not verification and this is a really subtle

00:26:06.600 but also really important distinction in fact the first sentence

00:26:13.799 of the first chapter of my book talks about the difference between specification and verification and I say

00:26:20.399 that if you get nothing else out of the book that distinction between specification and verification is the

00:26:26.440 one thing that I hope you get out of it unfortunately it's a really hard thing to get

00:26:32.960 across um um and we'll talk a bit more about this

00:26:41.360 concept as we go via VIA some examples um what makes I'm I'm gonna go back for

00:26:49.080 a second what makes a test understandable I'm going to pause for a

00:26:54.960 second because I feel like we've been going a little too fast for a second okay I'm caught up um what makes a test

00:27:06.760 understandable yeah a clear mapping to the implementation a clear mapping to

00:27:13.159 the implementation what do you mean by that it's not like convoluted you can just look like oh this specification the

00:27:20.200 first name is to be uppercase so you should be able to look at the same file

00:27:25.559 and see it's right there and find you should be able to find this ification very easily next to the impation and I

00:27:32.000 have to dig around for it yeah um this is hard to articulate it's like

00:27:38.360 the the test should be a good explanation of the of the behavior that

00:27:44.240 it describes but it's hard to talk about it without being kind of circular yeah I think it's just the two

00:27:51.240 need to be very you need to look at both and see them ideally in the same close to each

00:27:57.840 other so you can intu it what's going on yes you intu the implementation and

00:28:04.000 specification so okay I'm going to go to my my answers to this question does anybody have anything to add before we

00:28:10.320 go to the next slide yeah feto I thing need to fists uh are more understandable

00:28:16.440 when they're more concrete and simple yeah okay so feto said something really

00:28:21.519 important tests are easier to understand when they're concrete okay and I'm going to say

00:28:27.919 something now about specification versus verification I use an an analogy in the

00:28:33.399 book that I don't completely love but it's the least worst analogy I've come up with so far let's say you're visiting

00:28:38.919 a squirt gun Factory um and you want to make assertions about the design of

00:28:46.159 the Machinery that manufactures the squirt guns one thing you can do is you

00:28:51.240 can have a verification mindset and you can go to the factory line and pick up a

00:28:56.480 finished squirt gun and ask does this squirt gun work and you can test that

00:29:02.080 squirt gun and and say does this particular squirt gun that I'm holding in my hand

00:29:07.320 work or you can do the same exact thing grab a squirt gun but

00:29:13.880 ask is the squirt gun manufacturing Machinery working as

00:29:20.679 specified and even though you're using the same object in the second scenario

00:29:26.240 you're only using the squirt gun as a proxy to make assertions about the

00:29:32.480 Machinery so the difference is about your your target that you have in mind

00:29:37.840 when you're performing the tests this is why it's like a subtle difficult thing to to get across um but it's again the

00:29:45.320 Target that you have in mind

00:29:50.519 um okay so I think what makes a test understandable is first of all if you're

00:29:55.919 expressing it in terms of specification rather than verification it's easier to understand

00:30:01.919 if it's meaningful and what I mean by meaningful is if it targets ends rather

00:30:09.320 than means you guys have probably seen tests

00:30:15.240 that assert for the presence of active record

00:30:20.480 associations yeah useless thank you um why is it useless because it's a

00:30:27.960 configuration check I can just look at the model and see if there already it's it's like I believe rail should work I

00:30:36.640 don't need to test that I'm using rails correctly then you know I don't even

00:30:41.679 think that's a necessary part of the argument um because it has nothing to do with believing that rails should work um

00:30:49.240 but it's a common argument so I think it's important to address address that one it's not about believing that rails

00:30:54.679 should work if rails were to not work what would happen happen the behavior that that

00:31:01.039 Association enables would fail so you don't even need to believe real is going

00:31:06.480 to work or not because again if it doesn't work then your higher level

00:31:12.120 Behavior will will fail to work and if you have a higher level test then your test will

00:31:17.559 fail um so when you have tests for associations your testing the means to

00:31:22.880 an end not the end itself and it's redundant

00:31:28.440 because if you're if you have a test for the behavior that that Association

00:31:33.919 enables then again if that Association were to break then your higher level

00:31:39.240 test would fail so it's better to Target inss rather than means um and ins tend

00:31:46.799 to be uh at a higher level of abstraction and easier to understand

00:31:52.880 because it's like you test an association for example and it's like okay but like what does that mean that this Association like what does that get

00:31:59.120 me you don't know um but if you're testing the in feature then it's like oh

00:32:04.840 okay this is uh uh we need to be able to create a new customer record or something like

00:32:12.360 that I think tests are easier to understand when they're at the appropriate level of

00:32:19.039 abstraction and almost nobody ever does

00:32:24.159 this um when you open up most tests you're just presented with a giant wall

00:32:32.120 of details and the writer of the test is

00:32:37.799 leaving it up to the reader to look at all the details and think about all the

00:32:43.760 details and do the job of figuring out from the details what's the higher level

00:32:51.360 meaning and that is is wasteful because maybe you saved some

00:32:58.039 time in writing the test but now when people read it it's going to incur a cost every time

00:33:04.720 somebody has to read that test um and so it'll be much cheaper if you take the

00:33:11.159 work to add a layer of abstraction over your test so that you don't have to see

00:33:17.240 a whole bunch of details and figure out the meaning instead you can just see the meaning

00:33:22.840 directly um and then you don't have to think as much

00:33:27.960 the fourth thing here is cohesion uh what do you guys think I mean by cohesion in

00:33:36.200 tests similar Styles similar styles that could

00:33:41.960 apply what else what is cohesion

00:33:48.200 anyway holding together the way I Define cohesion is

00:33:55.120 grouping like with like if you look at a file and everything in that file is

00:34:01.360 about the same thing um then it has cohesion I think Ken Beck said uh

00:34:08.079 cohesion is when the same things in the file are tightly coupled with one

00:34:13.200 another in in a way that is appropriate for them to be tightly coupled he worded

00:34:18.480 it much more eloquently than that but it was that was the idea

00:34:24.560 okay how am I doing on time

00:34:30.320 um verification so here's an example of like a verification mindset versus a

00:34:35.960 specification mindset in the vast majority of the test Suites I look at the tests are expressed

00:34:43.000 in this first way it does X it does y correctly it Returns the right

00:34:49.520 z um it makes me extremely sad when I look at a test and it has the word

00:34:55.639 correctly in it why do you think that makes me

00:35:01.119 sad any ideas

00:35:08.480 yeah that's a good point I didn't even think of that in case anybody missed it what Cameron said was shouldn't the

00:35:15.160 tests always be asserting correct Behavior it's kind of Superfluous to say

00:35:20.640 that it's correct yeah that's the question what is

00:35:26.079 correct exactly so 10 minutes 10 minutes okay I didn't

00:35:32.320 mean to talk for so long um um um correctly sorry what did you say it asks

00:35:38.800 the question what does correctly yeah when you look at a test and it says it does such and such

00:35:44.880 correctly it's like okay but what does correctly mean so don't put the word correctly in your test instead say what

00:35:52.040 correctly means what's so funny fedo that I've done

00:35:59.119 terms of right so what I think is a better format for test is like under scenario a it does this under scenario B

00:36:07.960 it does this um and the other good effect of that is like sometimes when

00:36:13.800 you look at a test and it's like it does this it's like okay but like what's the

00:36:19.839 significance of that but if you say in this scenario it does this and you show an alternate scenario or like the

00:36:26.200 inverse scenario where it does this other thing it's like oh okay so like when this is the case it does this but

00:36:31.440 in this other situation it does this you can piece together the meaning more easily uh when it's like that okay we

00:36:39.880 already talked about some of the stuff I do want to share my definition of

00:36:45.680 abstraction because I think abstraction is a really important idea to me it's

00:36:50.920 the act of replacing lower level ideas with higher level ideas in order to make

00:36:57.000 a particular aspect of the world easier to understand thank

00:37:02.200 you I based on our uh short time together I value your opinion so I appreciate your thumbs

00:37:09.640 up um and by the way there's abstractions outside of programming I think we can

00:37:15.680 think about those and learn from those but we'll have to skip that for now cohesion is grouping like with

00:37:22.359 like okay so my talk is almost done here are kind of the things that I want to

00:37:27.920 leave you with um in order to write tests that are easy

00:37:35.359 to understand again I think it's it's really helpful to go down to the

00:37:40.520 fundamentals and give yourself a firm foundation of what are these basic

00:37:45.720 things what is a test what is application code what's the purpose of a

00:37:51.160 test and I challenge you to put these things as concisely as you possibly can

00:37:58.520 and commit the definitions to memory because I think you know there's so many

00:38:03.560 things in programming like words we throw around all the time test abstraction cohesion blah blah blah and

00:38:10.280 if we can't uh recite the definition of these ideas off the top of our head uh

00:38:17.720 then I I think it could be said that we don't we don't understand them as well as we could so I think it's a really

00:38:24.359 good idea to to do that exercise and these four things that make tests

00:38:30.640 easier to understand specification rather than verification meaningfulness remember

00:38:37.920 targeting ins not means abstraction replacing lower level details with higher level ideas in order

00:38:46.000 to make things easier to understand and cohesion um grouping lake with

00:38:52.079 Lake okay we already mentioned cinity Ruby but I want to tell you about it

00:38:57.880 it's either the 10th or 11th or the 11th and 12th I someday I'll uh I'll get it

00:39:04.000 right but it's on the website sincity ruby.com the dates on the website are correct uh 100 attendees maximum it's a

00:39:12.560 very small conference it's it's very in Intimate we spend two days together in

00:39:18.000 Las Vegas um and we have some good speakers again Arena and fto are speaking there

00:39:25.240 Chris Oliver from from go rails if you if you know Chris um and I'm not going

00:39:31.119 to read the whole list to you but but there's the rest of the speakers um

00:39:36.960 brief word about my Consulting uh to give a a quick taste one of the things that I often do is I'll work with a team

00:39:44.880 and we will have maybe a weekly or even daily session where we talk about this kind of stuff and I I challenge the team

00:39:54.079 with questions like what is the point of testing what is abstraction all these

00:39:59.680 kinds of things testing oop um project planning just like all

00:40:05.760 kinds of stuff and and we we think about these things and how to be more

00:40:11.280 efficient and productive and stuff like that again my uh monthly snail mail

00:40:20.200 programming newsletter can be found at nonsens monthly.com and I'm going to pass out some copies of this letter

00:40:26.040 tonight this is my book professional rails testing I brought a whole bunch of

00:40:32.760 copies with me but they're in my hotel room um are we going to like a drink

00:40:41.359 place a bar or something like that after maybe I'll go back and get them and give them to you yeah and here's all that

00:40:48.960 information in one place and that concludes my talk all

00:40:55.720 right uh thanks Arena

00:41:01.920 um have a clicker but I'll just I'll just use the key um yeah so tonight I'm

00:41:09.760 going to talk about um ask key authentication in Ruby on Rails I'll talk about very briefly what paskis are

00:41:17.839 um why or why why or why not you might want to use them um and then how and 20

00:41:25.560 minutes is not enough time to implement keys so I might be going through the code very quickly um I'll share the

00:41:31.480 slides and Sample code um so just heads

00:41:36.520 up um my name is Bart agapan I'm a senior software engineer at IDM um I'll

00:41:42.200 start we are Ruby on Rails shop and we are hiring so this QR code is for our

00:41:48.280 jobs board if anyone's looking um uh I've been a Ruby on Rails

00:41:54.040 developer since 2007 although really only ser seriously since 2015 um when I started doing it

00:42:01.040 professionally um so I love Ruby love Ru um this is my first uh SF Ruby Meetup

00:42:07.200 talk so thanks um and a quick disclaimer this is

00:42:15.480 a security and pasis talk I am not a security and pasis expert just an

00:42:21.359 Enthusiast so uh before you put any of this into production check with your security team

00:42:28.559 um so this is what I want to talk about in the brief amount of time what are pass Keys what are the pros and cons and

00:42:34.440 then finally how to add pass keys and I'll have a little um a demo at the

00:42:40.079 end so what are pass Keys um also known as web auen or web authentication it's a

00:42:47.040 uh worldwide Web Consortium standard for Secure Public key authentication for web

00:42:53.599 apps web web uh websites web apps um they claim to be more secure than

00:43:00.559 passwords and onetime passwords uh because they can't be fished um they are

00:43:06.440 they use strong cryptography um and they can be used for several

00:43:12.760 uh uh ways to keep people out of of accounts but uh getting into your

00:43:18.240 account it's can be used for two Factor mult uh two-factor authentication it can

00:43:23.599 be used for passwordless login and it can even be used for username list Lin B basically nearly fully Anonymous logins

00:43:31.839 but I'm not going to really talk too much about that um and then compared to existing

00:43:37.440 security methods these are some popular ones you may have seen uh username and password to factor off using uh

00:43:45.280 authenticators SMS email even phone calls my bank does that um and then

00:43:51.200 magic links and then just kind of comparing to pass Keys um T is often

00:43:57.920 fishable so somebody could call you up and and have a code sent to you and have

00:44:03.359 it read over the phone um many reasons that pass keys are

00:44:08.680 better um and I'm not I won't go through all this so list but um just take my word

00:44:15.680 for it um so pass Keys deal with

00:44:20.920 uh public key cryptography the private key never leaves your device there's a little asteris there that there are

00:44:27.839 cases where it does but um in a hopefully safe way it requires uh an

00:44:35.079 authorization gesture is what the standard calls it where user has to opt into using it um you can't write a pass

00:44:42.599 key on a post it and stick it on your monitor so it's already going to be more uh secure for some

00:44:49.200 people so obviously sounds like you should use them right

00:44:55.079 uh unfortunately there's been a lot of press recently about how pass Keys uh

00:45:01.359 aren't all that great so it depends um pass Keys it means you need to make a choice between usability and security

00:45:10.440 talk to your local staff engineer um so here are some recent

00:45:17.119 articles you may have seen these are just in the past few months um and of course dhh himself

00:45:24.680 wrote a blog post about how uh pass Keys maybe aren't aren't the best

00:45:29.920 so M Ruby Meetup Ruby on Rails Meetup talking about pass Keys hope this works

00:45:36.440 out um so actually I don't know how important it is to talk about this but

00:45:42.520 there are two types of pass keys of authenticators there's a platform authenticator and a roaming

00:45:47.839 authenticator platform authenticator is something like one password or your Apple echain uh roaming authenticators

00:45:55.040 like a UB key um your users are probably not going to

00:46:00.160 know the difference or that there these are even both pass Keys um and so that

00:46:05.640 can add some confusion that that you need to know when you're implementing this the um make sure that you're not

00:46:12.160 going to lock your users out or that um people will be able to use it when they expect to be able to use it uh I won't

00:46:19.079 be able to go into too much detail on that but all those articles that I just posted um go into that in some detail so

00:46:25.559 I'll share links to those if you're if you're interested um because things like this

00:46:30.720 can happen you can lose your UB key uh you can try to you can set up a pass key

00:46:35.880 on One OS and then try to use it on another and you may not be able to use it depending on what kind of key you set

00:46:41.480 it up with um you might delete your pass key um or through no fault of your own

00:46:48.160 maybe it just disappears somehow that happens to some people um and if that happens you may not be able to access

00:46:56.040 your account you need a fallback and so uh that's where the convenience versus

00:47:02.160 security trade-offs have to be made um so these are my non-expert

00:47:08.839 recommendations uh if you want to use pass keys for MFA you should probably have at least two forms of MFA either

00:47:16.319 two paskis or paskis and a authenticator or another um

00:47:21.599 two-factor um setup um or you're going to need to have a customer service

00:47:27.640 uh rep who can go and reset this stuff so that can be a cost um for

00:47:33.359 passwordless login you probably want to also have a password in case it doesn't work and then for username is

00:47:39.680 login like I said I wasn't going to talk about this but there may be are cases where it's okay to use it you don't care

00:47:46.000 about the the data so much or you have another way of doing account

00:47:51.520 recovery um so when you're deciding whether to use pass Keys just make sure you balance your user friendliness

00:47:58.599 because these are pretty user friendly if if used correctly um with security um

00:48:04.599 because making it more user SEC making it more user friendly can actually make your security go all the security uh

00:48:12.800 gains you made by using Pas keys can go away to make it user friendly and and recoverable and I'm not going to read

00:48:18.960 this whole list but um there's lots of things to consider despite that I'm still going to

00:48:25.480 tell you how to add p these derails um and maybe actually before I show you

00:48:30.640 that I'll just do the demo because I think this will make more sense when you

00:48:36.119 uh let you see how it works so I have written a small demo app

00:48:44.520 yeah all right yeah you're sure yeah um so the way this works is it's

00:48:52.760 just a signin and a you know a user settings page so I've written I have one account on

00:49:00.839 here and uh you'll notice I just have a username here not a password uh form so

00:49:07.040 this is a little different but what that oh shoot of course hold

00:49:14.319 on gotta run the

00:49:23.880 server yes this is actual running code right here here all right let's try this

00:49:37.520 again and of course I'm running it on a non-standard

00:49:45.040 port

00:49:53.079 nope okay all right never mind

00:49:58.799 my the server is

00:50:04.119 crashing I developed I ran it in Dev containers but now I'm not running Dev what did the demo do okay the demo well

00:50:10.680 I'll walk through the I'll keep going through here so the demo basically let you add pass keys to

00:50:17.119 your account um it showed you the UI for adding pass keys and then um logged in

00:50:22.960 with pass keys with passwordless login maybe I'll be able to get it working at the end um so I have a sample

00:50:31.799 app uh this is the the URL you can go check it out I'll also have a QR code at the end um this works with any recent

00:50:39.119 version of rails um this this Dem this uh setup example will be with JavaScript

00:50:45.640 using import Maps uh stimulus and uh in this case it was just the like rails

00:50:52.640 generate authentication setup um um so it works with

00:50:59.040 that um so it has two main dependencies a gem called webn and uh a JavaScript package called

00:51:07.599 webo and Json and so this is how you add them and then you'll need to add a pass

00:51:14.960 key model and connect it to your user um so a user can have many pass Keys um

00:51:24.359 and then this last piece is adding a pass key ID to this to your user model

00:51:29.839 that's actually optional there's other ways to do it but all of the um online examples that I found use that and it

00:51:35.400 makes just makes it a little easier um so this is the flow when you're creating a pass key for a user

00:51:42.640 you create on your server you create a challenge you send that challenge to the

00:51:47.680 browser excuse me the browser uh makes a API

00:51:53.559 call the JavaScript makes a call to browser to say click your authenticator or click

00:52:01.160 the button to uh create a pass key the authenticator will create a key pair at

00:52:06.799 that point and it'll be tied to the domain of the site that's um setting up the pass key and it will take the

00:52:14.559 challenge that the server sends and it will sign it using its private key and send it back to the server and the

00:52:21.200 server just verifies that the public key that is sent with this Challenge and the signed

00:52:27.520 um uh challenge match and if they do then they say okay good this is this

00:52:33.240 this authentication this authenticator matches uh or the the authenticator has

00:52:39.359 the private key that I think it does and then it saves that pass key to the user

00:52:45.440 um that's a kind of a simplified way of what it's doing but um generally generally speaking that's how it works

00:52:51.200 um so in order to do that you have to add a pass Keys controller um this is the code to do

00:52:57.160 that and then some routes you need to have uh a challenge route and then

00:53:02.520 there's the normal uh create and Destroy um so I'm not going to read

00:53:07.599 through everything here but the important stuff is um you have a form

00:53:12.839 that posts to your challenge URL um there's some stimulus controllers here

00:53:18.040 that are going to um kind of uh take the

00:53:23.640 data that comes back from that and process it a little bit more um and then send it to a third URL and

00:53:30.480 I'll show you the stimulus code for that um so this form just has a nickname for the authenticators that's just so the

00:53:37.720 user knows am I you know they can give a name like oh I'm using my UB key or I'm using one password they know which one

00:53:44.640 uh has been set up in case they ever want to delete it um this is the challenge code in the

00:53:52.799 uh the controller so this um again you don't need to look and everything but this calls the web aen uh Library it

00:54:01.200 generates a user ID that's unique and it generates options that are going to be sent to the the JavaScript front end um

00:54:10.319 and that's how uh that's how the authenticator knows um who this is for

00:54:16.400 what user this is for and what uh what domain um and you can see that it

00:54:23.079 excludes uh it has this exclude Clause there which basically says um don't

00:54:29.000 include any pass keys that are already associated with the user uh and then it stores it in the session uh so that it

00:54:35.760 can compare it later um this is the JavaScript this is a snippet of the JavaScript where it

00:54:42.680 creates it calls that web auen Json create that call is what tells the browser to ask for the user to you know

00:54:50.680 tap tap their key and then it posts that to the Callback URL which is also on the

00:54:57.400 which is the create endpoint on the pass key controller um sending the payload that the authenticator sent to

00:55:04.440 it um and then finally so this is the this is the endpoint that collects it it takes that um that payload it's feeds it

00:55:12.599 to the web offend credential from create method um and then it Compares what it

00:55:19.720 got with uh what's in the session so um it tries to pull

00:55:27.039 uh it creates a new Pass key on the user with the data that came back um and then

00:55:32.880 it verifies that that uh or this Pass Key credential verify is where it checks

00:55:38.760 that uh the challenge that it's sent matches the data that came back uh from

00:55:44.400 the authenticator and then it saves it um and then to log in that was it for

00:55:52.000 for that once it saves the pass key that pass key is now associated with that user in your in your system um to log in it's a very similar uh

00:56:01.000 process you create an you create a challenge in this case it's for authentication the authenticator already

00:56:07.640 has uh the same uh private key that's tied to the

00:56:13.559 domain that you're trying to authenticate to uh and sends back basically signs that challenge and sends

00:56:19.960 it back and then the your system will verify that the challenge matches one of the stored pass keys and because it can

00:56:26.680 be any of those pass keys that were sent before so in this case uh you probably already have a session controller if you

00:56:33.559 been using device um or if you use the the rails authentication creates a session

00:56:38.920 controller you need to add these two endpoints Challenge and

00:56:44.000 verify um in your url you add a uh a button to and not I'm kind of glossing

00:56:51.760 over the UI changes there there's a lot you might have to do to actually add the you know uh sign in with Pass key that

00:56:58.599 collection of the username first um whether you're going to do password list or whether you're going to do MFA or

00:57:06.240 even usern nameless um those are all kind of UI decisions this uh this

00:57:11.960 example is for um this could be used for either uh TFA or

00:57:18.240 passwordless um so this is just a button that's going to submit to that's going to post to

00:57:24.680 this challenge URL um it's going to send an email address that was previously

00:57:30.200 entered um and then there's going to be a login uh stimulus controller that's

00:57:36.000 going to intercept the data and then send it to the verify

00:57:41.079 URL um so this is what the challenge looks like you just assuming that you've typed in an email address already it's

00:57:47.520 going to load that User it's going to create a web aend credential

00:57:53.440 um it's going to say and this is a choice for you but in this case this

00:57:58.520 says any of the users pass keys are good for um logging in here you can if you

00:58:03.680 wanted to like restrict it to a UB key or restrict it to a certain Pass Key you

00:58:10.760 could do that um again you store the pass key

00:58:17.200 challenge into your sessions you can compare it later um and then we'll

00:58:23.680 scroll up uh this is later on in the same method

00:58:28.960 um you yes so you store the challenge in the session and then you're going to

00:58:34.200 send back those um that challenge to the to the front end this login controller is going to

00:58:41.920 intercept that um it's going to call that web Json in web off and Json before

00:58:47.760 it called create in this case it's calling get and get just means tell the user that I want them

00:58:53.680 to tap their key um and then it will send that data as Json to the Callback

00:59:00.720 endpoint that's the verify endpoint uh verify

00:59:06.000 action this will create a credential from the parameters that were sent by the authenticator it's going to load the

00:59:12.200 user from the session and then it's going to check all of the pass keys that the user has do any of them match the ID

00:59:19.559 of this uh credential that was just sent back and then um this verify call

00:59:26.599 uh is is where it checks if they match and if it does then it will update um

00:59:33.640 and log the user in um and what's not shown here uh I

00:59:39.079 guess you can kind of see it on the bottom any of these steps fail it just raises an exception so um your exception

00:59:45.000 handling is how you handle failed authentications um and at the end you also have to make sure you clear your

00:59:51.599 session um I tried the demo already I don't think it's going to work if I try

00:59:58.039 again because I'm missing uh actually you know what I'm just gonna try one

01:00:03.760 more time I'm going to try it with my Dev

01:00:13.680 container let's try this one more

01:00:23.559 time I close it no

01:00:34.039 down I'm going to start this up in a known

01:00:42.559 good

01:00:51.559 environment all right

01:00:58.000 okay hopefully this works

01:01:06.079 s uh Dev containers for the win um so I'm going to sign in with a user that

01:01:12.520 only has a username and password and not any um no pass Keys associated with so

01:01:19.160 when there's no pass keys this UI shows a password field I have a very secure password

01:01:26.680 that Chrome is going to tell me is not secure so this is my user I have no pass Keys associated with it and I'm going to

01:01:33.880 just add one I'm gonna add one password hopefully it's set

01:01:40.240 up and so yeah so you can see it I've

01:01:46.559 got um an offer to save a pass key so I'm just going to do

01:01:53.520 it it's saved and then you can see here now I have a I spelled one password

01:02:00.079 wrong but um so that was the nickname that I gave to this one and um I just

01:02:06.079 added this as a just a as an example but this is the the type that the

01:02:12.640 authenticator returned it says this is a platform authenticator and then I'll add a UB

01:02:18.240 key just for fun this in and then in order to do this and this

01:02:25.359 is one of the things it's a little confusing one password you have to click this and then you get another one and I

01:02:32.119 think here you can just hit the key doesn't say you can but that's how it

01:02:37.440 works so you can see this is a crossplatform pass key um so it's a

01:02:43.480 little different but now now my account has pass keys so I can log in with

01:02:48.920 it one password's nice and helpful here and in this case now it says sign in

01:02:54.160 with Pass key doesn't have the password anymore and now um that pass uh web auen get was

01:03:00.200 called and now I can just click the sign in button and I'm I in and that works

01:03:05.520 both with uh one password and with a physical

01:03:12.640 key yeah and that's

01:03:22.400 it do you have a question yeah have one password addon browser addon on

01:03:29.920 your operating system on the browser like would this not work is that two is

01:03:34.960 the private public key signing process something that's native to Chrome so

01:03:40.599 Chrome has a built-in Pass Key um implementation I believe uh Apple has

01:03:47.440 one built in so Safari can do it um I'm not sure I think Android has one

01:03:54.200 but I'm not sure about like fire Fox if there's a if there's a native one um but

01:03:59.240 there are lots of um like one password is not bit Ward and we'll do it I believe and um some

01:04:04.760 other

01:04:10.200 password yeah this is one of the problems with pass Keys is like not everybody knows

01:04:15.400 what's going to work and what browser it's going to work on and um it's it's not standardized yet so it may get

01:04:21.720 better question One browser sometimes the site we'll say I have pass Keys

01:04:27.640 and'll use Touch ID which is great and then other times it throws up QR code which is like no yeah what what

01:04:35.240 technically is going on in the implementation of the server that's doing that why why is there that difference the same so I think that's

01:04:41.799 not a server thing I think that's a browser thing um and in some cases like

01:04:48.440 you saw one password wants to be the authenticator but it allows you to fall back to using another one

01:04:55.960 sometimes I've had the apple one will jump in front and I can't use one password because it got there first so

01:05:01.799 it seems like they don't play nicely and that's part of what the problem

01:05:07.720 is can does handle this

01:05:13.720 natively uh I don't know it does use it oh okay

01:05:20.599 yes I thought yeah so none of this code was this didn't use any like special

01:05:27.520 software didn't use devis um it just used that webo and Gem but yeah if rodol

01:05:32.559 has their own implementation then this does not this demo does not apply but the the way it works should should be

01:05:39.119 the same yeah I think you mentioned you could use pass keys for anonymous sign

01:05:46.000 in but in the in the demo you were looking at by user ID how do you find the right Pass Key when you're doing

01:05:53.640 Anonymous do these pass Keys have IDE yeah so every Pass Key has a unique ID

01:05:59.240 um so that that would be how every user Pass Key then if you were doing usern

01:06:05.319 nameless login then the pass key would be basically sending you the user ID along with it's um the the answer to the

01:06:14.680 challenge so okay Co yeah I think that was it so this is a a

01:06:22.559 link to the demo app that I just was running um there's also these slides are linked

01:06:28.720 from that GitHub page um yeah thank you so

01:06:39.680 much uh repeat speaker at Ruby which is the main credential also um a co-

01:06:46.640 container of new component um which apparently sucks

01:06:51.680 I and second yes nice thank you the

01:06:57.119 employee of GitHub that's that's what we guess from from you company for sure for

01:07:05.039 sure well so I was goingon to do a really quick like um open mic thing but

01:07:10.079 two people mentioned react and so now I feel like I shouldn't say this but I think it's funny this is a very

01:07:15.200 important website that I found it says should I use react for that and it just says

01:07:21.680 no we have our answer

01:07:26.920 all right so yeah let's see let me make sure I have my like speaker notes and

01:07:33.279 whatnot need do like the full or the I I guess that might not be possible

01:07:38.920 with the zoom huh that's fine we'll just make it work okay have speaker notes for the me

01:07:45.640 up I mean I just want to be able to see the next slide it's fine um okay so I'm gonna be talking today about um using WM

01:07:53.520 A wasm compiled module from Ruby and I'll explain why you might want to do

01:07:58.839 that or maybe not um the first thing though that I want to say is that I know that product

01:08:04.720 board has offices in Prague right and I wondered if people from Prague are called programmers and they do

01:08:11.240 programming oh thank prog grammar is that a thing I feel like that might be a thing no okay

01:08:18.960 well um also my code that I'm going to present has no tests and I think that that's probably Jason I think probably

01:08:26.719 thinks that it's by definition and so you might not want to listen to this I don't know it's up to

01:08:33.759 you all right just a spike exactly yes um I'm

01:08:38.960 Cameron D work for GitHub um here's my Social Links and um our goal today is going to

01:08:45.040 be to compile a C library into a wasm module and then we're going to call those C functions from Ruby and

01:08:52.799 hopefully at the end we'll have learned how WM works this is the whole reason that I am giving this talk is I learned

01:08:57.880 a bunch about how WM works and I wanted to share that with somebody else and so that's why this talk

01:09:03.480 exists so what C library are we going to be using so onigmo who knows what onigmo

01:09:09.679 is nobody knows okay that's that's fine somebody raise it no onigo is

01:09:17.560 Ruby's regular expression engine it's actually a separate project you can find it on GitHub under K takata's username

01:09:23.960 it's used by other projects too I think I want to say python uses it as well um

01:09:29.080 and it's built into Ruby so most of the time you wouldn't actually I mean all the time you would just use Ruby's regx engine you wouldn't have to reach for

01:09:34.960 this we're using it because it is a non-trivial code base right so it does

01:09:40.120 something cool and useful um I mean everybody uses reg X's kind of whether they want to or not and it's because of

01:09:46.040 that it's like a decent wasm showcase so I talked about why I'm I'm

01:09:51.279 giving this talk um I learned some things about WM and how to use it from Ruby why did that happen um well it

01:09:57.199 happened because in 2022 or so I started working on Garnet JS and I talked about that at this Meetup before um there's a

01:10:04.560 URL for it it's really it's a implementation of the Ruby virtual machine in typescript the idea was that

01:10:10.040 it could be run in the browser and it's so it's RI implementation turns out that's really hard so I haven't actually

01:10:16.679 contributed to it for a little bit um but back in the day I needed to bolt on

01:10:21.719 a regular expression engine because Ruby has one and so garnet needed one and I

01:10:27.080 didn't want to write one because I'm just not smart enough to write a reg X engine and so I also didn't want to use

01:10:33.560 JavaScript which also has a regx engine because it's not 100% compatible Ruby so if I were to run you know the Ruby test

01:10:40.400 Suite it would fail likely on some of those tests it's not the same thing but I learned what's yes that's

01:10:47.239 right there were tests in that one so that was not terrible code as it turns out um but I could compile enigmo to a

01:10:54.480 waza module and then use that in the browser and that's why this whole sort of talk came about it's because I

01:11:00.400 originally wrote the code that you're about to see in typescript and then ported it to R for this talk to sort of showcase how it works it works very

01:11:06.520 similarly in both in both languages so so what is WM and we've heard about it I think a couple times at

01:11:12.640 this Meetup um it stands for web assembly WM it's a 32bit binary format so it's

01:11:19.960 not 64-bit which is super interesting at least I thought it was it's all processors these days are 64-bit

01:11:25.920 um it's executed by a virtual machine just like Ruby code the cool thing about it though is that it's embedded in the browser or Le it can be uh most browsers

01:11:33.360 support this now it's stack based like Ruby is um it focus on this linear

01:11:38.440 memory model so there's just a giant chunk of memory that's like all contiguous and you can read and write from that unlike what an operating

01:11:44.199 system would do in most cases which is to like fragment memory all over the place um and it also even though it's a

01:11:49.719 virtual machine it has pretty decent performance so that's why there are whole websites built in

01:11:55.960 with or with WM so if you're familiar with figma figma's actually built in C++ compiled to WM then runs in your

01:12:02.719 browser there's also a bunch of different like implementations and runtimes of wasm wasmer is a popular one

01:12:09.000 the one that I'm going to talk about today is called wasm time it has bindings to Ruby um WM 3 WME tiny WM is

01:12:14.960 an embedded version a lot of different implementations

01:12:20.360 yeah why would you use WM well it sort of offers the same benefits that Java

01:12:26.159 was supposed to offer right um so it compile it once you can run it anywhere it's got applications sort of you know

01:12:33.280 everywhere you can imagine desktop mobile embedded systems via tiny WM other things like that uh and of course

01:12:39.120 web browsers that's one of the big sort of reasons that I'm excited about why I'm you run code written in other

01:12:44.679 languages in your browser and that's I think we're just talking about stack blit stack blits does this a lot it's my understanding very cool so many

01:12:52.120 languages compile to them C and C++ sort of being two of maybe the the most um I

01:12:57.719 don't to say exciting two of the most sort of common ones that you might compile to to WM um so like people have

01:13:03.440 gotten postgress running in their browsers they've gotten FFM Peg running in their browser right all because it's

01:13:08.480 all written C can compile it to WM run your browser um but also go and rust and

01:13:13.520 Zig and even some of the net languages compiled to WM um Lua and even

01:13:19.400 typescript via this other project called assembly script yep and as put him on there

01:13:25.960 because you know poor Java really tried to make the whole compile once for everywhere thing happen so I thought

01:13:31.199 Duke deserved a little bit of a shout out there why not ffi why don't we just use

01:13:36.400 ffi to talk to C libraries well you can ffi stands for a foreign function interface and the idea is that it's uh a

01:13:43.239 lot of your the sort of the gems you know and love in the Ruby ecosystem rap there the rap C library is in ffi I make

01:13:49.239 calls to these C functions um and this is mostly enabled because Ruby itself is a c program so it can run

01:13:55.760 um you know code as well like link against code and c and and call those functions but in order to make this work

01:14:01.239 you need a c compiler if You' ever had to like install gems for rails app and it installs Noco giri and it's like building gem native extensions right

01:14:07.760 like it's having to compile lib XML behind the scenes for you and to do that it needs to have a seat compiler it

01:14:13.760 needs to have um you know that that code needs to be um you know written to work on your architecture and most most of

01:14:19.760 the time it works sometimes it doesn't issues with this um at least I have in the past like um it's potentially slow

01:14:28.320 and notoriously error prone sometimes you compiler errors make no

01:14:34.280 sense I think somebody joined and is talking Tanya LV

01:14:44.880 hello it's all good um and also the have this issue where like B you could provide you know like

01:14:52.239 pre-compiled binaries for um lib XML and for and that's certainly been done before um uh but that also means you

01:14:59.320 have to maintain that then so like every time that um flavor Jones releases a new version of XML or of um of noiri he has

01:15:06.400 to make sure that he also provides these binaries for each platform that somebody might be using Darwin Mac OS right um

01:15:13.320 Linux Windows all of those and you might not have access to a machine that can do that maybe you have a CI system that can

01:15:18.760 do that but those cost money right so the idea being that like having a binary per architecture and per OS can be

01:15:24.320 really hard and difficult to maintain right WM binaries on the other hand are

01:15:29.679 portable you compile them once you can pass them around they'll run on any system as long as you have a WM virtual

01:15:34.840 machine to run them on right so it's the same idea of java Java has a virtual machine installed on Mac or on Windows

01:15:40.960 you just download that and install it and then you can run your Java code same thing with WM um one of the big

01:15:46.120 differences of course being that WM can run again in your browser so let's look at uh this is this

01:15:51.880 is directly taken from the onigo Repository examples folder so it's a

01:15:57.239 little bit of C code and the only reason we're looking at C code is to understand the shape of how this Library works so

01:16:03.080 we have a regular expression object on N regx it's a pointer I mean it's a Memory address we have a pattern here don't

01:16:09.280 worry too much about all of the very like complicated like you know syntax of C here but the idea is we have a pattern

01:16:15.480 we have a string that we want to match with that pattern we call this onig new function pass in some parameters to it

01:16:23.159 um and that because we're passing a lot of the stuff by reference this is initializing that regular expression and compiling this this pattern that you

01:16:29.719 we're passing in and then to do a match we create a new region um we have some um pointers to

01:16:36.840 the the string in here and we pass those into this function called onig search and that returns to us an INT and that

01:16:42.920 int can be less than zero if it's less than zero that means an error condition if it's greater than equal to zero that means a normal

01:16:49.520 exit and so we can say okay what it also gives me the index at which the match occurred so I can print f printf means

01:16:56.320 like puts like puts this to the screen matched at position R right now and we can also Loop through all of the um the

01:17:03.840 region information that was captured here so there might be multiple matches right there's one sort of beginning match for the whole thing for the whole

01:17:10.040 match and there's different regions within that if there are capture groups which there are in this case uh and you can Loop through those with this numb

01:17:16.199 regs this region object now if that if it didn't work then we could also print

01:17:21.239 out an error message and then at the end because this is C we have to clean up after ourselves free the region and free

01:17:27.520 the regular expression okay so the general shape of this is we have a pattern the string we

01:17:33.480 call onig new get a region with onig new region new and then onig search and then we're

01:17:38.960 done so if we were to imagine a ruby API for this let's actually maybe say we do

01:17:45.000 write a test for this um we're going to do the same thing

01:17:50.159 right we have an a reg X object onigo onig regs regex sorry do compile we

01:17:56.400 match against a string and then we assert that what it returned to us is the correct information we got captures

01:18:01.639 and begin and end and match things like that so the idea would be at the end of this talk we could run this test and it

01:18:07.159 would pass so which functions do we need from onigmo we need onig new and actually

01:18:14.560 we're going to use onig new Deluxe because why not um onig region new a bunch of them and then also at the

01:18:19.960 bottom Malik and free and we'll talk about Malik and free in a minute those are lib C functions we need to use we

01:18:26.760 also need a couple of constants so onigo is really nice about like giving us all this cool encoding support so um we're

01:18:33.920 just going to use utf16 because that's what JavaScript used that's what this is ported from um we also need onig as key

01:18:41.080 or encoding ask key which is for the error messages that it returns to us um and then we're also going to do this case fold flag not important what that

01:18:47.040 is and syntax Ruby is just the flavor of enigma's uh the flavor of of of regex

01:18:52.520 we're going to use with it is the Ruby flavor so how do we compile onigmo to WM well

01:18:59.320 it's it's not that hard I I'll I have to say I really struggled with this because I'm not a c programmer I have not used

01:19:04.679 the C tool chain that much um but trying to to get this to work I there's a bunch of sort of prior art like cobbled

01:19:10.280 together and figure this out um so first thing we need to do is install the SDK a couple of other tools um and then we're

01:19:16.520 going to use clang or cang to compile using Target wazin 32 wazzy um wazzy by

01:19:21.800 the way stands for the web assembly system interface and it's the thing that let let you talk to IO so you can read files talk to the to standard in

01:19:28.600 standard out using wazzy we're going to link so after we we compiled a bunch of object files we'll

01:19:34.440 link them together with w and LD and then we can trim off some of the extra stuff using a couple of strip

01:19:40.840 tools we can do all this in a Docker container for portability okay so here's the clang

01:19:47.480 command and it's there's just some highlights here don't need to commit this whole thing to memory um this is

01:19:53.239 the build Target I mentioned before so instead of saying Target equals like x86 64 Linux or something you're saying WM

01:19:59.120 32 wazzy this is the CIS route this is where we find libc the optimization

01:20:05.360 level we're saying I want to build against libc this thing should have libc built into like it needs these these functions to to operate the lipy

01:20:12.080 provides bun of include paths so you know we're providing it a path to the current directory a path to the WM

01:20:18.480 directory which is where um actually I'm not sure why that's in there I'll have to think about that later um encoding directories in this

01:20:25.920 directory which is the where all the encoding support is here's all the C files we want to compile so once we run this code this this this command rather

01:20:32.800 we should get a bunch of object files. o files which correspond to each of the C files and they contain object code right

01:20:38.920 like we compiled these things into object code they're not executable yet or even a library they're just object

01:20:44.360 files and then we run aswm LD and this is super important so this is the

01:20:49.520 linking step and what we're saying is we want to be able to access these exports in our Ruby code so we need to tell the

01:20:57.080 Linker to export them so we can use them we also want to we don't have a main function this is just a

01:21:02.719 library um we're going to Output a file called enigma. WM it's the the linked

01:21:08.239 compiled final version and here's our search path again so we can link against lip C here we are linking against liy and finally we're saying like we going

01:21:14.120 to link all these object files together so just star. o these files okay so once

01:21:20.000 that's that's done oh and then we run our our strip command so just take out some of the extra stuff we don't need in there

01:21:25.520 so by the end of this process we've got onmo atwm and we can see that if we run WM object jump we can see all of the

01:21:31.960 exports that are in this thing so we've got our memory which is the thing we're going to read and write write to and

01:21:37.600 write to and read from um we have these constants that we needed like the UTF encoding one and the syntax Ruby one we

01:21:44.760 have Malik and free and onig free and all this cool I cut it off because there's too much here but all the functions that we talked about on the

01:21:50.159 previous slide those are now exported we can use them okay

01:21:55.199 so now we get to actually write some Ruby code and to do that we're going to bring in the WM time Gem and wasm time

01:22:01.719 is a runtime for basm that has Ruby bindings so it's essentially a um a

01:22:06.960 native extension that um is written in Rust and the nice thing about Russ it's a lot

01:22:12.920 easier to compile than C at least in my experience and it also you know includes pre-compiled binaries for a bunch of

01:22:18.600 platforms including my arm Mac OS machine and it allows us to call WM fun

01:22:24.520 functions from Ruby and we can also call Ruby functions from WM so it's it's it's kind of the the the key that sort of

01:22:30.040 unlocks all of the the WM goodness that we're going to do in a minute so let's start writing some Ruby

01:22:35.679 code we have our we require won time we've got our module enigmo and to sort of instantiate our WM

01:22:42.560 module we've got an instance class method here it's confusing an instance class method it's a class method called

01:22:49.480 instance anyway and in this thing we're going to make a new instance of the time

01:22:55.560 engine get the path to our module um the store thing not important what that is and then we're going to say instance.

01:23:01.760 new here and pass in our module the third argument by the way here is to pass Imports so you can pass like Ruby

01:23:07.639 functions that then can be called from code essentially those can also be passed in this this array at the end

01:23:13.440 here um and then that's what's we're going to return from this we memorize it and that's the start of our of our code

01:23:19.920 we're just going to be like essentially just like making this have this little function that gives us a thing that now we can use to to make calls to these

01:23:26.800 functions we're also going to grab these exports so WM time has this nice interface we can say like do export and

01:23:33.560 then in this case because it's a global constant there's a two Global method doget we're getting a reference to this

01:23:39.440 thing um we also um expose an instance or a a method for our memory object that

01:23:46.120 we're going to use to read to and write from or read from and right to um we'll see how that works in a

01:23:52.880 minute um Okay so now we're ready to do the first step of our process which is the the new function so onic new Deluxe

01:24:00.000 so to do this we're going to add a method called onig new Deluxe and we're

01:24:05.159 grabbing uh using our exports again grabbing that function this time instead of saying uh to Global we're saying two

01:24:10.560 Funk this is a function we want to call and then all we're doing is taking the arguments we're ped and we're just forwarding them on to to to to the

01:24:17.360 onigod new Delux function so it's a really simple way of calling it now there's a lot of sort of things we're not doing yet we haven't actually like

01:24:23.520 determined what we have to actually pass in here all we're doing is giving it like an entry point like you call this thing the C library will do something

01:24:29.400 and hand you something back and we also need to do this for malakin free because we also need to be

01:24:34.800 able to allocate and deallocate memory and so let's just add a little handy function that lets us declare these

01:24:40.679 these these functions more easily so now we can say exports Funk oning new Deluxe malikan

01:24:48.679 free so let's take a look at the C declaration this is in the header file for onigmo

01:24:54.360 this is the onig new Deluxe function you can see it takes five parameters so it also um references this

01:25:01.000 thing called an onig uar that's just a type def or an alias or an unsigned Char

01:25:06.400 so nothing fancy going on there um onig compile info is another struct and this

01:25:12.920 thing has a bunch of information about like how it should compile your regular expression um and then another thing

01:25:18.000 oning error info so if there an error happens this um struct will get populated with the error information and

01:25:24.760 we also need to pass one of these things in and so we can see that it has three Fields all of which are also pretty pretty standard or I would say we know

01:25:31.000 what they are okay so let's try to implement this so module Enigma we're going to do

01:25:36.239 compile info first we have our class here and we'll say size 24 that's because that struct we looked at a

01:25:42.520 minute ago has 24 bytes of information in it because remember WM is a 32-bit

01:25:48.000 format so we can let's just bring up our definition of that struct here we can just count these fields and find out how

01:25:53.679 much how bytes they take so an INT is four bytes um owning and codings are um

01:26:00.280 four byes those are just addresses or pointers to data structures right so those are all pointers are also just ins

01:26:05.840 really and so that's four bytes there and all the way down the line until we add those together and we get 24 bytes

01:26:12.880 whoops okay so we know that compile info struct is 24 bytes so let's create a new um instance

01:26:21.000 of this struct so we'll expose a method called create here on pile info and it's expecting you to pass it a ruby hash and

01:26:28.080 then in this guy we're going to say okay WM give me some memory give me 24 bytes of memory because that's how much I need

01:26:35.080 to represent the struct in memory and what Mal is going to do is it returns to me an address and I'm storing that in

01:26:41.800 this start Adder local variable and then because in C um a

01:26:47.000 struct's fields are packed as they're defined so we can just run down the list here and pull out all of these elements

01:26:54.520 of our of our hash right and stick those into an array and they're now in order

01:27:00.000 and then I can write those to memory at the starting address so when I called

01:27:05.119 Malo it gave me a a contiguous block of 24 bytes and that block of 24 bytes has

01:27:10.880 nothing in it it's it's probably garbage it could be zeros I'm not sure we're going to write over that with all of the

01:27:18.159 fields we were given and we're doing this in order to maintain struct order right we're using the pack function rub

01:27:23.960 pack function and we're saying lar just means take all like l means um a 32-bit unsigned integer so it's saying take all

01:27:31.159 of the items in this array and pack them into the 32-bit binary format and then we're going to write that so returns a

01:27:36.600 string and then we write that into memory at that address uh that start address that we got from

01:27:41.719 Malik okay and then we're going to return a new instance of compile info and all that we really need to represent

01:27:47.880 that is that address and so we say new here which invokes our Constructor our initialized function and we're just

01:27:53.520 storing that address in the the class so that's the whole that's all you

01:27:59.080 have to really do for this right we've got like a a bunch of fields we WR them to Memory return the address and we're good to go okay so how do we use this

01:28:05.440 thing in owning new Deluxe well we can just grab our we can define a nice

01:28:11.000 little extra function make compile info and what this is going to do is just give us a bunch of like sort of canned

01:28:16.600 options you could obviously change these or have these passed in somehow instead of doing it um you know like like just

01:28:22.880 can like this um but this is kind of how I started because it was easier I have no idea what num of elements means I've

01:28:28.960 looked through the code I cannot figure out what that means everywhere in the examples they just pass the number five

01:28:34.440 and so that's what I'm doing too no idea what that means um so yeah so that's the beginning

01:28:40.119 of our of our our regx class so we're gonna create some compiling so that's all it does so far next thing we can do and I'm going

01:28:47.040 to handwave over these structs these are just you could sort of follow the same pattern we did before look at the definition in the C code Define your

01:28:53.239 Ruby class pack the fields the right way so um first we create a pointer to a pointer to a reg X I don't know why we

01:28:59.000 have d double pointer there but that's in the code follow that that Paradigm we have an error info and then we also want to create our pattern and we have a

01:29:05.480 utf-16 string class that like takes your Ruby string converts it to utf16 writes

01:29:11.080 those bytes out to memory and gives you back the the address just like we're doing with compile info okay and now we can actually

01:29:18.159 finally call the new function um so we're saying owning new Deluxe we're passing in just like we saw in the

01:29:24.360 function signature we showed the C function signature we're passing in the regx pointer the pattern start and end addresses um the compile info address

01:29:31.639 and the error info address so hopefully what if everything has been hooked up correctly this will give us back either

01:29:37.719 an error code well I guess will give us back an error code or zero for normal exit so we can check to see okay did

01:29:43.960 this error code actually return correctly if it did then we can return a new instance of our reg apps thing and free some stuff in the process or we can

01:29:50.679 use this error code to string function get back at error code as a string and and then return that or raise a runtime error for

01:29:57.360 that okay I know we're going very fast but I don't have that much time I'm so sorry okay and then for onic search was

01:30:03.440 the last one we'll look at so this one also we have our C function here and this we're going to sort of we don't need to really do too much here we just

01:30:09.000 need to Define them here oning search oning region new and um to remind ourselves of what this looks like we'll

01:30:14.800 just paste it down there so in our regx function we'll for regx class we'll Define another function called search

01:30:20.920 this takes a string this is the string we want to match on there's a start and end position so we can determine like we

01:30:26.320 can say like I want to match only within this substring of this string um so we create our utf16 string

01:30:32.159 pointer um we grab a new region um and then we call search and we can just pass in our regx our start

01:30:39.159 address end address um and then we're GNA there's the the next two parameters are the um the the range in the string

01:30:47.600 so like if I want to start at character three as opposed to character Zero that's how I would do this you pass in those positions why am I multiplying two

01:30:56.199 here any

01:31:03.440 ideas okay that's because this is a utf16 string utf16 strings are always

01:31:09.080 two bytes so the start address is going to be start position which is a logical position times two which is the number

01:31:15.119 of bytes the bite offset within the string to start searching same thing for end pause um

01:31:21.320 and then that's essentially all we have to do for for search um there's obviously a little bit more to handle that exit code so we'll say Okay um in

01:31:29.360 this case we're looking for and to notice this negative2 variable this negative2 value rather comes from the C

01:31:35.080 code I had to read through that to find this um so if the error code is less than negative2 that means an error occurred and so we'll do the same thing

01:31:41.440 we did before error code to string raise that as an exception um if there's an onic mismatch in other word I think

01:31:47.880 that's negative 1 if negative one occurs that means there was no match found so we can just return n otherwise there was

01:31:53.920 a match found and so we can use our there's another struct called match data that also on Enigma defines um and

01:32:01.560 that's I haven't shown the code for that but essentially we just pass in our string and our region and it gives us back like this nice Ruby array of

01:32:09.840 matches and then we have to clean up after ourselves okay so let's see if it works okay great we make a new regx and

01:32:17.560 you can see that the instance variable reg apps here is just a number it's just a it's just an address into wazin memory

01:32:22.600 it's not actually a a ruby class or anything other than a number okay and we

01:32:28.000 going to say read. search and that gives us back this very nice match data object with our captures in it so um because

01:32:35.000 ABC matched the a right there it matched characters four comma five it's the range of characters that matched um so

01:32:41.080 yes it works okay I don't remember what this is oh yes our test now I I wanted to make

01:32:48.400 sure that I you know included a test here because of Jason's talk um and I decided to rename it a little

01:32:54.920 bit

01:33:00.040 oh so I don't I don't I don't know that seems better than the previous version I think yeah I think so all right so we

01:33:06.400 compile we're gonna do the same thing we showed on the previous slide we're just GNA match here and we assert that all of

01:33:11.840 these return the right values I'm sorry we check the specification that uh this particular

01:33:18.480 example uh adheres to the specification yes thank you yeah thank you um I I

01:33:23.840 thought so I didn't think it was you know at all but I I could be wrong all right that's it thanks for

01:33:38.119 listening so that was a huge worldwind the code is also on GitHub uh if you want to read through it in at more

01:33:44.400 Leisure um there's two projects here Enigma WM is the C library that has like

01:33:50.199 the clang command in it and whatnot and then odingo Ruby is the bindings to to that to make it work in

01:33:56.480 R any questions before I step down do we have time for

01:34:01.560 that so you got it Ruby runs in the browser no problem um with Garnet you mean oh boy

01:34:09.679 no no it doesn't um opal is amazing though check out opal it's another implementation that does run the

01:34:16.520 browser

01:34:22.360 yes um no I mean so Garnet's a really fun project but I

01:34:28.480 kind of hit a wall on it I didn't know how to make any more progress and so I kind of gave up on it um it was more of a learning tool anyway if I'm being

01:34:35.159 honest right um so so you know it's some point it'd be cool to pick it back up again but I

01:34:42.080 mean at this point I think we'll just let it let it let it live on GitHub and

01:34:47.520 you know die die of a dignified death

01:35:07.600 you the the JavaScript code if you're interested

01:35:13.880 yeah all right thanks everybody this this is Alex's first

01:35:19.320 presentation at of Ruby um and he works at Dr and contribut to open

01:35:26.159 source yeah tell tell us about yeah I have a slide so thanks um so apparently

01:35:33.520 there is a lot common with Cameron's talk in mine I just realized that first of all we use the same slides H the same

01:35:40.840 template but I changed colors but if you look at it's actually quite the same

01:35:45.920 yeah um i' also talk about things that compile and basically the thing you use

01:35:51.159 to compile um and

01:35:57.159 uh I there was something else but I kind of forgot let me just start um so we're

01:36:02.280 gonna talk about basil I'm very happy to be here and have the opportunity to talk about my name is

01:36:09.600 Alex I work as a quality architect at topt I also do a lot of Open Source work

01:36:15.040 um I have uh written a small Cleo manager called Mac if for Mac OS it's

01:36:20.119 very popular maybe some people here use it maybe not um I also uh maintain

01:36:26.480 selenium so you probably guys all know the selenium project to some extent I

01:36:32.320 I'm responsible for everything related to Ruby there and basil we use basil

01:36:37.400 there as well uh apart from that I have started a project called alumnium doai

01:36:43.320 it's an open source project for uh AA power test automation you're welcome to

01:36:49.400 check it out but it's in Python it's kind of not relevant uh but for the

01:36:54.639 purpose of this talk what matters is I'm also a member of the basil rule special interest group where I maintain

01:37:02.400 everything uh like the support for Ruby language within the basil ecosystem and that's what we're going to

01:37:08.440 talk about today so um does anyone here knows what basil

01:37:13.719 is okay how do you know Cameron if I may ask comp used it

01:37:25.520 large yeah okay I remember the third thing you mentioned that figma uses is written in C++ and compiled so figma

01:37:33.080 uses basil um a lot of different companies stripe uses basil and many

01:37:40.199 many different uh big tech companies uh basil is just a build and test tool that's kind of it so we have rake we

01:37:47.719 have make we have a lot of different tools for building basil is just one of them but um it was was created at Google

01:37:55.639 and it was created at Google you can imagine that it has a very specific um

01:38:02.119 requirements in terms of how it works B Google has monor repo and uh basil was

01:38:08.040 created for monor repo specifically and then at some point it was open sourced um and uh now it's I

01:38:16.320 think it's an Apache 2 project um so it's a build and test tool created at

01:38:21.719 Google designed for monor repos by the way uh does anyone uh here not know what

01:38:27.159 monor repo is so our monor repo is just like when you put all your code in a single G

01:38:35.360 repository or maybe a different version control um all your languages all pretty

01:38:41.239 much everything and then it creates a set of challenges with tooling and basically is

01:38:47.000 a solution to many of these challenges um so it's for monor repos

01:38:52.360 it's polyglot meaning it supports all different languages in your repository you can have Java you can have python

01:38:59.320 Ruby obviously um you can have rust um and it all works with basil and because

01:39:07.400 it was created at Google it is extremely efficient it is extremely scalable because it can support huge monor repos

01:39:14.320 like magga repositories with millions of source code files and uh it kind of

01:39:20.400 created um a set of design choices and trade-offs that they've done uh while

01:39:28.159 developing Google and one of them is that it has to be hermatic and by

01:39:33.599 hermatic I mean that when you use rake or make you use it through your operating system right uh like you

01:39:40.880 install make on your operating system and then you use this binary but uh in basil everything is managed through

01:39:47.199 Basel so like if you want to use if you want to compile something with basil it will actually download compiler put it

01:39:53.000 into sandbox and uh it won't really pollute your operating system it will

01:39:58.679 use that um and the last point we'll get back to this remote caching and remote

01:40:04.560 execution I kind of show you through the demo why it is important to mention okay

01:40:11.000 so basil is a build tool so what is uh build in basil um and if there is one thing that

01:40:17.400 I want you to uh take from this talk is that build in basil is just take a bunch

01:40:23.199 of of inputs run some action over them and produce output so inputs are files

01:40:29.040 maybe you have some uh Ruby source code like add RB remove RB and you have a gem

01:40:35.560 binary these are the inputs and then you run action with this input so maybe you run this G binary and you give it a

01:40:42.960 build command and you give all the source code files and it produces uh a gem file that's your

01:40:49.760 output and then uh what you action produce as an output can be input for

01:40:55.320 other actions so maybe you build several gems and then you want to package them in your Docker image so you run Docker

01:41:02.360 build over this inputs and that creates a tar file which is a Docker uh Docker

01:41:09.520 container image um and that's kind of it so you have a bunch of inputs you run

01:41:14.679 some actions that produces new files that can be inputs to other actions and so on and so on and I mentioned that

01:41:22.119 basil is also a test tool and test is kind of very similar to that you have

01:41:28.000 inputs your source code your test files your maybe the rspec binary and then you

01:41:34.000 run rpec and it gives you some kind of test result that uh and an exit code of

01:41:40.599 course um and that's pretty much it and I guess you can see the pattern that uh

01:41:47.000 all your build targets all your test is essentially becoming a huge graph within your uh source code so some files are

01:41:55.760 inputs for some actions and some files are inputs for some other actions and they kind of all wrap up together in one

01:42:03.159 huge graph and basil creates this graph builds it for you keeps it in memory

01:42:08.560 caches it and then operates over it it's a graph and it's very efficient in a

01:42:14.480 sense that um if for example you work on your source code and you change the test

01:42:19.760 file and you tell Basil hey build an has everything for me it knows that okay it

01:42:26.119 doesn't have to run gem build because you didn't touch these files uh you only

01:42:31.159 change a test so you have to rerun rpec and you don't need to run Docker

01:42:36.400 build or gem build uh because it's not relevant um I also mention it's polyglot

01:42:43.599 and because inputs and outputs are just files it can be anything right so maybe

01:42:48.800 you have some static assets that are not relevant for testing and building the gems but they are important for

01:42:55.679 packaging the docker image or maybe you have front end that you want to go

01:43:01.480 through some JavaScript tool chain and maybe Minify and again if you run your

01:43:08.199 specs it doesn't matter it's not an input uh but it is important when you

01:43:14.239 package everything inside the docker build and uh it's polyglot and when I

01:43:19.960 say it's polyot it's truly polyglot so it has built-in support for Java and python these are I guess because Google

01:43:27.520 uses those extensively um and whatever programming

01:43:32.560 language you have in mind name it there is a a support for for it there is no JZ typescript go rust rust has native

01:43:40.599 support for compiling to basm by the way um there is pack there is special rules

01:43:46.440 for Docker open container initiative Helm and Ruby so let's talk about Ruby

01:43:51.719 that's kind of where we're here um so Ruby is just a module for Basil

01:43:57.320 that supports provides native and decent support uh for for the programming

01:44:02.800 language so as I mentioned um basil will install all the

01:44:07.920 dependencies for you so it would the rules Ruby would install Ruby it would install dependencies through bundler and

01:44:14.199 then provide some kind of a conventional common interface that you would use to

01:44:20.040 uh build gems uh run tests run Rubik script and we'll show I'll show all of

01:44:25.280 that in a second but basically you can think of it as just a module within the

01:44:31.000 ecosystem all right demo let's try to do that and um is that big enough folks in

01:44:39.119 the back can you see that okay a little bigger is that

01:44:45.280 better yeah fine okay um if it's not good enough just let me know um okay so

01:44:52.280 for the demo I have a ruby gem um it's fairly I guess uh straightforward so we

01:44:58.719 have some gems back it's calculator I didn't came up with anything better than that um there is some files like

01:45:06.960 calculator there two classes to add and subtract and it's uh very simple just

01:45:12.920 you know basic operations uh basic math operations

01:45:18.719 um there are some specs for them like calculator aded works correctly corly

01:45:23.800 and subtract works correctly so we can run this it works um and and then we

01:45:33.000 have a Jem file with our spec and rubocop so probably I have some rubocop we can run it yeah fairly easy right

01:45:41.159 nothing nothing fancy um and yeah by the way I use Ruby 3.3 H maybe that matters maybe not uh

01:45:49.440 but yeah it's try I'm kind of trying to give it simple so um let me start by adding basil to this

01:45:56.199 and kind of show you what it does and then I hope we get to a point why you

01:46:01.639 might want to do that like why why what's why does it matter so um we start

01:46:07.639 by saying that there is a special file called module basil it's basically a way for module for Basil to say okay this is

01:46:14.199 a basil workspace this is a root and there you declare um what kind of

01:46:19.840 languages do you need so here we have we say we need rules Ruby uh and it will install the uh the module and then we

01:46:27.760 say um install please install Ruby for me um so just read it what's in Ruby

01:46:34.159 version file uh install it and then fetch some dependencies uh uh through

01:46:40.800 bundler so we need gem file we give it gem file gem file log the gem speack and

01:46:46.119 the version RB that's because the gem spec requires version RB so we have to

01:46:51.480 explicitly say these are inputs okay um and then we have uh build file and build

01:46:58.679 file is just uh where you actually Define the targets what you want to

01:47:03.880 build what you want to test so uh if I tell basil now to build and I use triple

01:47:11.520 dot this means recursively build everything it's not going to do anything you can see that it created some folders

01:47:17.920 these are Sim links for convenience uh but it hasn't done anything

01:47:23.440 because there are no test targets and even though we said we need Ruby we we we don't have any Targets that require

01:47:29.840 Ruby um so nothing has happened so let's try to do that let's try to add some

01:47:36.239 Target um and for example what we're going to do is um and uh GitHub copilot

01:47:44.119 is smart but not smart enough um the language uh by the way the language is

01:47:50.400 called starock it's uh looks like python it's not really python but it has things

01:47:56.679 like cycles and U loops and uh functions

01:48:01.840 so let's say uh we're going to build a gem okay so let's call it

01:48:07.040 calculator and it needs some inputs inputs are source code files so we're just going to take everything inside the

01:48:13.320 liap folder so this is I guess a regular glob expression um no dependencies we also

01:48:20.880 need to say where is our gems pack um

01:48:27.040 and that's that should be it so let's try to build it and let's see what

01:48:33.560 happens it worked yes awesome so you can see that it uh build a gem and it

01:48:40.719 actually prints the output from gem command and it put it somewhere in basil

01:48:46.280 Bean folder so um Let me refresh that yeah you can

01:48:53.199 see it over here and maybe we can unpack it and see what's inside so there is

01:49:00.719 this well these are our source code files right nothing else this is everything that we declared as an input

01:49:08.280 um so let's let's remove that um what happens if I do it again nothing

01:49:15.480 it says it you see it does not attempt to build again because nothing has changed in the inputs we said that for

01:49:23.320 these files do this and as long as nothing has changed in these files it's not going to do anything because what's

01:49:30.400 the point right the result is going to stay the same um okay so building gems is nice uh

01:49:38.199 what about rubocop um let's try to add rubocop so rubocop is a test in a sense

01:49:47.239 like you run it it returns an exit code uh one if it failed um an exit Code Zero

01:49:55.119 if it passed so let's just run rubocop over the same source code uh maybe let's

01:50:02.320 add specs yeah I think it kind of makes sense maybe let's add gem file I think

01:50:08.280 rocop runs over gem file and gems pack as well so these are all SRC file these

01:50:14.880 are all inputs for ruoko um and in order to tell basil to

01:50:21.880 actually run rub op because it's maintained by bundler we need to tell to

01:50:27.480 use a a rubocop uh binary that is available through bundler and it's it's kind of an

01:50:34.880 LS here so what what is here is the same thing that we have used here so can be

01:50:42.360 anything but uh the main thing is um we need to tell which binary we run and we

01:50:50.480 need to tell that we need the whole fondler dependencies to do that uh so if we run

01:50:59.800 now let's to basil test triple dot again run all the tests that are available you

01:51:06.520 can see that it actually installs bundler and installs a bunch of gems and then it runs the test and it

01:51:13.639 fails um and it failed because there are some legitimate

01:51:20.079 offenses uh but I yeah I think fail because I did not use rubocop yaml as an

01:51:27.320 input so let me just add it um here there is this you might be wondering why

01:51:33.880 it's not part of the src's field um it's a it's a kind of decision that basil has

01:51:41.719 made at some point that src's are just uh language specific files like Ruby

01:51:47.360 files in our case and data is everything that is not so but essentially it's the

01:51:53.400 same thing it's just a convention so we run the test okay cool they're passing

01:51:58.719 now and you can see there is no output uh by default if it's passing

01:52:05.079 there is no output but we can treat that and yeah you can see there is this

01:52:10.760 regular ubac okay cool it works you might be

01:52:16.679 wondering why what's the point to do that like you can still run bnda of CH

01:52:21.960 build so let's try to make something a bit more reasonable and fun let's try to

01:52:28.639 add some specs um so to add specs I'm going to create a build file inside the spec

01:52:35.480 directory that's where we are uh hold where we host or our tests and let's add

01:52:43.920 the same RB test and let's define a test for our first spec

01:52:49.599 file so uh we're going to call it a add and um we're going to include our spec

01:52:58.639 file oh sorry that's adpack RB and similar to how we use droc cope here we

01:53:06.040 need uh these two Fields um Main arpack and bundle that I

01:53:13.440 think that's it so let me try to run all the tests inside the spec

01:53:20.920 folder um you see that it R spe add and it failed and if we look at it it failed

01:53:28.440 because it actually cannot load calculator and it makes sense because for this test

01:53:36.880 all we have is just the test itself so the way basil works is that it will take

01:53:42.960 all the inputs put it in a separate folder in a sandbox environment isolated from your operating

01:53:50.239 system and then run our spec there because of that there is no calculator there so we have to declare somehow

01:53:56.760 dependency over it and um in basil you can't refer to files

01:54:04.480 from your parent folder because that would break the isolation and that would

01:54:09.639 potentially allow you to escape uh from the sandbox uh at all so the way you

01:54:16.280 would do that is by declaring proper dependencies inside this uh inside the

01:54:22.520 Le folder so let me let me just quickly show you how you would do that so again

01:54:28.199 we create a build file um this time let me just copy this time we say it's U it's not a

01:54:36.599 test it's just a ruby library and what we're going to do is have um we're going

01:54:44.079 to declare a library for each source code files you can join them but for the

01:54:49.480 purpose of this uh presentation we um we want to have them

01:54:55.880 separated okay um sorry typ yeah looks

01:55:02.520 good um yeah I think it looks good yeah um

01:55:08.440 and then we expose it the calculator RB and we need to do the same for all the

01:55:13.920 files inside sub directory so let's do that let's add one for

01:55:20.880 add RB um let's do the same for

01:55:26.840 subtract and let's do the same for version even though we don't care about it that much so um and

01:55:36.280 now if we switch back to our tests we can say that it actually depends

01:55:42.440 on calculator. ADD um calculator um

01:55:49.719 there is no such thing calculator do subract and the calculator file that is in the original

01:55:58.040 folder and try to run the test again so now basil we think that I need to put

01:56:05.000 these files to the same folder too and it fails because there is another

01:56:12.360 Concept in basil called visibility rules and we'll get back to it I just wanted to show you that it fails by default you

01:56:20.320 can't refer to things that are not uh publicly declared which

01:56:26.360 kind of might make sense in when you work in a huge code base um so let's just fix that real quick we're just

01:56:32.719 going to say that everything is public inside this folder and everything is

01:56:37.800 public here and rerun the tests and finally they're passing okay um cool

01:56:47.040 again why would you want to do that makes no sense just run our spec um so

01:56:52.480 let's add another test for subtract and we're going to run the

01:56:58.719 subtract spec actually we don't need subtract for adding test right it makes

01:57:05.280 it is not used there likewise we don't need add uh RB in in our subtracts spec so

01:57:12.239 let's try to run this two again um uh sorry

01:57:20.280 typo Okay so it run two tests one thing you can notice is that it it run two things not

01:57:28.360 one this is because by default every test is run in its own process in its

01:57:33.400 own directory with its own set of inputs so if you if you have a lot of tests um basil

01:57:41.520 out of the box would allow you to paralyze them and scale them as much as you want like use all the cores um but

01:57:49.599 another thing that you might notice if you run the test is that it says it's cached so what does it

01:57:55.040 mean it means that basil took all the inputs uh calculated them and cached

01:58:02.239 them run tests and as long as you don't change the inputs it's not going to rerun the tests because nothing is going

01:58:09.079 to change so if we go to let's say add B and uh dog add some changes and run all

01:58:18.639 the tests you can see that subtract Act is cached it's not being runed in the only

01:58:25.560 rerun tests that matter um and that is one of the most

01:58:32.480 important things uh with basil is that you can work on your source code as long

01:58:38.360 as you build everything properly you can work on your source code and you can run

01:58:43.440 all the tests and it will only run things that makes sense that matter for this

01:58:48.880 particular uh based on your on your changes and

01:58:54.520 um another thing that I that is very important is that uh because you can't

01:59:01.760 just refer to uh classes and constant that non-exist in your uh test you can

01:59:08.000 uh catch uh you can kind of set the boundaries within your application so

01:59:13.599 like I know many people use packwork these days which provides you a way to isolate uh part of your sour code and

01:59:21.800 particular folder and not allowed to use it so think of it as um basil provides

01:59:28.400 this out of the box it's kind of steroids because if for some reason I say well subtract now depends on ADD and

01:59:35.599 I try to run the test the test will fail because there is no such

01:59:41.079 file and I can fix that of course I can say that actually

01:59:46.880 subtract depends on ADD and now my test is passing uh but if you compare it for

01:59:53.360 example with packwork what you can do easily with basil is you can ship

01:59:59.199 everything to production and not worry about things breaking because you refer to constants that non-exist but you can

02:00:05.840 also run all your tests and catch uh violations that uh catch the like the

02:00:12.800 code using some other code it's not supposed to be using okay and

02:00:18.719 um um yeah let's let's revert that didn't make much sense anyway um but

02:00:26.400 yeah so that's that's that's the main uh thing with testing so you cache

02:00:32.400 everything and then you keep track of uh changes so for the demo uh couple other

02:00:40.920 things I wanted to show is that once you start adding these build files and you kind of takes a bit of work but ENT it's

02:00:48.960 it's it's not very complicated but then when you operate on large code base and you have multiple teams working with the

02:00:55.639 code you can start using the same build and test tool as a way to notify when

02:01:01.800 you for example want to change something so like if we uh we can add

02:01:07.079 deprecations and they will be printed we can uh we can

02:01:13.119 declare uh actually the whole source code as um let's do it private and then

02:01:20.400 we will see the same errors during the test execution as you've seen before um so yeah think of it as

02:01:29.360 a packwork plus a test Runner plus everything uh like a build tool and um

02:01:37.760 that is that that works out of the box and and with a lot of extra uh nice things uh I wanted to

02:01:46.520 mention we I mentioned the remote caching and remote execution um and I

02:01:52.159 want to kind of uh explain why I think it's very important so you've seen how it things

02:02:00.239 are being cached in your machine so you run test once and then you can rerun them as many times as you want it's not

02:02:05.480 going to do anything but because everything is input and file um you can take this cache and you

02:02:13.280 can put it in in the cloud and out of the box basil for example can put your

02:02:18.480 local cach to Google cloud storage and then some other developer in your team

02:02:23.760 would clone the repository run the same basil test and as long as their inputs are the same it's not going to again

02:02:30.400 it's just going to download the cache and uh you're not going to he's not going to run these

02:02:36.360 tests um likewise take it further everything is an input and file why not

02:02:42.760 we just put it like take the inputs put them to remote machine run the actions

02:02:49.280 there that will produce some other files and then we download these files and that's called remote build execution

02:02:55.920 again supported natively by basil uh there are open source servers you can run in your infrastructure there are

02:03:02.599 paid services that provide that but um Dev containers and a lot of

02:03:10.040 tools exist for remote execution and basil kind of have it out of the box with a big difference that you don't

02:03:17.079 really containerize anything you keep working in your local computer you just change the code in your ID and then

02:03:23.679 basil would package that applaud run something intensive there scale you can

02:03:29.119 run the test maybe with a 100 course parallelism that all happens inside the

02:03:34.360 cluster and then you get the results back um an example for that selenium I mentioned uh is massive monor repo so we

02:03:42.440 have Java we have rust trby C++ C all of that works through

02:03:49.520 basil and um it was RCI was taking like several hours to run everything and then

02:03:57.239 once we adopted basil once we uh started using the remote cach in remote

02:04:02.599 execution now you open the pr and you get it runs all the tasks and it gives

02:04:07.679 you like five minutes it runs everything that you actually care about because you changed these files and the rest is just

02:04:15.119 cach and it's uh it's so powerful uh concept having the inputs cashing them

02:04:23.159 and producing the outputs um so I think that was that that could be compellent

02:04:29.440 right so we have a lot of uh paral parallel tests and different solutions for that um and basil have this out of

02:04:36.480 the box um yeah so I think at this point

02:04:44.119 um uh that's mostly it I just wanted to say thanks uh some links for Basil the

02:04:51.760 is a GitHub repository for the rules Ruby we have a basil slack with Ruby channel so if you have any questions or

02:04:58.000 you would like to give it a shot uh you can go there and ask questions I'm there

02:05:04.400 um and then there is my email if you if you're not a fan of slack just shoot me

02:05:09.760 a message thank you very much I hope I managed to convince you that there is other build tools except for out there

02:05:25.559 question um about the adoption I think that's what people think about uh so you

02:05:31.159 mentioned selenium case when you guys were know adopting and um how much effort you take

02:05:40.920 um I would say it wasn't two Engineers I think

02:05:46.400 working on that and selenium being an open source project you never have full-time people working on it

02:05:52.360 but we have Stuart who is uh one of

02:05:58.520 the uh core contributors of Basil so he knows it very well but I would say it

02:06:05.159 took us maybe Year and that that's that's kind of

02:06:11.119 painful but again selenium being very specific we have a lot of languages uh and we actually written the

02:06:17.960 rues Ruby because nothing like that existed uh first lenum project so the

02:06:24.800 adoption is complicated um and I would only suggest to do that if you have a

02:06:30.040 monor repa with multiple languages then it's worth it otherwise no just just use

02:06:35.119 rake there's no

02:06:41.400 point for like a typical cicd pipeline say you want like build a Docker image

02:06:47.920 and then push it up somewhere how do how does basil fit in like the last last step to get it get your image live or

02:06:55.079 something like that so what we did is um you use rules oci that's a thing that

02:07:03.520 takes your inputs and uh builds a tar at a Docker image then that is pushed to

02:07:10.480 repository like a regular Docker uh registry but on top of that it's used as

02:07:16.199 an input for Helm package so uh in a release pipeline you just run uh

02:07:22.800 something like basil run deploy that will build the image that will then trigger the helm and uh at that point it

02:07:30.800 will finish

Sections in this recording

SF Bay Area Ruby Meetup - January 2025

SF Bay Area Ruby Meetup - January 2025

Key Points Discussed:

Conclusions and Takeaways: