00:00:11.080
hello again um this talk it's about uh it's called let's give rest a rest exploring
00:00:18.880
the state of grpc in Ruby um first let me introduce myself uh
00:00:25.720
I'm development team lead at thoughtbot it's a through in ra consultancy you might have heard about um we've been in
00:00:32.559
the market for 20 years more or less I'm from guara Mexico so that was like a
00:00:39.320
long trip over here um there in Mexico I
00:00:44.600
I'm the organizer of a ruby Meetup called Ruby MX and the organizer and the host so I'm really aware of how
00:00:51.960
difficult and how exhausting is to organize a conference even though my midle bits for 40 people I can't imagine
00:00:58.239
how hard is to organize this so kudos to the organizers it's been great
00:01:03.359
I'm also Co organizer at the Rails Bridge Mexican chapter if you have not
00:01:09.360
have not heard about ra reach is an initiative to introduce people with uh
00:01:15.280
low representation in the industry to start doing Ruby and it was an AR say to get here so
00:01:22.479
I took a bus and then a plane and then another plane and then another plane so it was like close to 24 hours travel
00:01:30.360
straight but I made it and it's been worth um let's start with the
00:01:36.439
story uh I a long time ago I joined a ruin rails consultancy as a traine in
00:01:41.920
2013 there in Mexico uh in October 2016 I left that
00:01:47.600
company to join a startup and at that time and I'm still in love with r Ruby
00:01:53.399
but why why did I move companies if I was doing Ruby I was getting paid for doing what I love
00:02:00.240
uh the reason is the stack and not the Rubi stack that I was using and I was
00:02:05.360
familiar with but the stack that I was going to work on remember at this time
00:02:12.760
um I had never work in go I had never worked with Cassandra I was using just
00:02:19.239
postgress uh kubernetes was kind of a brandy thing I have never heard about that angular 2 was in
00:02:26.000
Veda and we were using grpc and Proto off and from from this stack that was
00:02:33.239
new to me there was one specific part that caught my attention which is grpc
00:02:39.319
and protool which is what we're going to talk about today uh sadly five month five months
00:02:45.400
later when I joined this company I had to put my open to work Banner uh the
00:02:50.840
startup life was not for me but I learned a lot so that was really cool and in 2017 I joined another Ruben
00:02:58.040
rails consultancy and I i' fortunate enough to been doing Ruan rails full time since then so I really feel blessed
00:03:05.879
about that one um when I join when I came back to to Ruby I tried to bring this tool that
00:03:13.799
I love a lot which is grpc and protoo and I tried to like hey this is a really
00:03:18.840
cool tool I think it has a lot of Future Let's implement this in Ruby and I tried
00:03:24.799
this with a couple of projects there was a client that was implementing an MVP with microservices
00:03:30.360
yep that was not a good idea uh but I tried to implement this try to promote it and I did have no luck at all um and
00:03:39.439
the reason or the reason that I want to give this talk is I want to explore and I I wanted to know why grpc is more
00:03:48.480
popular in some languages compared to others and the first thing that came up to my mind it's okay let's let's try to
00:03:55.360
get some general idea about how popular it is and I search for G RPC go and I
00:04:02.920
got 12 more than 12 million records and then I did the same for JavaScript 4.4
00:04:10.519
million uh results let's try C 6.7 Million
00:04:16.560
results what about c 2 million not bad what about
00:04:23.000
Ruby uh do you have an idea how many 100 uh not not a little
00:04:31.039
bit more um not even half
00:04:36.400
million so let's do another travel to the past or let's move to 93
00:04:44.840
94 um in about 93 94 we have the popular
00:04:50.560
popularization of the web the first web page was was created in in August 1991
00:04:56.199
and was published a month later um the web start to get popular 93 94 when
00:05:01.720
Pages for General use were made available at that time there was only a fragmented description uh about the web
00:05:09.360
architecture and there was some pressure about setting a standard on the web interface that we were using um for
00:05:15.800
example some experimental extensions were already added to http to support uh multiplexing and proxy servers and there
00:05:22.960
were more extensions proposed uh but we didn't have a formal architecture and it was needed
00:05:30.000
um then we had a couple of work groups for the w3c and the EF the internet
00:05:36.759
engineering task force they began to work on some formal descriptions for the three three main components of the web
00:05:43.479
which is URI HTTP and HTML and there was one person that you
00:05:50.560
probably might hear about that person uh that was involved in three in the three
00:05:56.440
in the elaboration of the three standards a specific Ally http1 HTTP 1.1
00:06:02.479
and URI uh this person is Roy Fielding um Android
00:06:09.319
Fielding uh the following next six years from 1994 to 2000 he work on the res
00:06:16.880
architecture testing the Restriction of the standards uh for the the standards for the web that we already had and he
00:06:24.759
tried to use those standards and the limits to identify discrepancies and try to do some improve ments on the
00:06:30.039
architecture he was proposing and in 2000 he came up with uh
00:06:35.440
his PhD disertation called architectural Styles and the design of NW Network Bas
00:06:42.039
software um the long story short is we have this diagram uh that you might have
00:06:47.680
already seen in the past rest is designed to work uh for network-based
00:06:53.000
applications specifically following the client server pattern which we still use a lot uh more than that it was not just
00:07:00.759
for local applications it was designed for the scale of the web so the coping
00:07:06.400
of client and server should be very flexible because we didn't have control about the clients and we wanted or he
00:07:14.160
wanted to make the integration for web as accessible as possible for many people this loose coupling uh and also
00:07:22.520
that the communication was text based uh using a uniform protocol set the
00:07:28.199
foundation for the web requisit which was independency robustness support language direct transfer and a
00:07:34.720
low-level barrier for consumers for Content creators and developers um some of the relevant parts
00:07:41.840
of this architecture that you might be really familiar is that should be a or it is a stateless client server protocol
00:07:49.280
in theory H HTTP request has all the information that it's needed to complete
00:07:54.720
the request in practice we know that we store some state
00:08:00.720
uh we also have well-defined operations that can be applied to all resources and HTTP defines these operations the most
00:08:08.560
common post get put and delete also we have a universal syntax to identify the
00:08:14.879
resources and in a res system each resource can be redirected or can be directed uniquely by its
00:08:22.199
URI and we also count with Hyper media which allows the navigation between
00:08:27.240
these these resources and we can navigate from resource to another um let's do another trouble now
00:08:35.000
to the Past again but the future compared to 1994 so let's go to 2016
00:08:40.680
which was the microservices era um specifically in August 2016 uh
00:08:47.720
Google released some technology which is called grpc which is what bring us here
00:08:54.120
um grpc it's a remote call procedure open source system that was initially
00:09:00.360
designed by Google it uses http2 and uses protocol buffers for the language
00:09:06.720
interface description it provides features such authentication B directional
00:09:12.600
transmission of data and flow control uh blocking and no blocking bindings and
00:09:18.040
cancellations and timeouts and one of the features that call my attention the most is that it also offered
00:09:25.640
um how can I how can I say it uh cross platform uh code Generation by default
00:09:32.880
for many languages um the most common use case as I mentioned at the time was microservices but also uh it found like
00:09:40.200
some interesting niche in Mobile mobile web applications that needed to require
00:09:45.760
uh that wanted to connect to backend Services um the idea is simple we have
00:09:51.959
servers and we have clients and those servers communicate back and forth with the clients the the communication for
00:09:58.279
this case always start with the client the server gives a response that's pretty standard and even though at the
00:10:05.160
time grpc was something relatively modern H it's in reality a new version
00:10:10.360
of uh the RPC architecture um they offer support for several languages including Ruby C
00:10:18.000
cotlin node Java uh net Dart and others
00:10:23.600
and there are third party support for other languages not mentioned on that list what is our RPC uh if you for some
00:10:32.399
unfortunate reason had taken like a class or a course about distributed
00:10:38.279
systems uh I remember that I had to do this by a scratch I had to take care of communication between two
00:10:44.560
machines and one machine acts as a server and has some operations available and the client automatically or
00:10:52.560
automatically they only had to do a call that seems to be local but the internal
00:10:58.560
or or the code we have to implement had to figure out how to find a service that accept the call then receive the data
00:11:06.480
process it do the operation return the the data pack it and then the client
00:11:12.360
would just use that as a regular local method but in the in reality it's not local it's doing a a call to external
00:11:19.839
system um the idea behind RPC is that the service defines uh the interface
00:11:25.959
about what what are the operation that that the system can perform um the Char characteristics of
00:11:32.959
grpc again the ones that really when I the first time I work with this was like oh this is really cool I really need to
00:11:39.440
do this in Ruby uh definition definition of services and contracts which which is
00:11:45.360
what we mention that a service already exposes okay these are the operation
00:11:50.440
that I can do this is the data that I need and this is the data that I'm going to uh return and the way that grpc
00:11:56.639
implements it it's through uh protocol B offers or protop off uh which is a
00:12:02.040
binary format also developed by Google at the time that simplifies the storage
00:12:07.120
and interchange of data between applications um this is language agnostic this is how a prototype looks
00:12:15.720
like if you see we have a declaration of the syntax we can use a package and then
00:12:21.680
we have two main components we have a service in this case a hello service
00:12:27.199
this service has only one one operation the operation say hello this operation say hello expect a hello request and
00:12:35.440
it's going to return a hello response and if we see we have two messages the two me messages mentioned before we have
00:12:42.680
a Hello request that only has one attribute a greting that is a string and
00:12:48.320
it takes the position number one uh we also have their hello response that has
00:12:53.800
also one attribute reply and takes the position number one uh we have four type of method
00:13:02.639
calls the classic one or the simple which is what we do most of the time a
00:13:09.839
client sends a request to the server it waits for a response and the communication it's
00:13:15.560
done uh for the example that we're going to see this is the one that we're going to take a look because it's the most simpler and it's the one that it's
00:13:22.160
closer to a rest API which is what we're comparing to but also we have a server size
00:13:29.120
streaming when the client sends a request and then the server just gets a stream to read the the a sequence of
00:13:36.040
message back the client reads from reads from the return stream until there are no more messages and the commun
00:13:42.279
communication is done uh if you take a look the difference is the word stream
00:13:47.680
there in uh returns stream hello response
00:13:53.399
sorry we also have the yeah the other way around we have a
00:13:59.639
client a client side streaming where the client writes a sequence of messages and S sends them sends them to the server
00:14:06.480
against using a provided uh a provided stream once the client has finished writing the messages it waits for the
00:14:13.000
server to complete the response Returns the response and we're done or we have
00:14:18.759
one of the interesting ones uh V directional streaming which is pretty much a combination of the the previous
00:14:25.320
two both sides send a sequence of messages using a read write stream um
00:14:31.519
the two operate independently so each client and server can write and read and
00:14:36.639
write in whatever order they want um one example uh they could wait the the
00:14:43.079
server could wait for all the messages uh before writing the messages back or
00:14:48.639
could like do an alternation okay if I get get the server get a message from the client it returns a response and
00:14:55.639
then the client do that do this can do this again or any other combination that
00:15:00.759
we want again let's see some Ruby um there's a demo you can take a look at
00:15:06.519
that URL I'm going to share that at the end so you don't need to you don't need to write that at this
00:15:13.240
moment um as I mention the interesting part is the definition of a contract um
00:15:19.000
we're going to use the same contract well that we're already familiar with pretty much a hello it's a hello war
00:15:24.720
with grpc Ander above we have the hello service remember we only have one available say hello we receive a hello
00:15:31.880
request and we return a hello response okay we have the Proto definition what's next we need to create
00:15:39.800
create our server and other one of uh the other characteristic that really
00:15:45.240
caught my attention is that as I mentioned we have support for several languages but we also have a tool for
00:15:51.560
code generation based on this the previous Proto definition so one example
00:15:57.680
for for this example in Ruby this is how we generate some code based on that Proto definition uh where we have a tool
00:16:05.480
once we install uh grpc and Proto of machines it will be available so pretty much we specify where where what is the
00:16:12.680
folder for the protos where we want the output and what is the Proto that we're
00:16:18.560
going to to generate code from which in this case it's hello. prototo this is going to generate two
00:16:25.720
files we have a PB file um again we have uh a descriptor data
00:16:33.920
that it's going to be used to identify the service and also the interesting part is that module if you see it has
00:16:40.720
two definitions a hello request and a hello response which is exactly what we Define in our
00:16:47.120
Proto and another file we have a service file this is the base this is the the file that we're going to use to
00:16:53.120
implement the operations for service um again the relevant part this
00:16:58.519
is boiler plate that it's generated so we don't really need to get into the details we spec well the tool exports
00:17:06.919
the marshalling defaults for Marshall and Marshall in this case we're going to use en code and decode and the other
00:17:13.079
interesting part is if you see there's a stop that it's equal to service. RPC
00:17:19.039
stop class okay now we have these two generated files what's next well we need
00:17:27.240
to implement our service um what we need to do is we need to
00:17:32.520
inherit from from that uh generated service hello hello service service in
00:17:37.840
this case and then we implement the methods that we promise that we're going to deliver in this case we only have one
00:17:45.480
say hello uh what we receive as as input is the hello request and we receive the
00:17:52.520
call in this case we only care about the request um what we're doing is it's very
00:17:57.760
simple we're just putting uh we're putting some output to display the message that we're receiving which
00:18:04.720
is a hash in this case the way we can access uh the data it's through hello
00:18:10.000
request. greeing which again if we go back to our Pro definition this is what
00:18:15.159
we said we have we have a greeing so we have to or we have the data available in
00:18:22.600
that value and okay and then as a response we
00:18:28.400
need to we just create a new response we pass the data that we promise that we're going to pass in this case is just one
00:18:34.280
field reply and we're just doing a concatenation like hello and the greeting that we already have that's the
00:18:41.280
client that's the only thing we need for the client uh grpc already takes care of the packaging takes care of the
00:18:47.360
communication we're done to create a server it's also very
00:18:54.960
easy in this case where where we have uh the grpc part it's going to do the communication but we need to start that
00:19:02.080
somehow and the easiest way you can do it's you can define a port in this case we're going to run in the uh in the port
00:19:09.640
5051 it's kind of the standard at least for testing we create a new instance of
00:19:15.520
the server we add the uh we tell the server to listen to this port which is the one
00:19:22.360
that we Define also some logging just to have some idea on what's going on and
00:19:28.440
then this is the the critical part server. handle with this we're telling
00:19:35.120
okay you're a grpc server and you can handle the messages defined here and
00:19:40.159
then we just have like a way to terminate the program okay now we have a server what's next we need to create our
00:19:48.159
client again we already have a Proto we can do code generation is exactly the
00:19:54.880
same um just for this case I'm using a different folder to simulate that we're working on two different uh machines
00:20:01.960
this case it's client instead of server but the but the output is exactly the same because we're using the same Proto
00:20:09.520
definition and now we need to implement our client which is also something kind
00:20:14.840
of simple um remember this stop that we had in the generated file uh we use it
00:20:23.159
here it's like okay we have an stop but this is where you're are going to find the service says the real ones
00:20:30.000
so we're just specifying that they should be available in Local Host 50 or
00:20:35.240
51 and then we have another method that just says R say hello and we send this Stu if we see at the top the method it's
00:20:42.640
doing is just receiving the argument for command line argument or if we don't provide one we use uruko also some
00:20:50.039
messaging just to see what's going on and what we do is okay from the stuff we
00:20:56.080
have that this stuff already knows how to communicate with the real server uh let's call the same hello
00:21:03.120
method and we pass the request we we do a new instance of this of the request
00:21:08.720
with the message that again we already agreed to okay we're going to send a message with the gring value and that's it we get a reply as we
00:21:17.679
promis in the Proto and we can use that um if you see we don't have any any
00:21:24.279
visual clue that this is doing a call to a remote server instead of a local server it's just like a regular method
00:21:31.159
call um how we run these things just like regular Ruby apps we
00:21:37.320
don't need to do anything particular is the way we're on the server um then how we made the client do
00:21:43.600
a call to this method this Cas just run the application and we pass the argument
00:21:49.440
um this is pretty much the output when when we start the server we get some uh this case one warnings and when we run
00:21:57.400
rby client and the message that we want to send uh it's it's going to send the
00:22:02.600
message to the remote server the remote server receives the request and then
00:22:08.159
response and the client just writes hello euruko
00:22:13.760
2024 um that's the end of the demo what are the advantages of using
00:22:19.440
this because even though we are not in the microservices era anymore which glad
00:22:24.600
we are not there are some advantages and some cases where probably it can be useful for you to implement protocol
00:22:31.679
buffers and grpc instead of like a restful API one of the issues or one of the
00:22:37.720
things that tries to solve it we have seen this problem again and again
00:22:42.799
versioning we changed the contract for an API and we need to create a new version and then we have version one
00:22:48.279
version two version three um the way that we can solve this with uh this
00:22:53.760
technology it's how it's ver it's we we don't have like proper versioning but we
00:23:00.640
take advantage of protoo for example this message has
00:23:05.679
three um three attributes has the attribute query in the position number one page
00:23:13.039
number in two and results per page in three let's say that we want to get rid of page number like okay we don't need
00:23:20.320
page number we're not going to return page number uh the first thing that might comes to to our mind is okay I'm
00:23:26.360
going to delete that field but what's the problem there might be some clients that depend on that field so what do we
00:23:33.600
do we don't delete the fields we just deprecate them and we can just okay delete and
00:23:40.640
also mention like ay this um this number feeli it's already reserved don't try to
00:23:47.760
use it again because that's also one of the problems that we might have okay let's let's say that we remove the
00:23:52.840
number two and then in the future another developer says okay we're going to use number two for other things so
00:23:58.640
then we have like okay no you cannot do that because this is already reserved it was used in the past it's not used
00:24:04.279
anymore so you're not allowed to do that uh this is interesting uh it gets
00:24:10.279
tricky when you have like hundreds and hundreds of fields uh but but it's a way that it's like it's wor it's worth
00:24:16.840
exploring and it's something that we can try to mimic in our a press API instead
00:24:23.000
of changing our responses we can just like okay this field is replicated and now we have a new feel that it's
00:24:29.120
providing something different um other Advantage definitely
00:24:34.919
performance um Ron Fernando couple of years ago he discovered that an API with
00:24:42.120
a grpc it's definitely faster than a res API around seven to 10 times faster um
00:24:49.640
this is mainly due to http2 and also the packing that protocol buffers implements
00:24:56.760
so we have like very lightweight messages disadvantages of using this technology well first of all definitely
00:25:04.640
implementation uh when you work with Ruby and you're probably working with rails applications maybe you are
00:25:10.120
familiar with how to do rails apis but if you get into this technology there's a learning corve and we should take that
00:25:16.559
into consideration it's definitely more complex also debugging we are using
00:25:22.399
binary messages so trying to see what's grong it's harder than what what we when we get when we work with Json requests
00:25:30.200
and responses um when should we use
00:25:35.399
erpc um well it depends I don't know you I don't know what you're doing so I
00:25:41.600
cannot give you a specific advice but some things to consider definitely Pro proof of
00:25:47.799
Concepts it's very good to implement something quickly uh I think even for
00:25:54.000
for this example what's faster for me to to build the sample and even more
00:26:00.039
complex code that I have built for in the past it's faster than building an API and building the documentation for
00:26:06.559
the API I'm building testing and more stuff so for probably for quick
00:26:12.480
communication uh proof of Concepts maybe it's a good option and what it excels is for multil
00:26:19.399
language systems if for some reason you have teams that need to work on different languages let's give a try in
00:26:26.880
this as you see generating the code and and the boiler pre that it generates it really speeds up uh the development
00:26:34.760
process and the other aspect where it shines is definitely microservices it
00:26:40.080
was the the tool to use when you were building microservices with with go a long time
00:26:46.360
ago and other suggestion that I've seen that could work in your case it's for
00:26:51.399
internal communication systems when you are not exposing a public API probably
00:26:56.919
it's wor doing something more okay let's do a binary communication and let's do just grpc instead of implementing a rest
00:27:02.760
API for our internal systems and also some it's a very good option for low
00:27:09.039
bandwidth networks why because the packages are way smaller than a Json
00:27:14.480
message um it's very good let's say for iot it's definitely a good example or
00:27:21.000
it's a good um implementation of this technology and also as last
00:27:28.880
as one of your Innovation tokens I don't know if you have heard about this term um if you have not I really recommend
00:27:36.279
you to visit this page Bing technology. Club um by Dan McKinley uh I think it
00:27:42.519
was like a presentation that he did and then he transcript the presentation into
00:27:47.600
this page long story short is that He suggests that we should have three uh
00:27:54.279
Innovation tokens when we're building something and we can use them for whatever we want do we want to use a new
00:28:00.279
database you spend one do you want to use another frontend framework you spend
00:28:05.880
other you only have one where you going to spend this token on probably you can try again you don't lose the only thing
00:28:13.039
you might lose is some time trying to see if this fits your needs and if it doesn't okay just move over and try with
00:28:19.480
or go back to to rest um so it's a conclusion and trying to
00:28:27.880
give um response to my question why is grpc more popular in in some languages than
00:28:34.200
in others um I don't know I I mean I I
00:28:39.840
don't have the exact answer but I have some some guesses that I want to discuss
00:28:45.039
with you uh the first one is that I the G gpcm protocol has really strong
00:28:53.720
ties with microservices so if you are not doing microservices
00:28:58.760
you're probably going to discard this technology because it doesn't fit your needs the other is that res apis are
00:29:06.919
enough most of the time and more than enough definitely dress is a very good
00:29:12.000
uh architecture we are we already are familiar with that architecture and we
00:29:17.679
know how to solve common problems in R apis so most of the time it's it's
00:29:23.039
exactly what we need R is very good and it's mature so no reason to change that
00:29:28.720
most of the time and the last point or the conclusion is in theory yeah it's
00:29:36.480
very tempting to have a system that it's seven or 10 times faster that's that's
00:29:41.960
exhausting that's that's uh exciting uh but a speed is not everything to be
00:29:47.279
honest uh there are more factors as we mentioned it's more complex to build probably hard to maintain probably
00:29:52.799
you're going to run issues that you are not familiar with so you won't be able to solve them as effectively as with
00:29:58.200
other Technologies and related to speed um one of the examples of the things
00:30:03.919
that come to my mind is the one the proof that speed is not everything is that that's why most of us don't go to
00:30:11.120
work by plane if we only care about a speed why not taking a plane or like a bullet train because there are more
00:30:17.600
things to consider not only that um thank you that's all what I have for for
00:30:31.960
um just some upcoming thought word events in person office hours October TW uh 10th
00:30:39.919
and 11th in London so if you're around you can sign up to chat with the team
00:30:46.240
and we're organizing an THB open Summit when you will be able to collaborate with the THB team on your first
00:30:53.760
collaboration on our open source gems that's going to be the 25th thank thank you