building high performance apis in go using grpc and protocol buffers
TRANSCRIPT
Building High Performance APIs in Go with gRPC and
Protocol Buffers
Shiju Varghese medium.com/@shijuvar
GopherCon India 2017
Agenda
• gRPC
• Protocol Buffers
• Building APIs with gRPC
Monolithic Architecture
Catalog Management
Customer Accounts
Orders Management
Payment
Monolithic Database
Monolithic E-Commerce App
Shipment
eCom Store Web
Moving to MicroserviceseCom Store Web
Catalog Management
Customer Accounts
Order Management
Payment
Shipment
Store DB
Catalog DB
Customer DB
Order DB
Payment DB
Shipment DB
Communications on Microservices
• Inter-process communication between Microservices
• Serialisation and Deserialisation of Messages
• Scaling APIs into billions of APIs calls
• Building streaming APIs
• RESTful with JSON or RPC with Binary Encoding?
Why Not REST• Uses HTTP/1.x; Separate TCP Connection per
request
• Text on the wire; Not performance efficient
• Harder API Evolution
• Not Domain-Specific, Not Strongly-Typed
• Lack of Streaming capabilities
• High performance, open-source RPC framework
• Open source version of Google’s internal framework Stubby
• Uses Protocol Buffers as the IDL
• HTTP/2 for transport
• Bi-Directional streaming
• RPC is Efficient, Domain-Specific and Strongly-Typed
• Works across languages and platforms
• Google's language-neutral, platform-neutral, extensible mechanism for serialising structured data
• IDL - Describe once and generate interfaces for multiple languages
• Structure of the Request and Response
• Binary format for network transmission
• Supports multiple languages
Protocol Buffers
Communication between gRPC Server and Client app
Types of RPC Methods
• Simple RPC
• Server-side streaming RPC
• Client-side streaming RPC
• Bi-directional streaming RPC
gRPC Workflow
ProtoBuf Definitions
protoc Compiler Go
Ruby
Java
gRPC Server
gRPC Client Define 1 Compile 2
Implement 3
Generate Code
protoc --go_out=plugins=grpc
syntax="proto3";packageorder;
serviceOrderService{//AsimpleRPCrpcCreateOrder(Order)returns(OrderResponse){}}messageOrder{stringid=1;stringstatus=2;int64created_on=3;messageOrderItem{stringcode=1;stringname=2;floatunit_price=3;int32quantity=4;}
repeatedOrderItemorder_items=4;}messageOrderResponse{stringorder_id=1;stringerror=2;}
typeserverstruct{}
func(s*server)CreateOrder(ctxcontext.Context,in*pb.Order)(*pb.OrderResponse,error){//Implementationgoeshere}funcmain(){lis,err:=net.Listen("tcp",port)iferr!=nil{log.Fatalf("failedtolisten:%v",err)}//CreatesanewgRPCservers:=grpc.NewServer()pb.RegisterOrderServiceServer(s,&server{})s.Serve(lis)}
gRPC Server
gRPC Clientconn,err:=grpc.Dial(address,grpc.WithInsecure())iferr!=nil{log.Fatalf("Unabletoconnect:%v",err)}deferconn.Close()client:=pb.NewOrderServiceClient(conn)createOrders(client)
funccreateOrders(clientpb.OrderServiceClient){order:=&pb.Order{Id:uuid.NewV4().String(),Status:"Created",CreatedOn:time.Now().Unix(),OrderItems:[]*pb.Order_OrderItem{&pb.Order_OrderItem{Code:"knd100",Name:"KindleVoyage",UnitPrice:220,Quantity:1,},&pb.Order_OrderItem{
Code:"kc101",Name:"KindleVoyageSmartShellCase",UnitPrice:10,Quantity:2,},},}resp,err:=client.CreateOrder(context.Background(),order)}
Installation # Install Protocol Buffers compiler
$ brew install protobuf (macOS)
(Or download from https://github.com/google/protobuf/releases/tag/v3.0.0)
# Install Go Plugin
$ go get github.com/golang/protobuf/protoc
$ go get github.com/golang/protobuf/protoc-gen-go
# Install Go RPC Framework
$ go get google.golang.org/grpc
medium.com/@shijuvar linkedin.com/in/shijuvar email: [email protected]
THANK YOU
Articles:Building High Performance APIs In Go Using gRPC And Protocol Buffers
Benchmarking Protocol Buffers, JSON and XML in Go