gRPC & Protocol Buffers for Beginners

Protocol Buffers: the contract and the format

You describe your data and your service in a .proto file. A message is a set of fields (like a record). A service lists the calls (rpc) that clients can make.

syntax = "proto3";package inventory.v1;service InventoryService {  rpc CheckStock (CheckStockRequest) returns (CheckStockResponse);  rpc WatchStock (WatchStockRequest) returns (stream StockUpdate);   // server streaming}message CheckStockRequest {  string product_id = 1;  int32  quantity   = 2;}message CheckStockResponse {  bool   available       = 1;  int32  units_in_stock  = 2;  string warehouse_id    = 3;}

The numbers (= 1, = 2, = 3) are field numbers. They are what is actually sent over the network ("the wire"), not the field names. This is one reason Protobuf is small.

Then a code generator (protoc, the Protobuf compiler, or tools like Buf) creates client and server code in your languages (Go, Java, Python, TypeScript, C#, Rust and more):

Every team uses the same contract. In compiled languages the compiler (the tool that turns code into a program) catches mismatches before anything runs.

Why Protobuf is smaller and faster than JSON

The same data in two formats:

• No field names are repeated in each message. Only small field numbers are sent.
• Numbers are packed tightly (small numbers use fewer bytes, called varint encoding).
• Reading binary is much faster than reading text. Binary means raw bytes, not readable characters.

The trade-off: you cannot read it by eye. You need tools like grpcurl (like curl, but for gRPC) or Postman to look at it.

Evolving schemas safely

APIs change over time. Protobuf supports backward compatibility (new code can read old data) and forward compatibility (old code can read new data), if you follow a few rules:

• ✅ Adding new fields is safe. Old clients ignore fields they do not know. New clients see default values (such as 0 or an empty string) when an old server does not send the field.
• ✅ Removing a field is OK, but mark its number and name as reserved (the compiler then refuses to let anyone use them again):
  ProtobufCopy

  message CheckStockResponse {  reserved 3;  reserved "warehouse_id";  bool  available      = 1;  int32 units_in_stock = 2;}

• ❌ Never change a field's number.
• ❌ Never reuse an old field number for something new. Old data or old clients will read it wrongly.
• ❌ Do not change a field's type (like int32 to string).
• Put a version in the package name (inventory.v1) and create v2 for breaking changes.

gRPC runs on HTTP/2

gRPC sends its calls over HTTP/2 (lesson 3), a newer version of HTTP that sends binary data and can carry many requests at once. This gives gRPC:

• Multiplexing: many calls at the same time over one connection, without waiting for each other.
• Binary framing (messages are cut into small binary pieces) and header compression (repeated header data is made smaller).
• Streaming in both directions: data can keep flowing as a series of messages.

Four kinds of calls

Deadlines and cancellation

In gRPC, every call can carry a deadline: "I need an answer within 300 ms". A deadline is the latest time by which the answer must arrive. It travels with the request to downstream services (the services that your service calls in turn). If service A calls B with 300 ms left, and B spends 100 ms, B's call to C carries the remaining ~200 ms. (Your code must pass the call's context to the next call for this to work.) When the deadline passes, everyone stops working on that request.

This prevents wasted work. It also helps stop cascading failures (Part 7), where one slow service makes the services that call it slow too. Always set deadlines. gRPC has no deadline by default, so a call could wait forever.

gRPC also has its own status codes: OK, NOT_FOUND, INVALID_ARGUMENT, DEADLINE_EXCEEDED, UNAVAILABLE, PERMISSION_DENIED, RESOURCE_EXHAUSTED (rate limited) and more. They are similar in purpose to HTTP status codes.

Interceptors (middleware)

Interceptors are pieces of code that run before and after every call. They are the natural place for:

• authentication (checking tokens),
• logging and metrics,
• distributed tracing (following one request across many services, Part 8),
• retries with backoff (trying again after a growing wait, Part 7).

The load-balancing pitfall

A load balancer shares requests between several servers. gRPC keeps long-lived HTTP/2 connections and sends many requests over each one. This causes a problem with L4 load balancers (lesson 12). An L4 balancer works at the connection level. It picks a server once per connection and does not look at the requests inside. An L7 balancer reads each request.

Once a connection is assigned, every request on it goes to the same server, so load becomes very uneven.

Fixes:

• An L7 load balancer / proxy that understands HTTP/2 and balances per request (Envoy, NGINX, a service mesh, or a cloud L7 LB with gRPC support).
• Client-side load balancing: the gRPC client knows all server addresses (from DNS or service discovery) and spreads requests itself.

gRPC in the browser

Browsers do not give JavaScript the low-level HTTP/2 control that gRPC needs. Options:

• gRPC-Web, a browser-friendly version of gRPC. A proxy (such as Envoy) translates it to real gRPC.
• Connect, a newer protocol that works with gRPC and also works over plain HTTP with JSON.
• Most commonly: REST or GraphQL for browsers and mobile, gRPC between backend services.

REST vs gRPC

Alternatives

• Apache Thrift: created at Facebook. It has a similar idea, with its own data format and RPC framework.
• Apache Avro: popular in data pipelines and Kafka (a system for streams of events). The schema is stored together with the data.
• JSON-RPC: simple RPC that sends calls as JSON.
• MessagePack / CBOR: compact binary encodings of JSON-like data, without schemas.