How gRPC Works Under the Hood

You've probably seen gRPC described as "REST but faster." That framing is technically true and practically useless. The speed isn't magic — it comes from specific design decisions about serialization, transport, and connection management that have real implications for how you build and operate services. If you don't understand what's actually happening, you'll hit production problems that feel inexplicable.

Let's walk through the full stack, from the .proto file to the bytes on the wire.

The mental model

gRPC is a layered system. Each layer has one job, and the layers compose cleanly:

The .proto file is the contract. Code generation produces typed stubs so you never construct HTTP requests by hand. Protobuf handles serialization into a compact binary format. gRPC adds a thin framing layer on top of Protobuf to delimit messages on the wire. HTTP/2 provides multiplexed, bidirectional streams. TCP does the rest.

Each of these is independently interesting, and each is a place where things can go wrong.

Protocol Buffers: encoding that doesn't waste bytes

Start with the .proto definition:

message UserRequest {
  string user_id = 1;
  int32  page    = 2;
}

When you serialize this, Protobuf doesn't write "user_id": "abc123". There are no field names on the wire. Each field is encoded as a tag followed by a value. The tag encodes two things: the field number and the wire type, packed into a single varint:

tag = (field_number << 3) | wire_type

For user_id (field 1, wire type 2 = length-delimited): tag = (1 << 3) | 2 = 0x0A.

Wire types tell the parser how to read the following bytes:

Varints are the clever part. Each byte uses 7 bits for the value and 1 bit (the MSB) as a continuation flag. Small integers — the kind that dominate in real APIs — encode in one or two bytes. The integer 150:

150 in binary: 10010110
Split into 7-bit groups (little-endian): 0010110 | 0000001
With continuation bits: 10010110 00000001
On the wire: 0x96 0x01

The value 1 encodes as a single byte 0x01. Status codes, boolean flags, small counts — all single byte.

Don't take my word for any of it. Encoding the UserRequest above with protobufjs 7.5.4:

{ userId: "abc123" }             -> 0a 06 61 62 63 31 32 33   (8 bytes)
{ page: 150 }                    -> 10 96 01                  (3 bytes)
{ page: 1 }                      -> 10 01                     (2 bytes)
{ userId: "abc123", page: 2 }    -> 0a 06 61 62 63 31 32 33 10 02  (10 bytes)

Note the key: userId, not user_id. protobufjs camel-cases field names, and it does it silently — pass { user_id: "abc123" } and you get a zero-byte buffer, with verify() returning null as if nothing were wrong. That is a genuinely nasty way to lose a field.

First byte 0x0a on field 1, exactly as the tag formula predicts. 150 on field 2 comes out as 10 96 01 — the 10 is the tag, then the two varint bytes. The string user_id appears nowhere in the output.

That last message is 10 bytes. {"user_id":"abc123","page":2} is 29. So: smaller, yes — 2.9x on this particular message. The ratio is entirely a function of your field names and value distribution, so measure your own payloads rather than trusting anyone's blanket multiplier, including this article's.

One important consequence: field order doesn't matter and missing fields are zero-valued. 10 07 0a 01 78 puts field 2 before field 1 and decodes to exactly what the in-order 0a 01 78 10 07 decodes to; feed the decoder zero bytes and you get { userId: "", page: 0 }. Splice in a tag for field 999 and the decoder walks past it. This is what makes schema evolution safe — you can add fields without breaking existing clients, as long as you don't reuse field numbers.

HTTP/2: one connection, many streams

HTTP/1.1 has a head-of-line blocking problem. Requests share a connection sequentially, so a slow response blocks everything behind it. HTTP/2 fixes this with streams — independent, bidirectional channels multiplexed over a single TCP connection.

Each gRPC call gets its own HTTP/2 stream. Streams are identified by an integer ID (client-initiated streams use odd numbers; server-initiated use even). The stream lifecycle is independent — a long-running server-stream RPC and a quick unary RPC can share the same connection without blocking each other.

HTTP/2 also compresses headers using HPACK. On repeated calls to the same service, headers like :authority, content-type: application/grpc+proto, and user-agent are sent once and referenced by index on subsequent requests. For microservices making thousands of calls per second, this is a real throughput win.

The 5-byte framing layer

Here's the part most explanations skip. HTTP/2 DATA frames can be split or coalesced arbitrarily — the framing boundary has nothing to do with message boundaries. gRPC adds its own framing layer inside the DATA frames:

┌────────────┬──────────────────────────┬──────────────────────┐
│  1 byte    │  4 bytes                 │  N bytes             │
│  Compress  │  Message-Length (uint32) │  Serialized message  │
│  Flag      │  big-endian              │  (Protobuf bytes)    │
└────────────┴──────────────────────────┴──────────────────────┘

Straight out of PROTOCOL-HTTP2.md: Compressed-Flag is 0 / 1 ; encoded as 1 byte unsigned integer, and Message-Length is the length of the message encoded as 4 byte unsigned integer (big endian). So 0 for uncompressed and 1 for compressed, with the codec named by the grpc-encoding header (gzip, deflate, snappy, identity) and the peer's acceptable codecs advertised in grpc-accept-encoding. A 4-byte length field caps a single message at just under 4GB; grpc-go's actual default is 4MB (defaultServerMaxReceiveMessageSize = 1024 * 1024 * 4).

The parser reads 5 bytes to know exactly how many more bytes to read for the message. No delimiters, no scanning. Multiple messages can follow each other back-to-back in a streaming call.

Request headers are standard HTTP/2 HEADERS frames with specific pseudo-headers:

:method: POST
:path: /helloworld.Greeter/SayHello
:authority: example.com:443
content-type: application/grpc+proto
grpc-timeout: 5S
te: trailers

The :path encodes the service and method directly. No URL routing, no query parameters. grpc-timeout is a value plus a unit character, and S is seconds. te: trailers is required by the spec, which annotates it as "used to detect incompatible proxies" — an intermediary that strips or rejects it is one that will also mangle the trailers gRPC needs to deliver status.

Status lives in trailers, not in the HTTP status

This trips people up. A gRPC server answers 200 OK even on errors — the spec is explicit that it "uses status 200 (OK)" for error responses, and that a non-application/grpc content-type should get a 415 precisely so plain HTTP/2 clients don't misread a gRPC error as success. The actual RPC status comes in HTTP/2 trailers: a HEADERS frame carrying grpc-status, with END_STREAM set, sent after the DATA frames.

grpc-status: 5
grpc-message: user not found

grpc-status: 5 is NOT_FOUND. A 200 OK with grpc-status: 2 (UNKNOWN) is a failed RPC. Status must be sent in the trailers even when it's 0.

There's one shape worth knowing: Trailers-Only, permitted for calls that produce an immediate error. It's a single HEADERS block with status, content-type and trailers and no DATA at all — so "trailers always come after the body" isn't quite true, and a parser that assumes a body will trip on it.

A non-200 HTTP status, on the other hand, means something other than your gRPC server answered — a proxy, a mesh sidecar, a misrouted request. The spec tells implementations to expect broken deployments to do this and to synthesise a status for the application layer. Which is the observability point: if you're monitoring HTTP status codes at your load balancer, you are watching the health of your infrastructure, not of your RPCs. Those live in trailers.

The Channel abstraction: not just a connection

From the client SDK perspective, you don't manage TCP connections — you manage a Channel. A Channel is a logical connection to a service name. Underneath, a name resolver turns that name into a list of addresses and the load balancing policy creates a subchannel per address it decides to use; each subchannel owns at most one HTTP/2 connection at a time.

# This creates a Channel, not a TCP connection
channel = grpc.insecure_channel('user-service:50051')
stub = UserServiceStub(channel)
 
# Each call reuses the channel's HTTP/2 connection
response = stub.GetUser(UserRequest(user_id="abc"))

The Channel state machine has exactly five states, per gRPC's connectivity-semantics-and-api.md: IDLE, CONNECTING, READY, TRANSIENT_FAILURE, SHUTDOWN. When a connection breaks, the channel reconnects on its own with exponential backoff — that part is automatic. What is not automatic is retrying the RPC that died with it: gRPC retries require a retry policy in the service config, and without one an UNAVAILABLE is handed straight back to your code. Confusing "the channel recovers" with "my call was retried" is a good way to lose requests quietly.

Streaming: four call types

gRPC supports four call patterns, all built on the same HTTP/2 stream primitive:

For bidirectional streaming, both sides send DATA frames asynchronously. HTTP/2's flow control prevents either side from overwhelming the other — each DATA frame consumes from the receiver's flow control window, and the receiver periodically sends WINDOW_UPDATE frames to allow more data.

service ChatService {
  rpc Chat(stream ChatMessage) returns (stream ChatMessage) {}
}

On the wire, this is just one HTTP/2 stream with DATA frames flowing in both directions until one side sends a HEADERS frame with END_STREAM (the trailers).

The load balancing footgun

Here's where gRPC in Kubernetes bites people. gRPC's default load balancing policy is pick_first: absent a policy in the service config, the client picks one resolved address and sends all traffic there. One connection, all requests. I'm leaning on Datadog's Lessons learned from running a large gRPC mesh at Datadog (April 2024) for the rest of this section — every number below is theirs, from a fleet actually running at that size.

In Kubernetes, a standard ClusterIP service gives you one virtual IP. All gRPC traffic goes to that one IP, and kube-proxy distributes connections at the TCP level. With HTTP/2, you typically have one long-lived TCP connection per client. Kube-proxy sees one connection → it goes to one pod. Your fleet of 20 pods and all traffic hits pod 1.

The fix: use a Kubernetes headless service (DNS returns individual pod IPs) plus gRPC's round_robin policy:

channel = grpc.insecure_channel(
    'dns:///user-service-headless:50051',
    options=[(
        'grpc.service_config',
        '{"loadBalancingConfig": [{"round_robin": {}}]}',
    )],
)

With round_robin, the channel creates a subchannel to each resolved IP and rotates RPCs across them. Each pod gets its fair share.

But there's a second problem: gRPC only forces a re-resolution when a connection closes. There is no background poll to fall back on — grpc-go's DNS resolver waits for the next ResolveNow and merely rate-limits itself to one resolution per 30 seconds ("Success resolving, wait for the next ResolveNow. However, also wait 30 seconds at the very least to prevent constantly re-resolving," per internal/resolver/dns). Scale out from 5 pods to 10 and the new pods are invisible to every client that already has a healthy connection. The fix Datadog landed on: set MaxConnectionAge on the server to force periodic connection churn.

grpc.NewServer(
    grpc.KeepaliveParams(keepalive.ServerParameters{
        MaxConnectionAge:      5 * time.Minute,
        MaxConnectionAgeGrace: 30 * time.Second,
    }),
)

Roughly every 5 minutes — grpc-go adds jitter to MaxConnectionAge, so it isn't on the dot — the server gracefully drains the connection with a GOAWAY. Clients reconnect, re-resolve DNS, and pick up new pods. Datadog picked five minutes as "a time period that strikes a balance between limiting connection churn and re-resolving often," running gRPC "on the order of tens of millions of requests per second between tens of thousands of pods."

Practical implications

Deadlines, not timeouts. gRPC models a point in time, not a duration — and by default it sets no deadline at all, so a client with no explicit deadline will wait effectively forever. Once you set one, it propagates to downstream calls in the implementations that support propagation. If your edge service has a 2-second deadline and an internal call takes 1.9 seconds, the remaining 100ms budget is what the next hop gets. This is how you avoid the cascading-timeout problem where each service helpfully starts a fresh clock.

Keepalive configuration is mandatory in production. A broken TCP connection can sit ESTABLISHED for a very long time with nothing flowing, because nothing tells the application until the OS gives up retransmitting. Datadog puts a number on it for their fleet: "The default for our Linux distribution is 15 retransmitted packets limit for any socket (this takes 15 minutes)." Configure keepalive pings on both client and server:

// Client
grpc.WithKeepaliveParams(keepalive.ClientParameters{
    Time:    30 * time.Second, // send ping after 30s idle
    Timeout: 5 * time.Minute,  // deliberately high — see below
})
 
// Server
grpc.KeepaliveParams(keepalive.ServerParameters{
    Time:    30 * time.Second,
    Timeout: 5 * time.Minute,
})

That Timeout looks absurd until you know what it actually controls. On Linux, enabling keepalive also makes gRPC set the socket's TCP_USER_TIMEOUT to the keepalive timeout (grpc-go does this in internal/syscall; it is a no-op on platforms without the option), and Datadog's conclusion is that TCP_USER_TIMEOUT, not the ping, is what actually detects the dead connection. Their recommendation is therefore the opposite of the intuitive one: "set the gRPC keepalive timeout value to a high value, such as five minutes, so it will never trigger because of the client timeout setting."

The failure they watched people walk into was on the other knob. Users lowered keepalive Time further and further to catch drops sooner, and ran into the server's ping-enforcement policy: gRPC servers default to a minimum ping interval of five minutes, hand out a strike per ping inside that window, and on the third strike close the connection with a Too Many Pings error, failing every in-flight query on it. Tuning Time down aggressively trades one class of failure for another.

Interceptors are the middleware story. gRPC has no middleware in the HTTP sense. The equivalent is interceptors — functions that wrap the stub or server handler. Every service should have at least a logging interceptor (to record grpc-status in traces, not HTTP status) and a deadline interceptor.

The 4MB message limit is real. grpc-go's defaultServerMaxReceiveMessageSize and defaultClientMaxReceiveMessageSize are both 1024 * 1024 * 4. Hit it and you get RESOURCE_EXHAUSTED. Large file transfers don't belong in unary RPCs — use client streaming and chunk the payload, or offload to object storage and pass a reference.

The underlying thing to internalize: gRPC is not a REST replacement with a different wire format. It's a different model — typed contracts, streaming as a first-class citizen, connection-level flow control, and deadline propagation baked into the protocol. The performance comes from using HTTP/2 correctly, not from Protobuf alone. The operational complexity comes from long-lived connections doing things TCP load balancers never expected.