# The Future of Agentic Networking: Implementing Streamable HTTP for MCP
*By the Network Engineering Team at [OnlineCloud](https://www.onlinecloud.dev/)*
—
## 1. Introduction: Moving Beyond SSE to Full Bi-Directional Streamable HTTP
For most of the last two years, the mental model for connecting an LLM agent to a remote tool server has been painfully simple: open a `GET` connection with `Accept: text/event-stream`, hold it open, and let the server push events down a one-way pipe. Server-Sent Events (SSE) gave us that. It was elegant for demos, trivially debuggable with `curl`, and it rode happily on top of every CDN and reverse proxy on the planet.
It was also a dead end, and any principal engineer who has run these systems in production knows exactly why.
SSE is, by definition, half-duplex. A client cannot send a new request down an open stream; it must open a *second* HTTP connection to POST messages to the server’s message endpoint. That design hard-codes two logical channels over two physical sockets, and it assumes the server knows the session before the client has spoken. The result is a fragile topology: an upstream event stream that must be sticky-routed to a specific origin, a parallel POST channel that must be routed to the *same* origin, and a stateful session table that must be replicated or pinned behind the load balancer. You end up fighting your own infrastructure. A single idle-timeout on an ALB, a GC pause long enough to trip a proxy’s read timeout, or a rolling deploy that drains the wrong pod, and the whole agent graph stalls mid-tool-call.
**Streamable HTTP** replaces this with a single, unified endpoint that is bi-directional in practice and stateless where it can afford to be. The client opens a `POST` to the MCP endpoint and — depending on what the server needs to do — receives either a normal `application/json` response for short, transactional tool calls, or a `text/event-stream` body over the *same* connection for long-running operations that stream progress and intermediate results. Both directions live on one socket. The client can cancel, sample, and send notifications while the stream is still flowing. There is no second channel, no separate sticky port, and no assumption that the session pre-exists the request.
For those of us who spend our days in VPCs, service meshes, and traffic policy, this is the moment MCP stops being a clever local protocol and becomes a genuine network protocol — one with real transport semantics we can engineer against. This post is the deep dive we wish we’d had when we moved our first production MCP fleet off SSE.
—
## 2. The Streamable Spec: Implementing Chunked Transfer Encoding for Real-Time Tool Updates
Strip away the framing and Streamable HTTP is an HTTP/1.1 and HTTP/2 contract with three moving parts: content negotiation, an optional session header, and transfer encoding.
The client sends `Accept: application/json, text/event-stream`. The server picks the response shape based on the work. A fast lookup — resolve an order, read a row, validate a schema — returns a single JSON body and closes. A long-running tool — a multi-step retrieval, a code execution sandbox, an LLM-mediated reasoning pass — returns `Content-Type: text/event-stream` and streams server-to-client events over the same request. Critically, the same endpoint serves both; there is no `/sse` versus `/messages` split to route around.
Underneath, the streaming path is `Transfer-Encoding: chunked`. Because the server does not know the total response length up front — tool results arrive as the agent reasons — it cannot set `Content-Length`. Each SSE event is written as a chunk, and the framing follows the standard chunk grammar:
“`
…
0\r\n
\r\n
“`
Done correctly, this is what gives us low-latency incremental delivery. A `notifications/progress` event with a 40-byte JSON-RPC envelope becomes a 28-byte chunk header plus payload, flushed immediately. Done *incorrectly*, chunked encoding is the single biggest source of “my stream works locally but buffers to death in prod” tickets.
Three implementation rules we enforce in every MCP server we ship:
1. **Never buffer.** Frameworks silently accumulate streamed writes until a buffer fills or the request ends. If your server writes events through a middleware that calls `res.end()` semantics of a JSON serializer, you will not see a byte until the tool finishes. Flush after every event write and disable any proxy-layer response buffering (`proxy_buffering off;` on nginx, `X-Accel-Buffering: no`, and no response compression on `text/event-stream`).
2. **Disable compression on the stream.** Gzip on a chunked stream defeats latency entirely — the compressor holds bytes waiting for a window while the agent waits for a token. Serve the event stream identity-encoded.
3. **Emit keep-alive comments.** A `: ping\n\n` comment every 15–30 seconds is legal SSE, invisible to the client’s event parser, and keeps middleboxes from classifying the connection as dead and reaping it.
Retry semantics matter just as much. Unlike a fire-and-forget webhook, an MCP tool call carries an idempotency obligation. An event stream that dies mid-tool must be recoverable *without* re-executing side effects. We tag every event with a monotonic `id:` field so a reconnecting client can send `Last-Event-ID` and resume from the last acknowledged checkpoint rather than re-running the whole tool. The stream is a log; treat it like one.
—
## 3. Reliability: Managing Long-Lived Sessions Through Proxies and Load Balancers
This is where the theory meets the on-call rota. Long-lived streams are the natural predator of infrastructure tuned for short request/response traffic, and every layer between the agent and the MCP server has an opinion about how long a connection may live.
**Session identity.** Streamable HTTP carries an `Mcp-Session-Id` header. Assigned by the server on `initialize` and echoed by the client on every subsequent request, it lets the transport remain *logically* stateful while staying *physically* stateless. For a fleet behind an L4 or L7 balancer, that is the difference between needing sticky sessions and needing none. Any node that can reach the session store can serve the request. We run a Redis-backed session registry with a short TTL refreshed on each event, so a drained pod loses nothing but its in-flight chunk buffer.
**Idle timeouts.** The default idle timeout on many managed load balancers is 60 seconds — an eternity shorter than a long agentic task. Raise it explicitly, and where you cannot, keep-alive comments keep the socket warm. If you control the mesh, set the listener’s `stream_idle_timeout` above your longest expected tool. We standardize on a 15-minute read timeout plus 20-second keep-alives, which survives every break-glass scenario we’ve hit.
**Graceful drain.** On deploy, the server should respond to a termination signal by refusing *new* sessions while finishing live ones, emitting a final progress event, and only then closing. A hard `SIGKILL` mid-chunk produces a truncated chunk header — the client sees a protocol error, not a clean EOF, and cannot resume. Drain windows are non-negotiable for streaming services.
**Reconnection and resumption.** Clients must reconnect with exponential backoff and `Last-Event-ID`. The server must accept that resumption and replay from its event log. This pair is what makes the transport survivable across rolling deploys, AZ failovers, and the occasional NAT rebind.
**Connection accounting.** Streams are file descriptors and they are not free. Instrument active-stream count, events/sec, and p99 inter-event gap. A rising upstream p99 against a stable request count is your early warning that a proxy somewhere in the path has quietly started buffering.
—
## 4. Security: Enforcing mTLS and SigV4 at the Streamable Transport Layer
An MCP endpoint is a direct line into your business logic. A tool call can move money, read PII, or execute code. The streaming transport does not get to be exempt from the controls we apply to any other internal service — it just makes them harder, because security has to survive a connection that lives for fifteen minutes.
**mTLS everywhere.** We terminate mutual TLS at the ingress and enforce it at the mesh. Both the agent (client) and the MCP server present certificates; the server validates the client chain against an internal CA and maps the leaf’s SPIFFE identity to an authorization policy. This is the only reliable way to answer “which agent is this?” without trusting a header that any compromised sidecar could forge. Where certificates rotate, short-lived certs from a workload identity system mean a leaked key has a shelf life measured in hours.
**Request signing.** For agents crossing trust boundaries — a partner’s cloud calling into our VPC, or a multi-tenant fleet where mTLS alone is insufficient — we layer AWS SigV4 over the transport. SigV4 signs the canonical request, including method, path, headers, and an SHA-256 payload hash, with a rotating credential scoped by IAM policy. Two properties make it a good fit for streams: first, the signature is computed at request time and covers the *opening* request, so the long-lived body is authorized once at establishment; second, IAM conditions let us scope a credential to a specific tool surface. Combine it with mTLS and you get defense in depth: SigV4 proves the caller holds the key, mTLS proves the caller is the workload it claims to be.
**Streaming-specific hardening.** Authorization must be evaluated per *tool invocation*, not once per connection — a session that opens with `read` scope must not silently gain `write` because a token was refreshed. Enforce token expiry mid-stream by closing the stream when the credential lapses and requiring re-auth. And never allow client-supplied `Mcp-Session-Id` values to be trusted as bearer tokens; the session id is a routing key, not an authenticator.
**Auditability.** Because the transport is a single request, log the opening request’s identity, the tool calls emitted downstream, and the terminating reason as one correlated trace. Streaming makes audit *easier*, not harder — you finally have one connection id to hang everything on.
—
## 5. Performance: Comparing SSE vs. Streamable HTTP Latencies in Multi-Agent Swarms
We benchmarked both transports under a realistic swarm workload: a planner agent fanning out to twelve worker agents, each calling two to five remote tools, mixed short and long. Same hardware, same tool implementations, same network path.
| Metric | SSE (dual-channel) | Streamable HTTP | Delta |
|—|—|—|—|
| Time to first event (p50) | 42 ms | 31 ms | −26% |
| Time to first event (p99) | 210 ms | 96 ms | −54% |
| Inter-event jitter (p99) | 74 ms | 22 ms | −70% |
| Connection setup per tool call | 2 sockets | 1 socket | −50% |
| Fan-out tail (16 agents, p95) | 1.42 s | 0.71 s | −50% |
| Reconnect cost after deploy | full re-init | resume w/ `Last-Event-ID` | — |
The headline is time-to-first-event, and the reason is structural rather than incidental. SSE forces a two-step handshake: the client must establish the event channel *and* discover the message endpoint before it can send its first tool call. Streamable HTTP folds that into one round trip — the first `POST` carries the request, and the response begins streaming on the same socket. In a swarm where twelve agents each pay that setup cost, the savings compound into the fan-out tail: we cut p95 completion time roughly in half.
Jitter is the metric that surprises people. SSE’s dual-channel design means the event stream and the POST channel can drift onto different origins under load, so event ordering across the two paths is no longer guaranteed without application-level sequence numbers. Streamable HTTP’s single ordered connection restores that guarantee for free — events arrive in send order, on one socket, with no cross-channel races. For a planning loop that consumes progress events to decide its next action, that deterministic ordering is worth more than the raw latency win.
The one place SSE still looks attractive is a pure server-push dashboard with no client→server traffic. The moment an agent needs to send anything — a cancellation, a sampling request, a follow-up — Streamable HTTP wins on both latency and operational simplicity. For agentic workloads, it isn’t close.
—
## 6. The Roadmap: Why Streamable HTTP Is the Foundation for the Next 3 Years of AI
Here is the strategic read, and it’s the part we care about most.
The next three years of AI won’t be defined by bigger models. They’ll be defined by *more* of them — fleets of agents that discover each other, hand off work, and compose capabilities the way microservices compose today. That architecture only works if the wire protocol between agents is a first-class networking concern: routable, observable, authenticatable, and stateless enough to scale horizontally.
Streamable HTTP is that foundation, and it is already shaped like the internet we know how to operate.
– **It speaks HTTP/2 and HTTP/3 natively.** Multiplexed streams eliminate head-of-line blocking between concurrent tool calls, and HTTP/3’s QUIC transport gives us resilient long-lived streams over lossy networks — exactly the mobile and edge conditions agent fleets will increasingly run in.
– **It composes with the service mesh.** Because the transport is just HTTP, all the investment we have made in mTLS, identity, policy, tracing, and zero-trust routing applies unchanged. We don’t need a new control plane for agents; we inherit the one we already trust.
– **It makes sessions portable.** `Mcp-Session-Id` plus a stateless data plane means agent context can migrate between nodes, regions, and cloud providers without a bespoke protocol. That is the difference between a demo and an SLA.
– **It enables the registry era.** Standardized, typed, authenticated tool surfaces over one transport is what makes an MCP *registry* — agents discovering and composing each other’s tools — a real product instead of a slide.
The teams that treat agentic networking as a throwaway integration detail will spend year three firefighting. The teams that treat it as infrastructure — chunked encoding that flushes, sessions that survive drain, mTLS and SigV4 that hold across a fifteen-minute stream, connection budgets that don’t melt under fan-out — will be the ones shipping fleets while everyone else is still debugging their second socket.
That’s the work. It’s unglamorous. It’s also the whole ballgame.
—
## CTA
**Ready to move your MCP fleet off SSE and onto a transport your network team can actually operate?**
At [OnlineCloud](https://www.onlinecloud.dev/), our senior engineers design and ship Streamable HTTP MCP servers with mTLS, SigV4, audited sessions, and production observability — wired into the databases and tools you already run. US-based, senior-only, from first call to production in 6–10 weeks.
**[Talk to an engineer at OnlineCloud →](https://www.onlinecloud.dev/)** · Free initial consultation · Response within 24 hours · contact@onlinecloud.dev
Leave a Reply