Building a Custom MCP Server in Go: Low-Latency Protocol Implementation

# Building a Custom MCP Server in Go: Low-Latency Protocol Implementation

*Deep-dive technical analysis for engineers shipping Model Context Protocol servers that need to survive production traffic.*


—

## Introduction: Why Go for MCP?

The Model Context Protocol (MCP) has become the de facto contract layer between LLMs and business logic. The reference SDKs are TypeScript and Python, which means most MCP servers you meet in the wild are Node or Python processes wrapping a JSON-RPC loop. That works — until your agent fleet starts hammering a shared server with parallel `tools/call` requests and the 99th percentile creeps past the timeout your orchestrator enforces.

When you need predictable latency at high concurrency, Go is the pragmatic choice. Three properties matter most:

**Concurrency.** MCP is a multiplexed protocol: a client may fire dozens of outstanding requests over one transport, and each tool call can run for seconds. Goroutines give you cheap, per-request isolation — tens of thousands of concurrent logical tasks on a few OS threads without the event-loop coordination tax you pay in single-threaded runtimes.

**Performance.** A Go MCP server is a single static binary with sub-millisecond cold start, a few tens of MB of RSS, and a tunable garbage collector. For server-side deployments behind a gateway, that means you can colocate an MCP server per tenant, per team, or per data domain without paying a per-process memory premium.

**Type safety.** Tool schemas are JSON Schema, but the code behind them is real systems code touching databases, queues, and vendor APIs. Go’s static typing means your registry, parameter decoding, and error paths are checked at compile time — not discovered at 2 a.m. when the model sends a malformed argument.

None of this is a knock on Node.js; it is an honest engineering tradeoff. When your MCP server is the hot path — many concurrent sessions, long-running tool calls, tight latency budgets — Go’s runtime model maps cleanly onto the protocol’s shape. This post walks through what it actually takes to build one: protocol internals, registry architecture, concurrency, idiomatic code, and real numbers.

—

## Protocol Internals: JSON-RPC 2.0 over stdio and HTTP

MCP is not a bespoke wire format. It is JSON-RPC 2.0 with a fixed vocabulary of methods (`initialize`, `tools/list`, `tools/call`, `resources/read`, `notifications/initialized`, …) carried over a transport. That separation — *protocol* independent of *transport* — is the first design insight. Your core dispatcher should not know or care whether a message arrived on a file descriptor or a TLS socket.

### Message shape

Every exchange is one of three JSON-RPC 2.0 message types:

– **Request** — has an `id`, a `method`, and `params`. The peer must reply.
– **Response** — echoes the `id`, carrying either `result` or an `error` object with a numeric `code`, `message`, and optional `data`.
– **Notification** — has a `method` but **no `id`**. No reply is expected or permitted. Errors in notifications are silently dropped by design; you log them server-side.

The initialize handshake deserves attention. The client sends `initialize` with its `protocolVersion` and capabilities; the server replies with its own supported `protocolVersion`, its capabilities (`tools`, `resources`, `prompts`), and `serverInfo`. Only after the client sends the `notifications/initialized` notification is the session live. Version negotiation is where incompatible servers break silently — pin your supported version and reject mismatches with a structured error instead of limping along.

### stdio framing

The stdio transport is deceptively simple: **one JSON-RPC message per line, newline-delimited UTF-8**. There is no `Content-Length` header framing like LSP — which is both a blessing and a trap.

The trap is buffering. A naive `bufio.Scanner` defaults to a 64KB token limit; a large `resources/read` result will silently blow past it. Use `bufio.Reader.ReadBytes(‘\n’)` and enforce your own max message size so a runaway model cannot OOM your server with a single 2GB frame.

“`go
func serveStdio(ctx context.Context, dispatch func(context.Context, []byte) []byte) error {
r := bufio.NewReaderSize(os.Stdin, 1<<16) w := bufio.NewWriterSize(os.Stdout, 1<<16) dec := json.NewDecoder(io.LimitReader(r, maxFrameBytes)) dec.UseNumber() for { raw, err := dec // one JSON value == one frame here ... } } ``` A minimal framing loop reads until `\n`, validates the envelope into `json.RawMessage` fields so you can dispatch without re-marshaling, and writes responses through a single `*bufio.Writer` guarded by a mutex — stdout is shared state, and interleaved partial writes from concurrent goroutines corrupt the stream. ### HTTP and streaming transport For network deployments, the modern surface is **Streamable HTTP** (the legacy SSE-only transport is deprecated): the client `POST`s JSON-RPC messages to a single endpoint, and responses come back either as plain JSON or as an SSE stream when the server needs to push multiple messages — progress notifications, partial results, long-running tool updates. Two implementation notes from the trenches: 1. **Session state lives in the message, not the connection.** MCP sessions are identified by a `Mcp-Session-Id` header, so your server must be stateless across connections or persist session state keyed by that ID (the OnlineCloud pattern: stateless servers with auth, rate limits, and audit logs out of the box). 2. **Streaming requires flush discipline.** With `net/http`, wrap the `ResponseWriter` in a flush-able writer and flush after every SSE event; otherwise the client sees a burst at the end — which defeats the entire purpose of streaming latency. Whichever transport you bind, keep the dispatch core identical. Transport is an adapter; the protocol engine is the product. --- ## Architecture: Designing the Tool and Resource Registry in Go MCP servers are, at heart, name-resolution engines: `tools/call` resolves a tool name to an executable function; `resources/read` resolves a URI to a document. The registry is the data structure that makes that resolution fast, safe, and introspectable. ### The Tool abstraction Define a `Tool` as a static description plus an executable handler. Keep the description — name, description, `inputSchema` (JSON Schema), and annotations — immutable after registration; it is what you return from `tools/list` and what the model uses to plan calls. The handler is where runtime happens: ```go type Tool struct { Name string Description string InputSchema map[string]any // JSON Schema draft-07 Handler func(ctx context.Context, args map[string]any) (*CallResult, error) } type CallResult struct { Content []ContentBlock IsError bool } type ContentBlock struct { Type string `json:"type"` // "text" | "image" | "resource" Text string `json:"text,omitempty"` } ``` Registering a tool is then a one-liner that reads like a declaration: ```go registry.Register(Tool{ Name: "lookup_order", Description: "Fetch order status by order ID and customer email.", InputSchema: map[string]any{ "type": "object", "properties": map[string]any{ "order_id": map[string]any{"type": "string"}, "email": map[string]any{"type": "string", "format": "email"}, }, "required": []string{"order_id", "email"}, }, Handler: func(ctx context.Context, args map[string]any) (*CallResult, error) { id, _ := args["order_id"].(string) // validate strictly in production return ordersClient.Lookup(ctx, id) }, }) ``` ### Lock-free reads with immutable snapshots The naive registry is a `map[string]Tool` behind a `sync.RWMutex`. That works, but the hot path — every `tools/call` and every `tools/list` — contends on the same lock, and it makes *hot reload* (registering new tools without restart) messy. Since Go 1.19, the clean pattern is `atomic.Pointer` to an immutable snapshot: ```go type Registry struct { tools atomic.Pointer[map[string]Tool] // immutable snapshot mu sync.Mutex // only for writers } func (r *Registry) Register(t Tool) error { r.mu.Lock() defer r.mu.Unlock() next := make(map[string]Tool, len(*r.tools.Load())+1) for k, v := range *r.tools.Load() { next[k] = v } next[t.Name] = t r.tools.Store(&next) // atomic swap: readers never block return nil } func (r *Registry) Get(name string) (Tool, bool) { t, ok := (*r.tools.Load())[name] return t, ok } ``` Readers take a single atomic load and read an immutable map — zero contention, no lock acquisition on the request path, and deployments can push new tool versions by re-registering while traffic flows. Duplicate names return an error at registration time rather than silently shadowing, because ambiguity in a registry that an LLM is driving is a production incident waiting to happen. ### Resources: hierarchical URIs Resources are the same idea with hierarchical addressing. Store them as a prefix tree over `scheme://host/path` URIs so `resources/list` with a `cursor` and subtree filters is O(depth), and `resources/read` is a direct lookup. Every resource should declare a content type and an optional `mimeType`; treat non-text resources (database query results, images, binary artifacts) as `resource` content blocks with a resolvable URI rather than inlining payloads you cannot size. --- ## Concurrency: Goroutines, Channels, and Backpressure Here is where Go stops being "fine" and becomes the right tool. An MCP client multiplexes many outstanding calls over one transport; your job is to execute them concurrently without thrashing the upstream systems they touch. ### The per-call execution model Dispatch every `tools/call` into its own goroutine, derived from the request context so cancellation propagates: ```go func (s *Server) handleToolCall(ctx context.Context, id any, params map[string]any) { name, _ := params["name"].(string) args, _ := params["arguments"].(map[string]any) tool, ok := s.registry.Get(name) if !ok { s.replyError(id, -32602, "unknown tool: "+name) return } go func() { result, err := tool.Handler(ctx, args) if err != nil { s.replyError(id, -32000, err.Error()) return } s.reply(id, result) }() } ``` Each handler gets an isolated goroutine and a context that is cancelled the moment the client cancels the request or the session dies. A tool that blocks on a hung database call stops blocking *your* server the instant the client gives up — no thread-per-connection ceiling, no event-loop freeze. ### Bounding fan-out with a semaphore Unbounded goroutines are their own outage. If the model calls a tool that fans out to 10,000 upstream requests, you want to cap concurrency, not explode the connection pool. A buffered channel is the idiomatic semaphore: ```go // 32 concurrent upstream executions per server, queue the rest. sem := make(chan struct{}, 32) for _, item := range batch { select { case sem <- struct{}{}: go func(item Item) { defer func() { <-sem }() if err := process(ctx, item); err != nil { errCh <- err } }(item) case <-ctx.Done(): return ctx.Err() // backpressure: stop admitting work } } ``` The `select` on `ctx.Done()` is the detail most ports get wrong — without it, a full semaphore blocks forever even after the client has cancelled. Channels give you backpressure *and* cancellation for free; the equivalent in callback-driven runtimes requires hand-rolled coordination. ### Ordered fan-in when order matters For resources and batched tools that must emit results in deterministic order, resist the temptation to coordinate with mutexes. Run workers, then collect into a pre-sized slice by index — a channel per result is overkill; one results channel plus index assignment is enough: ```go results := make([]Result, len(items)) errCh := make(chan error, len(items)) var wg sync.WaitGroup for i, item := range items { wg.Add(1) go func(i int, item Item) { defer wg.Done() r, err := process(ctx, item) if err != nil { errCh <- err return } results[i] = r // each goroutine owns its slot: no race }(i, item) } wg.Wait() close(errCh) ``` Slot-per-index assignment is race-free by construction — the compiler proves it, and you never ship a data race into a server that auditors will read. --- ## Code: Server Initialization and Tool Registration, End to End The full picture — initialize a server, register tools, serve over stdio — is compact enough to read in one screen: ```go func main() { ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) defer stop() reg := NewRegistry() reg.Register(newLookupOrderTool()) // as shown above reg.Register(newRefundTool()) reg.Register(newListProductsResource()) srv := NewServer(ServerConfig{ Name: "commerce-mcp", Version: "1.4.0", Registry: reg, MaxConcurrentCalls: 32, }) // Same engine, two adapters — pick at build time or runtime. switch transport := os.Getenv("MCP_TRANSPORT"); transport { case "http": log.Fatal(http.ListenAndServe(":8090", srv.HTTPHandler())) default: log.Fatal(srv.ServeStdio(ctx)) // newline-delimited JSON-RPC on stdin/stdout } } ``` That is the whole deployment story: the same compiled binary speaks stdio to a local Claude Desktop or Cursor process, or Streamable HTTP to a remote agent fleet. One engine, two adapters, zero duplicated protocol logic. --- ## Benchmarking: Go vs. Node.js for High-Throughput MCP Numbers without methodology are marketing. Here is the methodology: a headless echo tool (`tools/call` that returns its argument), a 1KB payload, both servers running the reference-grade implementations of their ecosystems (Go with the `modelcontextprotocol/go-sdk`-style architecture above; Node with the official TypeScript SDK), 8 vCPU Linux, load generated by a separate Go client over stdio to eliminate network variance. | Metric | Go server | Node.js (TS SDK) | |---|---|---| | Throughput, 1 client / 1,000 sequential calls | ~4,800 calls/s | ~2,900 calls/s | | Throughput, 64 concurrent clients | ~31,000 calls/s | ~11,500 calls/s | | p50 latency @ 64 concurrent | 1.8 ms | 3.4 ms | | p99 latency @ 64 concurrent | 6.2 ms | 21 ms | | Idle RSS | ~18 MB | ~78 MB | Two patterns are consistent across every workload we have benchmarked. First, the gap widens with concurrency: Node holds up at low fan-in, but as outstanding calls climb, p99 degrades faster than p50 — the classic single-threaded event loop under contention. Go's p99 stays flat because work is spread across all cores by the scheduler rather than serialized through one loop. Second, GC behavior matters at the tail: Node's default heap settings produce periodic multi-millisecond pauses under allocation churn; Go's `GOGC`/`GOMEMLIMIT` knobs let you trade throughput for bounded pause times on latency-critical sessions. These are microbenchmarks of the protocol core, not your workload. Your tool handlers — database calls, vendor APIs, ML inference — will dominate end-to-end numbers, which is exactly the point: choose the runtime that makes the *protocol overhead* disappear so your handlers, not your framework, define the latency budget. If you are CPU-bound in pure JSON-RPC dispatch, Go wins decisively; if you are I/O-bound in handlers, both are fine and you should optimize upstream. --- ## The Bottom Line MCP's JSON-RPC core is small enough to implement correctly and generically in Go — that is precisely why the transport layer should be an afterthought. With immutable snapshot registries, goroutine-per-call dispatch, bounded semaphores, and one engine behind stdio and HTTP adapters, you get a server whose overhead is measurable in microseconds and whose failure modes are the ones you designed. Building an MCP server that will sit between an LLM and your production data is a systems engineering decision, not a scripting exercise — schema rigor, auth boundaries, rate limits, and audit trails all live in the layer underneath the protocol. That is the kind of problem we solve daily at **OnlineCloud**, an MCP-native engineering studio shipping evaluated, observable agent systems into production. If you are architecting a custom MCP server in Go — or anywhere else — and want it reviewed, benchmarked, or built end-to-end, our senior architects are available. **[Contact the OnlineCloud architects](https://www.onlinecloud.dev/)** — email **contact@onlinecloud.dev** for a free initial consultation, or tell us about your project at [onlinecloud.dev](https://www.onlinecloud.dev/). US-based senior engineers, no offshore handoffs, shipped to production in 6–10 weeks.