Blog
How Much Latency Does an MCP Gateway Add? 6.6 Milliseconds in Our Latest Test.
Our latest test measured 6.6 ms of added latency per tool call on Hetzner with an AMD Ryzen™ 5 3600, with the gateway and the upstream server on MCP revision 2026-07-28. The same build adds 9.6 ms against a 2025-era server. We made 21,900 timed calls across three repetitions. Here are the setup, exact figures, and raw data.
· Updated · Erik Jonsson Thorén, Founder, Gatana
Short answer: Gatana added 6.6 milliseconds per tool call in our test on 7 October 2026, with the upstream MCP server on protocol revision 2026-07-28. Against a server on the 2025 revisions, the same build added 9.6 ms. The test ran on Hetzner with an AMD Ryzen™ 5 3600 processor. The all-2026 gateway path took 8.1 ms at the median, against 1.5 ms direct.
This is the third run of the day on the same rig. The first measured the previous build. The second measured the release that added protocol revision 2026-07-28, and the third a release that keeps the caller lookups in the gateway process for 30 seconds. Each run kept the unchanged paths as a control.
The difference between the two upstream figures is the handshake. The 2026-07-28 revision has none: the gateway sends the tool call and nothing else. On the 2025 revisions the gateway first sent initialize, initialized, and a GET probe before every call. That cost about 4 ms inside one cluster, and it costs more the farther away the upstream server is.
The test used warm sequential calls from inside the same cluster. It measures this setup. Client location, public routing, upstream distance, and concurrent load can change the result.
What we measured
- Target. A minimal MCP server with one tool,
echo, that returns its input. It speaks both protocol eras. A second deployment of the same file has 2026-07-28 turned off, so it looks like a 2025-era server to the gateway. - Hardware. The test ran on Hetzner with an AMD Ryzen™ 5 3600 processor.
- Position. The probe, gateway, and both echo servers ran inside the same cluster. The test excludes a laptop-to-cluster internet round trip.
- Paths. Direct to echo through its public hostname, and through the gateway to each echo server. The gateway checks identity and access, resolves the tool, connects to the upstream server, calls the tool, and records the call.
- Protocol. MCP over streamable HTTP, with kept-alive connections and 20 warm-up calls per run. Every gateway path ran twice: once with a 2025-era client that initializes a session, and once with a 2026-07-28 client that sends no handshake. Discovery and warm-up calls were excluded from the timed samples.
- Repetitions. Three interleaved sets. Each set had 1,000 direct public echo calls per client era, 1,000 gateway echo calls per combination of client and upstream era, 500 successful built-in calls per client era, and 300 direct internal echo calls.
- Validation. All 21,900 timed calls succeeded. We checked HTTP status, JSON-RPC errors, tool errors, and whether each echo result matched its input.
The per-run figures are in gateway-latency-2026-10-07-auth-cache.json.
Results
Times are milliseconds. The rows below use the 2026-07-28 client for the all-2026 path and the 2025-era client for the all-2025 path. The overhead rows subtract the direct path median of the same client era from the gateway path median in each repetition.
| Measurement | Run 1 | Run 2 | Run 3 |
|---|---|---|---|
| Direct public echo, median | 1.5 | 1.4 | 1.5 |
| Gateway echo, 2026-07-28 upstream, median | 7.9 | 8.3 | 8.1 |
| Gateway echo, 2026-07-28 upstream, 95th percentile | 26.1 | 21.0 | 46.0 |
| Gateway echo, 2026-07-28 upstream, 99th percentile | 38.2 | 38.5 | 65.5 |
| Gateway median minus direct median, 2026-07-28 upstream | 6.4 | 6.9 | 6.6 |
| Gateway echo, 2025-era upstream, median | 12.2 | 10.9 | 10.5 |
| Gateway median minus direct median, 2025-era upstream | 10.7 | 9.6 | 9.2 |
| Successful built-in call, median | 6.9 | 7.1 | 7.5 |
| Direct internal echo, median | 0.5 | 0.5 | 0.5 |
The median of the three overhead estimates for the all-2026 path, 6.4, 6.9, and 6.6 ms, is 6.6 ms. For the 2025-era upstream it is 9.6 ms. On the build without the cache, the same two paths measured 9.5 and 13.5 ms on the same rig, and the direct paths did not move. These are differences between path medians. They are not the median or 95th percentile of individual request overhead.
The medians fell and the tails did not. The 95th percentile of the all-2026 gateway path was 21 to 46 ms across the three runs, against 22 to 23 ms in the previous run. The gateway container in this test runs with a CPU limit of half a core and was at that limit in both runs. The faster path raises the request rate on the same capped container, which we take to be the cause. We did not test that.
The era of the client made no difference. A 2025-era client and a 2026-07-28 client saw the same medians on every path, within 0.4 ms. The saving comes from the upstream side alone.
What the caller cache changes
Every call through the gateway identifies the organization, the token, and the user behind it, with their profiles and teams. Until this release that was six database queries per call, about 3 ms in our traces. The gateway now keeps the result of those lookups in its own process for 30 seconds, so a client that makes many calls in a row pays for the queries once per 30 seconds. The trade-off is that a revoked token or a disabled user stays valid on a gateway replica for up to 30 seconds.
Our traces show the saving directly. Before the release, the organization lookup took 1.4 ms and the token and user lookups 1.3 to 1.6 ms per request. After it, both steps take 0.04 ms and run no query. A typical all-2026 request spent 5.7 to 5.9 ms on the gateway, against 9.1 to 9.6 ms before.
What MCP 2026-07-28 changes
The 2026-07-28 revision of the Model Context Protocol removes the initialize handshake and the session. A client may ask server/discover once, and every request then carries its protocol version and capabilities in its own _meta. A gateway that stores the discover result of a server can connect to it with no messages at all, and send the tool call alone.
Our traces show what that is worth. With a 2025-era upstream, the connect step of one tool call took 3.6 to 4.1 ms: initialize 1.4 to 1.6 ms, initialized 1.3 to 1.4 ms, and a GET probe 1.1 to 1.3 ms, each a round trip through the public ingress. With the same server on 2026-07-28, the connect step took 0.2 ms and sent nothing. The tool call itself took about 2 ms either way.
Gatana records the protocol each server speaks when it refreshes the server’s tools, and uses the stored discover result on every call after that. A server that still speaks the 2025 revisions keeps its handshake, so the 9.6 ms figure is what a 2025-era server costs through the gateway today.
What the added latency includes
The gateway checks the caller’s token and access rules, reads server and tool metadata, connects to the upstream MCP server, calls the tool, and records the result. The echo comparison includes all of that work.
The built-in control is skills_list_skills, with a query chosen to match no skill. It returns successfully without calling an upstream MCP server. Its run medians were 6.8 to 7.5 ms, close to the all-2026 echo path. With the handshake gone, the upstream call adds about 1 ms over the gateway’s own work. The rest is the server and tool rows, about 1.4 ms, and request handling.
Earlier results
We first measured this path on 5 October 2026, before and after a performance change on the same day. We measured it three times on 7 October: in the morning on the previous build, after the release that added protocol revision 2026-07-28, and after the release that added the caller cache. The direct path stayed close to 1.4 ms throughout.
| Measurement date | Direct echo median | Gateway echo median | Added latency estimate |
|---|---|---|---|
| 5 October, morning | 1.4 ms | 53.6 ms | about 52 ms |
| 5 October, afternoon | 1.4 ms | 30.3 ms | 28.9 ms |
| 7 October, morning, previous build | 1.4 ms | 14.8 ms | 13.4 ms |
| 7 October, protocol 2026 release, 2025-era upstream | 1.4 ms | 14.9 ms | 13.5 ms |
| 7 October, protocol 2026 release, 2026-07-28 upstream | 1.4 ms | 10.9 ms | 9.5 ms |
| 7 October, caller cache release, 2025-era upstream | 1.4 ms | 11.1 ms | 9.6 ms |
| 7 October, caller cache release, 2026-07-28 upstream | 1.5 ms | 8.1 ms | 6.6 ms |
The earlier figures and raw data remain available: 5 October, 7 October, morning and 7 October, protocol 2026 release. The 7 October runs on different builds are a historical comparison on one rig. The two upstream eras inside each run are a controlled comparison: same build, same day, same echo code, interleaved, with only the protocol revision different.
What it means in practice
An earlier analysis of one consenting customer organization’s audit log put the median real tool call at 1.0 second. We retain that baseline for context; we did not measure it again for this update. 6.6 ms is under 1% of a one-second call.
The gateway provides caller identity, access checks, credential storage, and an audit record. This benchmark measures their cost on the stated hardware and request path. It does not measure cold starts, concurrent load, or a complete user request. The all-2026 gateway echo 95th percentile was 21.0 to 46.0 ms across the three runs, so the headline is not a latency guarantee.
This run and the earlier figures are listed on the measurements page, with their dates, methods, and raw data.