Flattened vs. Hierarchical: What the Gateway Actually Changes
An MCP gateway sits between an agent and the MCP servers it connects to, and decides how tools get exposed. A flattened gateway lists every connected server's tools in one shot, every turn. A hierarchical gateway exposes only a meta-tool for discovery, so the agent looks up what it needs on demand. That choice is a design variable that moves task success, token cost, and security at the same time.
Once a deployment's tool pool runs into the hundreds, a flattened gateway's per-turn token count and latency grow linearly with that pool, because the full list rides in context every turn. A hierarchical gateway stays roughly flat, since it only pulls in the tools actually looked up. Before adopting either, plot that cost curve against your own tool pool size.
Capability Gaps Close; Security Gaps Don't
In a study that benchmarked seven commercial gateways, pushing the base model up closed the task-success gap between the two architectures to 87.2% vs. 85.1% — nearly identical. The indirect-attack success rate stayed at 47.2% vs. 30.9%. The assumption that a stronger model shrinks the architecture's security impact doesn't hold in this data; if anything, a stronger model leaves gateway architecture as a cleaner, more isolated security variable.
Almost all of that gap comes from "tool injection" — malicious content planted in a tool itself. For "environment injection," where the malicious content rides in a tool's returned data, the two architectures land within 3 points of each other. Switching to a hierarchical gateway does not close this path.
From Design to Operations: An MCP Gateway Selection and Operations Checklist
Set three target metrics. Keep the task-success-rate (TSR) gap under 2 points. Target at least a 15-point drop in tool-injection indirect-attack success rate (ASRind) from the architecture switch. And cap the token-growth multiplier of hierarchical vs. flattened at the 1,000-tool mark to set a cost ceiling. Looked at separately, a choice that seems secure can still fail on cost or latency.
A common failure pattern is treating an architecture switch as the whole fix. Switching architectures cuts attacks planted in tools, but barely touches attacks planted in what tools return. Move to a hierarchical gateway without adding a separate content-filtering layer, and that path stays wide open.
A second failure pattern is trusting a low attack-success number without checking task success alongside it. A low task-success rate means attacks fail for the same reason legitimate tasks do — capability collapse, not safety. Without TSR and ASR reported together on the same table, you can't tell the two apart.
The recovery strategy starts with separating defenses by attack path. Use gateway architecture as the defense line for tool injection, and add a separate content-filtering and sanitization layer for environment injection. When comparing vendors, judge by each product's actual numbers rather than its architecture category — products within the same "hierarchical" bucket can still vary widely.
Build an operations checklist: reproduce the token/latency curve against your own tool pool before deploying, re-measure TSR and ASR on the same table whenever the model or harness changes, and track the cost/success crossover point between flattened and hierarchical as exposed tool count grows. Treat single-run benchmark numbers as directional signals, and confirm the choice by reproducing it in your own tool environment.
The improvement loop starts with re-running a red-team task set quarterly, tallying tool-injection and environment-injection success rates separately. Redraw the noise-tool cost curve whenever a new MCP server joins the tool pool, and when attack patterns shift, keep content-filtering rule changes and gateway configuration changes in a separate change log that serves as next quarter's regression baseline.
Takeaways at a Glance
Target a TSR gap under 2 points, a 15-point-plus cut in ASRind, and a token ceiling at the 1,000-tool mark; use the architecture switch as the tool-injection defense only, add a separate filtering layer for environment injection, and always report TSR and ASR together so capability collapse doesn't get mistaken for safety.
References
GatewayBench: Unified Capability and Security Evaluation Platform for MCP Gateways — arXiv
Ask AI about this article
The assistant has read this article. Ask anything — it answers from the text and says so when something isn't in it.
Loading the chat…