The Coordination Axis Multi-Agent Systems Add
The failure taxonomy Anthropic assembled while running its own multi-agent research system skews toward coordination breakdowns rather than prompt quality. Typical cases: subagents re-running the same query, spinning up too many subagents for a simple confirmation request, or continuing to search well after enough evidence has already been gathered. Vague tool descriptions can send a subagent down an entirely different path, which is why tool-interface design deserves the same weight as prompt design as a failure cause.
An Observability Model That Cuts Failures Into Reproducible Units
Agent calls are non-deterministic — rerunning the same prompt can still take a different path — so a single log line rarely pins down the cause. OpenTelemetry's GenAI semantic conventions nest each LLM call as a chat span and each tool call as an execute_tool span under a top-level invoke_agent span, tying the entire reasoning chain into one trace. Attaching attributes like gen_ai.usage.input_tokens, output_tokens, and finish_reasons to those spans lets you narrow down which subagent's tool call spiked in tokens, or why a response cut off, from a single trace view instead of a full-text log search.
From Reproducing the Failure to Preventing Its Return: A Multi-Agent Debugging Roadmap
Nail down the pass bar in numbers before designing the system. A reasonable starting point: 100% trace coverage (every subagent call and tool execution carries a span), a subagent duplication rate under 10%, a mean time to recovery (MTTR) under 30 minutes, and a human-intervention rate under 15% after a final failure. Without a bar, adding tracing stalls at "looks good" vibes.
The most common failure is duplicated work across subagents. When the lead agent doesn't split subtasks cleanly, two or three subagents end up re-running the same query, burning tokens and latency. The second is over-spawning — launching many subagents in parallel for a simple confirmation request, where coordination overhead outgrows the task itself.
Third is unbounded search from a missing stop condition — continuing to search after enough evidence is in, or accepting low-credibility content as evidence. Fourth is path deviation from vague tool descriptions, where a subagent calls the wrong tool entirely; no amount of prompt polishing fixes this if the tool interface itself stays imprecise.
Recovery favors isolation over full retries. Re-run only the failed subagent from its checkpoint and preserve the other subagents' results instead of rewinding the whole task. Escalate to a human confirmation gate after more than two retries, and if the failure keeps recurring, fall back to a safe mode that runs with fewer subagents.
Before shipping, run a fault-injection scenario that deliberately times out one subagent or feeds it an empty response. Fix the log schema to include the invoke_agent span ID, the child execute_tool span ID, token usage, retry count, and the finish_reason, and mask any tool arguments carrying user-identifying data before they're persisted.
Pulling the top failure-trace patterns each week and reinjecting them into the regression test set means the same coordination failure gets caught automatically in the next release. Keep prompt/tool-description changes and orchestration-logic changes in separate change logs, so you can later trace which change actually lowered the duplication rate.
Key Execution Points
Most multi-agent failures start in coordination and tool interfaces, not the model. Set the bar at 100% trace coverage, a duplication rate under 10%, and an MTTR under 30 minutes; narrow the failure point with invoke_agent and execute_tool spans; then recover through isolated re-runs and a reduced safe mode — and the next incident follows the same playbook.