The Sub-Agent That Interrupts Wins: POrchestra Lifts GAIA2 from 44.5% to 55.5% at 36% Lower Cost

Source Document

Xueyang Feng, Chaozhuo Li, Xiaohe Bo, Jiakai Tang, Chenxi Li, Qiwei Ye, Xu Chen, "Proactive Agentic Orchestration", arXiv:2610.13485 [cs.CL], submitted 2026-10-09, DOI 10.48550/arXiv.2610.13485. Affiliation: Gaoling School of Artificial Intelligence, Renmin University of China; The Chinese University of Hong Kong, Shenzhen; Beijing Academy of Artificial Intelligence (BAAI). Code released (github.com/XueyangFeng/POrchestra).

This is a preprint without peer review. The ethics statement notes compliance with the "ICLR Code of Ethics," suggesting an ICLR 2026 submission, though acceptance is unconfirmed. The authors' affiliations are universities and a non-profit research institute (BAAI), so no clear conflict of interest from promoting a proprietary model appears. That said, all three backbone models used in evaluation (DeepSeek-V3.2, Gemini-3-Flash, DeepSeek-V4-Flash) are commercial APIs, and the exact unit-cost methodology sits only in the appendix, which we did not recompute for this review. Due to this session's network restrictions, the source was checked against a pre-collected snapshot of the arXiv HTML full text (collected 2026-10-12T22:11:36Z, a primary-source copy retained by GitHub Actions) — since it is the same primary document, our verification standard is unchanged.

Study Overview

Long-horizon agentic orchestration typically has a manager delegate fixed goals, tools, and context to sub-agents up front, then wait for a completion or error signal before re-coordinating. The authors treat this delay as the key bottleneck — in dynamic real-world settings, a sub-agent only discovers a blocked tool or a changed goal after starting the task, and existing designs give it no way to speak up at that moment.

Their proposed POrchestra lets a sub-agent proactively contact the orchestrator at any logical point during execution to request harness adjustments (goal, tools, context, budget), while preserving its local execution history throughout. Building on this, On-Policy Orchestration Distillation (OPOD) traces a sub-agent's revision requests back to the orchestration decisions that preceded them, turning them into targeted improvement signals, and uses these alongside the final reward to train the orchestration policy. Evaluation spans three benchmarks — GAIA (static knowledge/tool tasks), GAIA2 (an ICLR 2026 benchmark testing time constraints, dynamic events, and ambiguity resolution), and SWE-Bench Verified (software engineering) — across three backbone models: DeepSeek-V3.2, Gemini-3-Flash, and DeepSeek-V4-Flash.

Key Results

The paper's Table 1 compares eight methods per backbone (single-agent ReAct and Mini-SWE; offline orchestration Skill-MAS and AutoAgents; re-orchestration MAS2, Orchestra-o1, and AOrchestra; and the proposed POrchestra). GAIA's "Average" is not a simple mean but a task-count-weighted average across Levels 1-3 (which have unequal task counts); GAIA2's "Average" is a simple mean of its four sub-metrics (Execution, Search, Adaptability, Time) — we recomputed this arithmetic and it matches the paper's reported figures exactly.

POrchestra outperformed every baseline (ReAct included) across all three backbones on GAIA, GAIA2, and SWE-Bench. The sharpest reversal appeared on GAIA2: with Gemini-3-Flash, AOrchestra beat ReAct on GAIA (76.4% vs. 69.1%) but lost to it on GAIA2 (39.8% vs. 46.9%) — orchestration that wins on static tasks can underperform a single agent on dynamic ones. On GAIA2 with DeepSeek-V4-Flash, POrchestra reached 55.5%, 11.0 points above the strongest baseline, ReAct, at 44.5% (a 24.6% relative gain).

A cost analysis (RQ3) on the same combination (DeepSeek-V4-Flash, GAIA2) found that POrchestra beat ReAct by 11.0 points at 36% lower per-task API cost, and beat Orchestra-o1 by 11.7 points at 31% lower cost.

The paper also trains a small Qwen3.5-9B orchestrator with OPOD. SFT alone reached 72.4% on GAIA and 43.8% on GAIA2; adding OPOD lifted these to 77.9% (+5.5pp) and 49.2% (+5.4pp). Ablating each signal separately: removing outcome reward (RWR) drops GAIA to 75.8% (-2.1pp) and GAIA2 to 46.1% (-3.1pp); removing process feedback (OPD) drops GAIA to 76.1% (-1.8pp) and GAIA2 to 46.1% (-3.1pp). The overall drops look similar, but removing OPD specifically hits GAIA2's dynamic sub-metrics much harder — Adaptability (-7.8pp) and Time (-7.9pp) — consistent with the authors' claim that the two signals play distinct roles.

Credibility Assessment

Four points support trust here. First, the same backbone is applied across eight methods in a controlled comparison, so model effects and orchestration effects don't get conflated. Second, the ranking is consistent across three benchmarks of different character (static GAIA, dynamic GAIA2, and software-engineering SWE-Bench) across all three backbones. Third, code is public (github.com/XueyangFeng/POrchestra), leaving a path for external replication. Fourth, the authors publish ablations removing each OPOD signal (RWR, OPD) separately, partially isolating where the gains come from.

Caveats are just as clear. This is an unreviewed preprint, likely under ICLR review with no confirmed outcome. All three evaluation backbones are commercial APIs, so cost figures could shift with provider pricing changes; the precise costing method sits only in the appendix and we did not recompute it. We found no mention of run counts or variance (standard errors, confidence intervals) in the body — it's hard to tell from this paper alone how much of the reported gap exceeds run-to-run noise. No conflict of interest stands out, but we could not find a funding-source statement in the body or appendix. There is also a contradicting signal: as the Related Work section below notes, AOrchestra and MAS² each report larger relative gains (16-20%) on their own benchmark suites, yet in this paper's GAIA2 (a dynamic setting) those same methods underperform a single agent — which benchmark you use can change which orchestration looks "best."

Related Work

• Ruan et al. (2026). AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration (arXiv:2602.03786) — Prior work and this paper's main comparison baseline. Its own evaluation (Terminal-Bench 2.0, SWE-Bench, GAIA) reports a 16.28% relative gain over the strongest baseline, yet in this paper's GAIA2 (dynamic setting) it scores below single-agent ReAct.
• (2026). MAS²: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (arXiv:2509.24323, ICLR 2026) — Prior work. A three-stage meta-agent design (Generator, Implementer, Rectifier) that generates and corrects the multi-agent system itself; also used as a comparison baseline (MAS2) in this paper.
• Froger et al. (2026). GAIA2 (arXiv:2602.11964, ICLR 2026) — Benchmark origin. Designed to test time constraints, dynamic events, ambiguity resolution, and collaboration; this paper uses it as the stage for its claim that fixed-harness orchestration struggles in dynamic settings.

Because of this session's egress restriction, these three works were independently identified and confirmed via WebSearch rather than by reopening their full text. Accordingly, the figures AOrchestra and MAS² report for themselves (16.28%, up to 19.6%, etc.) are used only in the relational descriptions above, not in this review's "Key Results." All three match, by title, authors, and core claims, the exact works this paper itself cites and compares against (AOrchestra, MAS2, GAIA2), so the risk of a bibliographic mismatch is low.

Reviewer's Take

First, we judge the most valuable result here is not the 55.5% figure but the AOrchestra reversal. Watching the same baseline beat ReAct on static GAIA and lose to it on dynamic GAIA2, within one paper, shows how "orchestration always helps" can break down depending on benchmark character. When deciding whether to adopt a multi-agent orchestration layer, relying only on a vendor's self-reported relative gain risks missing exactly this kind of reversal.

Second, we find OPOD's design — tracing post-hoc revision requests back into training signal — more broadly useful than the specific cost figures. In human-operated agent pipelines, "please redo this" request logs are commonly discarded; tracing them back to identify which decision triggered the rework is a technique that could be partially applied just by changing logging structure, without redesigning the orchestration framework itself.

Third, the absence of run-variance reporting makes it hard to gauge the confidence interval behind the 24.6% relative gain. On a benchmark like GAIA2, where some sub-metrics cover only a few dozen tasks, this paper alone cannot settle whether a few-percentage-point gap from a single run exceeds normal run-to-run variation. Re-confirming with repeated runs in one's own environment before adoption seems warranted.

Practical Takeaways

• Open a "talk-back" channel for sub-agents — don't force a pre-fixed goal, toolset, and budget through to the end; design a communication path for the sub-agent to request adjustments mid-execution when blocked. This cuts latency versus waiting for a completion or error signal.
• Test static and dynamic conditions separately — before adopting an orchestration layer, measure performance separately on tasks where conditions stay fixed versus tasks where they change mid-run. As with AOrchestra here, a method can win on one and lose on the other in your own environment too.
• Log revision requests as training signal — when a "please redo this" request happens, record the orchestration decision that preceded it; this data supports OPOD-style post-hoc training, or at minimum, manual review.
• Compare cost alongside success rate, under the same conditions — the assumption that "adding a coordination step always raises cost" varied by baseline even in this paper (RQ3). Decide on adoption by placing the success-rate gain and the per-task cost change side by side.
• Test a small orchestrator with distillation before scaling up — before running a large orchestrator in production, check in your own tasks how closely a small model can match it using OPOD-style process-feedback distillation.

Conclusion

POrchestra's contribution is that a relatively simple design change — letting a sub-agent speak up first — paid off on both static and dynamic tasks. On dynamic GAIA2 specifically, existing orchestration methods sometimes underperformed a single agent; this paper traces that to fixed harnesses and prescribes mid-execution adjustment, improving both success rate and cost at once. Still, this is an unreviewed preprint with no run-variance reporting, so the numbers are best read as directional signals to reproduce in one's own benchmark before adoption. For sub-agent operations framed around budget and resumption, see Sub-Agent Workflow Budget and Resumption Design on the sunny34.com blog.

References

• Proactive Agentic Orchestration — arXiv abstract (source)
• Same paper, HTML full text — used for table and cost figures (via snapshot)
• AOrchestra (arXiv:2602.03786) — related work
• MAS² (arXiv:2509.24323) — related work
• GAIA2 (arXiv:2602.11964) — benchmark origin
• Sub-Agent Workflow Budget and Resumption Design — sunny34.com blog