Split the Actions, Split the Rewards
AgenticRag-R1 splits six actions — <think>, <search>, <plan>, <summary>, <backtrack>, <conclusion> — into a hierarchy and assigns a distinct reward to each, instead of the single trajectory-level reward that prior RL-based agentic RAG systems used. Trained on three Qwen2.5 backbones (1.5B, 3B, 7B) with GRPO-family RL and evaluated across seven multi-hop QA benchmarks, average F1 beat the strongest baseline by 3.27 to 5.58 points across all three sizes. Paired with a filter that discards low-information-gain trajectories, this is more than a finer-grained reward — it changes what the agent optimizes for.
One Average Table Won't Tell You Whether to Ship
The catch: that edge is not independent of step budget. Capped at 10 steps on Bamboogle, AgenticRag-R1 scored only 10.03%, far behind the Search-R1 baseline's 37.05%. Raise the same pair to 30 steps and AgenticRag-R1 climbs to 35.47% while Search-R1 drops to 28.19% — the ranking flips. Adopt on the strength of the average table alone, and a service that has to keep step counts low for latency reasons can end up worse off.
Field Guide: A Rollout Checklist for Step Budget and Reward Separation
Before reproducing benchmark numbers, decide the step ceiling your service can actually afford. Divide your p95 latency target by average time per step and you get the real usable step count in production. Below roughly 20 steps, the framework's advantage is likely to shrink, so set your target F1 delta at +3 points or more over the strongest baseline and re-evaluate adoption within that ceiling. The gain also scales with backbone size — 1.5B's +3.27pp grows to +5.58pp at 7B — so teams cutting cost with a small backbone should discount the expected upside.
Collapsing reward design into one signal is another common failure path. Removing the memory-related reward cut MusiQue, an out-of-domain benchmark, from 16.48% to 7.63% — more than half. The RAG reward, by contrast, contributed more to in-domain benchmarks like 2Wiki and HotpotQA. Since the two rewards cover different failure modes, tuning only one lets the other degrade silently in production. Among individual actions, removing Backtrack caused the largest drop (-3.85pp average F1, versus -1.95pp for removing Plan) — if that backtracking behavior gets dropped from logs or weakened in the prompt, pipeline quality slips first, so give it monitoring priority.
Don't trust the paper's benchmark numbers at face value before shipping. Many of the cited baseline scores are the authors' own reproductions in their environment, not externally verified figures, so reproduce at least once on your own dataset at your actual step ceiling. Measure at both your service's step limit and double that point to check for a ranking reversal, and log the action type per step, the per-action reward value, and information gain at trajectory end as required fields — without them you can't tell whether a regression comes from a tight step budget or a reward-design flaw.
Training method leaves room to improve too. RL alone landed at 33.55% average F1; initializing with SFT before RL pushed it to 35.20%, so it's worth securing cold-start SFT data. Keep a per-release table of performance by step budget, and the next model swap becomes a matter of filling in the same table with new numbers.
Takeaways at a Glance
Memory-based agentic RAG beats strong baselines on average, but that edge depends heavily on step budget. Fix the step ceiling your service can afford first and re-evaluate within it, monitor memory and RAG rewards separately, and watch Backtrack loss as a leading indicator — that's how you catch the deployment risk an average table hides.