Source Document
Xinke Jiang, Yue Fang, Zhibang Yang, and 12 co-authors (corresponding authors Xu Chu, Junfeng Zhao, Yasha Wang), "AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing", arXiv:2608.29622 [cs.MA], submitted 2026-08-30, DOI 10.48550/arXiv.2608.29622. Affiliation: National Engineering Research Center of Software Engineering and School of Computer Science, Peking University (one co-author is affiliated with GRG Banking Equipment, Guangzhou).
A session-level egress failure blocked direct access to the arXiv source, so the full text was cross-checked against a primary-source snapshot GitHub Actions collected at 2026-09-01T22:02:48Z (UTC). This is a preprint without peer review, funded by the National Natural Science Foundation of China (Grant No. 62506010). One co-author is affiliated with a banking-equipment manufacturer (GRG Banking), but nothing in the paper suggests that company has a direct stake in RAG or search products.
Study Overview
The paper asks two questions: can splitting memory manipulation into its own action space with fine-grained, action-aware rewards ease the tendency of existing RL-based agentic RAG toward short, stereotyped reasoning caused by coarse action spaces and trajectory-level rewards, and does this design hold up consistently as backbone size, task difficulty, and step budget change? The authors propose AgenticRag-R1, which hierarchically separates actions such as <think>, <search>, <plan>, <summary>, <backtrack>, and <conclusion> and rewards each differently, combined with an information-aware trajectory rejection strategy that filters out low-information-gain rollouts. They train three Qwen2.5 backbones (1.5B, 3B, 7B) with a GRPO-family RL method and evaluate on seven multi-hop QA benchmarks: 2Wiki, HotpotQA, Bamboogle, FRAMES, MusiQue, NQ, and TriviaQA.
Key Results
Across all three backbones, AgenticRag-R1's average F1 beat the strongest baseline under the same condition (the best of ReACT, TC-RAG, ReSearch, Search-R1, AEPO, ARPO, and Mem1).
| Backbone | AgenticRag-R1 avg. F1 | Strongest baseline | Gap |
|---|---|---|---|
| Qwen2.5-1.5B | 27.65% | 24.38% (ReSearch) | +3.27 pts |
| Qwen2.5-3B | 33.55% | 30.16% (Search-R1) | +3.39 pts |
| Qwen2.5-7B | 36.60% | 31.02% (ReACT) | +5.58 pts |
In the component-ablation experiment on the 3B backbone (Table 2), removing the memory-related actions (Plan, Summary, Backtrack) dropped 2Wiki from 32.92% to 27.54% and HotpotQA from 44.00% to 28.77%. On the reward side, removing the memory reward cut the out-of-domain MusiQue benchmark from 16.48% to 7.63% — more than half — and removing both the RAG and memory rewards brought the average down to 19.94%. In the leave-one-action-out test (Table 4), removing <Backtrack> hurt more than removing <Plan> (average F1 -3.85 pts vs. -1.95 pts).
One result the paper does not foreground: in the long-horizon experiment that varies the maximum step budget from 10 to 30 (Table 3), AgenticRag-R1 actually trailed the baseline under a small step budget.
| Task | Model | 10 steps | 30 steps |
|---|---|---|---|
| Bamboogle | AgenticRag-R1 | 10.03% | 35.47% |
| Bamboogle | Search-R1 | 37.05% | 28.19% |
| 2Wiki | AgenticRag-R1 | 23.91% | 47.45% |
| 2Wiki | Search-R1 | 24.12% | 44.27% |
On Bamboogle at 10 steps, AgenticRag-R1 (10.03%) fell short of a third of Search-R1's score (37.05%), but by 30 steps AgenticRag-R1 (35.47%) had overtaken Search-R1, whose score had actually fallen (28.19%). The authors summarize this as AgenticRag-R1 improving monotonically as steps increase, while TC-RAG and Search-R1 are unstable or regress.
Credibility Assessment
What supports the numbers: the pattern of beating baselines held consistently across all three backbones (1.5B/3B/7B); the component, reward, and individual-action ablations point in the same direction (removing anything memory-related always hurts); and a separate SFT-only (22.40%) vs. RL-only (33.55%) vs. SFT+RL (35.20%) comparison confirms RL is the key contributor. The caveats, beyond being an unreviewed preprint: the Search-R1, ARPO, and other baseline numbers in the tables are not taken from the original papers but reproduced by these authors in their own setup (not externally verified figures), and, as shown above, there are genuine cases — a short step budget, or the 7B backbone on NQ (-6.41 pts) — where AgenticRag-R1 trails the baseline. The code is public, so replication is at least attemptable.
Reviewer's Judgement
First, I judge the paper's practical value lies less in the average-case claim that "separating memory actions with fine-grained rewards helps" than in the fact that this advantage depends heavily on the step budget. Performance dropping to a third of the baseline under a short step budget means that in production settings where latency or cost forces a tight step cap, this framework could actually be the worse choice. A decision should not be made from the average-performance table alone.
Second, the memory reward mattering far more on an out-of-domain benchmark (MusiQue: 16.48% → 7.63%) than on in-domain ones reads as a signal that memory-manipulation learning is tightly coupled to domain generalization, while the RAG reward instead contributed more to in-domain benchmarks (2Wiki, HotpotQA). Since the two rewards appear to cover different failure modes, simplifying down to just one of them looks like a real risk.
Putting It to Work
- Fix the step budget to production constraints first — evaluating only under a generous budget overstates the performance you'll actually see in a latency- or cost-capped deployment.
- Monitor the memory reward and the RAG reward separately — since one drives out-of-domain and the other in-domain performance, tuning only one can quietly hurt the other.
- Check the contribution of the Backtrack action first — it caused the largest drop of any single action when removed, so prioritize it in prompt and logging design.
- Consider SFT initialization alongside RL — SFT initialization lifted average F1 from 33.55% (RL only) to 35.20%.
Conclusion
The paper's conclusion is that separating memory manipulation into its own action space with fine-grained rewards beats strong baselines on average. But that edge is not uniform — under a short step budget, and in some combinations (the 7B backbone on NQ), the baseline did better. Given the unreviewed-preprint status and that most comparison numbers are the authors' own reproductions rather than externally verified figures, the safer reading is "the outcome depends on conditions" rather than "better on average." Cases where instruction adherence itself breaks down when a RAG agent goes into production are covered in Only One in Four Passes: A Compound-Instruction Gate for Noisy Production RAG.