Source Document
Jiahe Geng, Jinpeng Wang, Kun Yuan, "Compact-Memory LLM Agents via Online Max-Member Clustering and Atom-Aware Packing", arXiv:2609.04915 [cs.AI], submitted 2026-09-04, DOI 10.48550/arXiv.2609.04915, CC BY 4.0. No affiliation or funding source is stated anywhere in the paper.
This is a preprint without peer review. There is no statement of author affiliation, acknowledgments, funding, or conflicts of interest anywhere in the full text, references included -- not "undisclosed," but simply absent from the document. We therefore leave the conflict-of-interest question unresolved rather than assuming none exists. This session's network egress was blocked even against a control target (example.com), so the paper could not be fetched directly; the numbers below were checked against a GitHub Actions snapshot of the primary source's full HTML text (collected 2026-09-07T22:03:03Z, source=html, not truncated).
Study Overview
The question narrows to one thing: when prompt tokens are capped at roughly 2-5k, how should a streaming record of conversation and action be compressed to best preserve answer quality? The authors split this into two design choices. One is an online merge rule that decides whether an incoming chunk joins an existing cluster ("atom") or starts a new one -- a cosine threshold τ, where a chunk merges if either the cluster centroid or any existing member exceeds τ. The other is how retrieved clusters are placed in the prompt: grouped by atom with headers, versus flattened into a single ranked list.
Evaluation runs on the paper's primary benchmark, AMA-Bench (208 episodes, 2,496 QA pairs), and an independent long-horizon persona benchmark, RealMem (10 personas, 1,415 QA pairs), with LoCoMo-Plus, raw LoCoMo, and LongMemEval serving as conditional evidence and boundary conditions. Comparators include Full-Context, BM25-RAG, Streaming-Proto, Online K-Means, DP-means, Streaming-PCA (Oja), and reproductions of MemGPT, MemoryBank, and A-MEM run inside the same pipeline.
Key Results
At a 4k-token budget, RSM-full reaches 83% of Full-Context quality on 32% of the tokens (0.311/0.373 ≈ 83%; 4,001/12,519 ≈ 32%). Against compact-memory competitors at the same budget the margin is thinner: +0.9 pts over Streaming-Proto (single-seed paired bootstrap, p=.091, not significant) and +1.9 pts over Budget-RAG (p=.004).
| Method | AMA-Bench avg (~4k) | Total tokens | RealMem accuracy |
|---|---|---|---|
| Full-Context | 0.373 | 12,519 | -- |
| RSM-full (proposed) | 0.311 | 4,001 | 0.4684 |
| Streaming-Proto | 0.302 | 3,982 | 0.4388 |
| Budget-RAG | 0.292 | 4,146 | 0.4615 |
| A-MEM (2025 method, reproduced) | 0.273±0.003 | 4,218+414 | 0.4519 |
| No-Memory | 0.169 | 388 | 0.2317 |
Under four-seed averaging the margin widens: +3.5-6.0 pts over Online K-Means across the entire ~2.6k-~5k range (p<.001), and +4.6-9.5 pts over Streaming-PCA (Oja). A 2×3 factorial decomposition attributes this advantage not to the retrieval rule but to two other components -- the merge rule accounts for +5.7 pts over K-Means and +5.5 pts over DP-means (both p<.0001), and the grouped packer adds +5.02±1.00 pts over flat concatenation (three-seed mean, p<.0001). The retrieval rule itself (top-singular-vector ranking) contributes nothing once clustering and packing are held fixed (the A-D contrast, p=.40).
The same pattern reproduces on RealMem, an independent persona benchmark: +0.69 pts over Budget-RAG (p=.006), no significant difference from BM25-RAG (+0.27 pts, p=.47), and significant gains over Streaming-Proto (+2.97 pts) and A-MEM (+1.65 pts, both p<.001). But the appendix's domain-level breakdown against K-Means tells a mixed story: RSM wins on 4 of 6 domains (OPENWORLD-QA +10.27 pts, WEB +8.51 pts, TEXT2SQL +8.32 pts, SOFTWARE +4.18 pts, all p<.01), while K-Means matches or beats it on EMBODIED-AI (-2.68 pts, p=.042) and Game (-1.58 pts, not significant).
Credibility Assessment
Three things earn trust. First, the 2×3 factorial decomposition separates the merge rule from the retrieval rule and states which one actually contributes, with confidence intervals and p-values on every contrast. Second, AMA-Bench results are repeated across three to four seeds and RealMem across three stream-permutation seeds, and the direction agrees across both independent benchmarks. Third, a "bridge check" giving K-Means the same grouped packer (+4.90 pts, p<.0001) controls for the possibility that prompt assembly alone drives the advantage.
The caveats are just as clear. The embedding stack is a single BGE model and judging relies on a single designated judge model, so there is no cross-embedder or cross-judge replication -- a limitation the authors state themselves. The reproduced MemGPT and MemoryBank baselines fall to or below the No-Memory floor on RealMem and are classified as separate reproduction limitations, which does not fully rule out that these comparators were underestimated. The EMBODIED-AI and Game reversals appear only in an appendix table, not in the main text summary. Conflict-of-interest and funding disclosures are absent from the paper, as noted above, so this remains unresolved. As for contradicting evidence, the paper itself reports raw LoCoMo and LongMemEval as "boundary regimes" where the compact-memory advantage weakens or disappears.
Reviewer's Judgement
First, this paper's real contribution is not "compact memory works" but separating out which part of it works. The finding that the retrieval rule's contribution vanishes once clustering and packing are fixed (p=.40) is counterintuitive: teams often invest engineering effort in retrieval-algorithm sophistication, but if this data holds, how chunks are merged and how the result is assembled may be the far larger lever.
Second, it is a missed opportunity that the EMBODIED-AI and Game reversals sit only in an appendix and are not reflected in the abstract or conclusion. Given that the authors' own discussion ties compact-memory gains to streams with "enough repeated latent structure," knowing which task types reverse the effect is arguably the paper's most actionable piece of information for practitioners.
Third, while a single embedder and single judge is a real limitation the authors acknowledge, having two benchmarks of different character (AMA-Bench, RealMem) reproduce the same direction is more reassuring than a single-benchmark paper would be. That said, this "reproduction" still sits inside one embedding stack, and that distinction matters when reading the claim.
Putting It to Work
- Tune the merge rule first -- before investing in retrieval-algorithm sophistication, tune the write rule (cosine threshold) that decides when streaming chunks merge into one unit. Here the merge rule's contribution (+5.7 pts) exceeded the retrieval rule's (roughly 0 pts).
- Assemble prompts by cluster -- group retrieved items by their originating cluster instead of flattening them into a rank-ordered list. This single change accounted for a +5.02-pt gap.
- Expect gains only in the 2-5k token band -- the authors are explicit that this advantage is confined to the compact-memory regime. Above roughly 12k tokens, Full-Context and Full-RAG still deliver higher absolute quality.
- Validate per domain before adopting -- given the domain breakdown showing this approach can backfire on embodied-agent workloads (robotics, game-style simulation), run your own A/B against K-Means on your actual domain before rolling it out.
- Check whether your embedder and judge are fixed -- these results rest on a single embedding model and a single judge; if your stack differs, re-measure rather than importing the benchmark numbers directly.
Conclusion
RSM-full is not a new retrieval algorithm; its claim is that two design choices -- how streaming memories are merged, and how the merged result is assembled -- explain most of the compact-memory quality gap, and it backs that claim with two independent benchmarks. The statistical work is careful, but the boundaries are equally clear: a single embedder, a single judge, and domain-level reversals mean the numbers should be read with an implicit "under these conditions" attached. For a different approach to cutting long-horizon cost through context compression, see Context Compaction for Long-Running Agents.