When Reuse Is Worse Than No Cache: Reviewing KVShareArena

Source Document

Xi Shi, Qian Lou, "KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints", arXiv:2609.10266 [cs.CL], submitted 2026-09-09, DOI 10.48550/arXiv.2609.10266. Affiliation: University of Central Florida. Code to be released as a pip package (kvsharearena) with a public leaderboard and dataset.

This is a preprint without peer review, classed cs.CL, on benchmark design for KV-cache reuse in LLM serving infrastructure. Both authors are academic researchers at the University of Central Florida; no funding source or conflict-of-interest statement appears in the text. The compared methods — CacheBlend, KVPacket, RelayCaching and others — are public implementations from other groups' own authors, and the paper states that each ported implementation passed a score-equivalence gate or a component-level check, leaving a documented reproduction path. This session's egress proxy blocked all outbound fetches (confirmed against the control domain example.com, and unresolved after three retries at 10-minute intervals), so the full text was verified against a GitHub Actions snapshot of the primary-source HTML fetched at 2026-09-11T22:03:40Z rather than a live fetch.

Study Overview

Two questions drive the paper. When a RAG system assembles a different set of retrieved chunks per query, or a multi-agent coordinator reads reports written by other agents, how much accuracy does reusing an already-computed KV cache recover? And is reuse still safe when the checkpoint that wrote the cache differs from the one reading it? The metric is Performance-Gap-Recovered (PGR): the fraction of the gap a method recovers between a no-cache floor and a full-recomputation ceiling. Prior KV-cache benchmarks measured only exact-prefix reuse, the easy case a free position fix already handles. KVShareArena instead scores nine method families — compression, recomputation, and trained repairs — on two tracks (retrieved chunks and agent reports) using LongBench-family data.

Key Results

Paid recomputation and trained repairs beat free position-alignment only on questions that need several sources at once, such as multi-hop QA; compression never wins on any column.

On the Agent Reports track, naive assembly scores PGR −.824 — worse than the no-cache floor of 0. The free position fix alone recovers to .243, but going beyond that requires paying for recomputation: only CacheBlend and RelayCaching deliver significant gains, while the recompute-free trained adapter KVPacket ties the free baseline.

Robustness splits further when the producer checkpoint changes. Across 84 method-cells, the trained adapter KVPacket shows a mean PGR drop of −.074 and a worst cell of −.199, with 4 of 6 cells significantly below zero, while recompute- and alignment-based methods mostly stay within ±.02. This failure is silent: answers remain fluent and confident but name the wrong entity, with no visible signal of degradation. On infrastructure cost, per-request latency with a cache in hand falls roughly 90% against full recomputation for every reuse method, and given a cache-build time of .44–.73s on scientific-paper QA, a node-resident cache pays for itself after two to three reads.

Credibility Assessment

Three things support trust: the core comparison holds the model fixed and varies only the reuse method, the same pattern repeats on an independent confirmation board, and the benchmark, data and code are released as a pip package with a public leaderboard for outside verification. There are caveats too. The paper itself flags that follow-up papers sometimes report CacheBlend baselines without running the released implementation, a preemptive warning about inconsistency with other literature. It also discloses that 15 of 120 FRAMES-variant questions were title-only empty pages and shows recomputing without them changes no conclusion — but the defect itself remains on record. Cost figures are marked as draft, limited to same-backend rows, with a median over only 20 runs per row.

Reviewer's Judgement

The most practically valuable number here is not the PGR ceiling but the −.824 floor. It quantifies a real risk in the "we already have a cache, so reuse it" instinct for multi-agent pipelines: concatenating without repair can leave you worse off than not caching at all.

KVPacket's dual identity is also worth flagging — top-tier on a fixed checkpoint, but the single largest and quietest degrader once the checkpoint changes. Adopting a trained reuse method calls for tracking checkpoint-swap frequency in production, not just the accuracy table.

The compression family's pattern — fine on single-document tasks, sharply worse on agent-report reuse — is worth noting too: an assumption that compression is harmless can break depending on the payload.

Putting It to Work

• Make position alignment the default — never reuse multi-agent report or RAG-chunk KV caches unrepaired; put the free position fix in the baseline pipeline.
• Weigh recomputation for multi-source queries — if a meaningful share of queries need several sources at once, don't stop at position alignment; consider a CacheBlend-style partial recompute.
• Pair trained adapters with a checkpoint plan — check how often the serving checkpoint rotates before adopting one, and set a re-validation step for each swap.
• Re-validate compression per payload — a compression setting cleared on single-document summarization needs separate validation on payloads like agent-to-agent report handoff.
• Decide cache residency by reuse frequency — measure build cost against per-reuse savings, and keep only payloads past break-even resident on the node.

Conclusion

KVShareArena's contribution is not a new reuse algorithm but a shared yardstick for when reuse pays off and when it costs. The nearly free position fix handles most of the gap, but multi-source queries and multi-agent report handoff need paid recomputation for a significant gain, and the trained shortcut fails silently the moment a checkpoint changes. With the benchmark and code public, running your own payload through the same axes before picking a reuse strategy is the safer path. For the operational side of caching cost, see Prompt Caching Cost Checklist.

References

• KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints — arXiv abstract (source)
• Full HTML text of the same paper — used to verify tables and PGR figures (via snapshot)
• Prompt Caching Cost Checklist — sunny34.com blog