Where Cache Reuse Turns Worse Than No Cache

When one agent in a multi-agent system reuses the KV cache of a report another agent wrote, concatenating it as-is drops the percentage-gap-recovered (PGR) metric to −0.824. Treat the no-cache floor as 0, and this result sits below even that — reuse is actively harmful here. Adding free position correction lifts PGR to .243, but that still recovers less than half of what full recomputation would.

The same reuse technique behaves differently on the RAG document-chunk track. Single-document queries reach a respectable PGR of .620 with position correction alone, but multi-hop queries that must synthesize several sources stay negative at −.196 even after correction. That gap is why reuse strategy needs separate validation per payload type — single document, multi-source, or agent report.

Checkpoint Swaps Make Learned Reuse Fail Silently

KVPacket, a learned adapter that compresses and transforms the cache without recomputation, ranks near the top when the serving checkpoint stays fixed. The moment that checkpoint is swapped, though, it swings the hardest of any method across 84 method-cell combinations — average PGR drops −.074, and the worst cell falls to −.199. Recomputation- and position-correction-based methods mostly stayed within ±.02 under the same swap.

The more dangerous part is that the failure carries no visible signal. Answers still read fluent and confident, they just point at the wrong entity. A team judging deployment health from output logs alone is likely to miss this drop every time it rolls a checkpoint.

From Design to Rollout: A Multi-Agent Cache Reuse Safety Gate Checklist

At the planning stage, declare the PGR floor after position correction as your minimum acceptance bar. Rather than adopting the observed baseline (.243) as-is, re-measure the same axis on your own payloads, and once multi-source queries make up 30% or more of requests, weigh recomputation-based CacheBlend (about 15% recompute, PGR +.11) or RelayCaching (about 28% recompute, +.08) against staying with correction alone.

Decide cache residency with numbers too. Building a cache takes .44–.73 seconds, so only payloads reused at least 2–3 times should get a residency policy — otherwise build cost outruns the savings. Cache-hit requests cut latency by roughly 90% versus full recomputation, and that figure belongs in the same residency decision as an upper bound on the gain.

Four failure patterns keep recurring: naive concatenation with no position correction; a learned adapter failing silently after a checkpoint swap; a compression method that passes on single documents but collapses specifically on the agent-report track; and treating position correction alone as sufficient for multi-source queries when it is not.

Recovery branches should be automatic by default. When a checkpoint-swap event fires, cut over the learned-adapter path immediately, force a fallback to position correction, and only re-enable the adapter path once it clears a regression eval on the new checkpoint. Waiting on human approval leaves a window where silent wrong answers keep shipping.

Put position correction on the operations checklist as a mandatory step with no exceptions — skipping a free step is how you end up worse than no cache at all. Required log fields: cache-hit status, reuse method (none / position-corrected / recomputed / learned-adapter), checkpoint ID, payload track, and cache build time, so post-hoc comparison stays possible.

Pre-deployment scenario tests should include a checkpoint-swap simulation. Confirming ahead of time how many of the 84 combinations would fall below your threshold if the learned adapter carried over unchanged to a new checkpoint is what keeps a silent accuracy drop from surfacing only after the real swap ships.

The improvement loop starts with refreshing a per-payload table weekly. A compression setting that passed on single documents needs re-validation the moment the payload character changes, such as agent-to-agent report handoff, and every checkpoint change should get its own tracked regression run for the learned adapter, kept as a separate changelog.

Takeaways at a Glance

Reusing a KV cache across a multi-agent or RAG pipeline calls for position correction as a no-exceptions default, weighing recomputation-based methods once multi-source queries make up a real share of traffic, and a gate that automatically reverts learned adapters to position correction on every checkpoint swap. The two numbers behind that gate: naive concatenation lands at PGR −0.824, worse than no cache, and a learned adapter falls silently to −0.199 after a checkpoint swap.

References

KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints — arXiv

위치 보정만으론 부족하다: 멀티에이전트 KV캐시 재사용 KVShareArena 리뷰 — sunny34.com Research