Why Agent Memory Needs Two Tiers
Keeping an agent's conversation grounded takes a store that reads and writes recent turns fast, plus a separate store that searches millions of knowledge entries by meaning. Cramming both into one system pits latency against consistency. Cloudflare Workers KV is a key-value cache with single-digit-millisecond reads; Vectorize is a vector database that indexes up to 10 million vectors per index. Agreeing on the two stores' different consistency models up front makes incident response far less painful later.
The 60-Second Price of KV
Cloudflare's own documentation states that a KV write is visible immediately at the edge location where it happened, but can take up to 60 seconds — or the cacheTtl value passed to get() — to reach other regions. The sharper edge case: a negative lookup, "key not found," gets cached too, so querying a just-created key from another region can return a false miss.
What Vectorize Indexes Can and Cannot Hold
Vectorize supports up to 10 million vectors per index and up to 1,536 dimensions per vector at 32-bit precision, with 10 metadata indexes per index and up to 10KiB of metadata per vector. String metadata is only filterable on its first 64 bytes, so short values like category or tag belong at the front. Index data is immutable per snapshot version, which is why lookups routed through R2 and Cloudflare's cache are unusually fast.
Field Guide: Running the Cache and the Vector Index Together
Fix target numbers before writing any code: p95 vector search latency under 150ms, context cache hit rate above 80%, and a re-ask rate under 5% caused by stale cache reads. Keep only the last 2-3 turns of session context in KV and push the full conversation history to D1 or R2, which limits both the KV value size and the blast radius of that 60-second propagation delay.
The most common failure is a request in another region treating a just-updated knowledge document as missing because of a cached negative lookup. The fix is a bypass branch: right after a write, the same request path queries the source of truth — D1 or an external API — instead of KV, and only switches back to the cache path once 60 seconds have passed.
The second failure pattern hits after an embedding model swap, when the new embedding dimension no longer matches the existing Vectorize index and every insert fails outright. Embedding the model version in the index name lets you roll back to the previous index instantly if a rebuild goes wrong, and keeping metadata filters to only the fields actually used in queries — within the 10-index cap — avoids repeated index rebuilds.
The third failure pattern is feeding low-similarity documents into the context anyway, which invites hallucination. Discard results below a similarity threshold, and when nothing survives that cut, branch to a "insufficient evidence" response instead of guessing.
Before shipping, reproduce and pass three scenarios: querying another region right after a KV write, inserting with a mismatched embedding dimension, and exceeding the 10-metadata-index cap. Logs should carry cache-hit status, lookup latency, the similarity-score distribution of returned vectors, and the index version in use, so regional latency skew shows up on a dashboard immediately.
Keep raw user utterances and document text out of metadata — store only the values used for filtering, and fetch the underlying text from separate storage after an access check, to narrow PII exposure. Embedding swaps, metadata schema changes, and cache-policy changes each carry different regression risk, so log them separately; tallying weekly how many responses were wrong due to negative-lookup caching and how many search results got discarded below threshold gives you the evidence to retune cacheTtl or the similarity cutoff.
Takeaways at a Glance
Put session context in KV and knowledge search in Vectorize, but only after sizing up both stores' delays and limits in numbers. Block negative-lookup staleness and embedding-dimension mismatches with bypass branches and version tagging, and discard sub-threshold matches safely — that combination holds the p95 150ms and 80% cache-hit targets while cutting down how often the agent hands back a stale answer.