# CacheBlend for RAG: reuse document chunks beyond an exact prefix

Understand selective recomputation in CacheBlend, why reordered retrieved chunks challenge ordinary prefix reuse, and how to evaluate answer quality.

Author: Retrivium team
Updated: 2026-11-03
Canonical URL: https://retrivium.ai/journal/cacheblend-rag-non-prefix-reuse/
Language: en

A retrieval assistant often revisits the same documents in different combinations. Ordinary prefix caching helps only as far as the serialized request has the same beginning. If a new search places an earlier document behind different text, that document may no longer belong to the reusable prefix. CacheBlend addresses this serving-side problem after retrieval has already selected the evidence.

## Locate the missing reuse

Write down the exact model input for successive questions. Mark the system instructions, retrieved passages, separators and question. A document appearing in both requests does not mean its preceding context is unchanged.

The [vLLM prefix caching guide](https://docs.vllm.ai/en/latest/features/automatic_prefix_caching/) describes shared-prefix reuse. CacheBlend should therefore be evaluated against that existing baseline, rather than against an unnecessarily cache-free deployment. Retrieval ranking quality remains a separate concern: a faster prefill cannot make the wrong evidence relevant.

## Understand selective recomputation

The [LMCache CacheBlend guide](https://docs.lmcache.ai/kv_cache_optimizations/cacheblend.html) describes combining previously computed chunks with selective recomputation. It is not a promise that arbitrary KV blocks can simply be concatenated without considering their surrounding context.

This distinction determines the quality test. A passage's attention state was computed with a particular history. Reusing it under another history requires the method's supported reconstruction or recomputation behavior. Evaluate complete answers, including dependencies across passages, rather than treating a successful cache lookup as proof of equivalence.

## Keep integration generations separate

Current multiprocess integrations and older in-process examples can expose different switches and execution paths. The [LMCache configuration reference](https://docs.lmcache.ai/api_reference/configurations.html) includes legacy blending settings; those names should not be copied indiscriminately into a different integration generation.

Record the engine, LMCache revision, connector mode and the documented model/backend limitations. Check how chunk boundaries are represented in the actual token stream. A separator visible in source text may be transformed by a chat template before the model receives it, changing the intended boundary.

## Build a document-order test

Choose questions requiring one passage, several independent passages and an explicit relationship between documents. Replay them with evidence in the original order, a changed order and a partially replaced set. Hold the retrieval results fixed so the experiment isolates context processing.

Compare answers with the ordinary engine baseline. Assess citation correctness, missing constraints and cross-document reasoning. Then record first-token latency, loaded bytes and recomputed work for each case. Aggregate latency alone can hide a bad result on the questions where chunk interaction matters most.

## State the useful operating region

CacheBlend is most interesting when documents repeat but exact prefixes do not, and repeated prefill represents material work. Unique uploads or short contexts may offer little reusable state. Large transfers can also erase the benefit of avoiding arithmetic.

Keep a documented fallback when the selected backend or request type is unsupported. Report measured quality and latency together for the tested workload, without turning a result from one document corpus into a universal acceleration claim. Cache reuse is a serving optimization whose value depends on both repetition and preserved task performance.

## Related reading

[Prefix caching: when repeated context helps a GPU pool](/journal/prefix-caching/), [A high-RAM RAG server: deciding what to keep in memory](/journal/high-ram-rag-servers/).