# Late chunking for RAG: embed context before pooling fragments

Late chunking encodes broader document context before pooling token ranges into chunk embeddings.

Author: Retrivium team
Updated: 2026-11-03
Canonical URL: https://retrivium.ai/journal/rag-late-chunking-contextual-embeddings/
Language: en

Late chunking encodes broader document context before pooling token ranges into chunk embeddings. It is different from independently embedding overlapping fragments, and it requires a compatible encoder and explicit span handling rather than merely moving a text splitter later in application code.

## Locate the contextualization step

The [author explanation of late chunking](https://jina.ai/news/late-chunking-in-long-context-embedding-models/) describes token representations formed with long-context information before chunk-level pooling. Ordinary early splitting removes surrounding context before the encoder sees each fragment. The difference concerns what information can influence token representations, not a guarantee that a longer string always gives better search. Preserve the actual text window supplied to the encoder when comparing the approaches.

## Keep pooling boundaries reproducible

The [reference implementation](https://github.com/jina-ai/late-chunking) provides a concrete method to inspect. Record chunk spans and their mapping to encoded tokens. Character offsets, tokenizer offsets and token positions are related but not interchangeable. Cleaning whitespace or adding a title can invalidate a previously computed mapping. The test should preserve both the original document and the serialized text used during encoding so the same span can be reconstructed afterward.

## Use a cross-reference example

Prepare an illustrative passage introducing the device name Atlas and a later paragraph referring to it indirectly. Ask a question whose answer needs that relationship. Compare independently encoded fragments with late-pooled fragments from the same document window. Keep query encoding and retrieval settings fixed. A successful example demonstrates that particular reference relationship; it does not establish a general improvement for every long document or language.

| Boundary | Information to preserve |
|---|---|
| Encoder window | Context actually visible to the model |
| Chunk span | Tokens included in its pooling |
| Retrieval result | Source passage associated with the vector |

## Distinguish model limits from document size

The [original paper](https://arxiv.org/abs/2409.04701) studies contextual chunk embeddings using long-context embedding models. A document longer than the supported window still needs an explicit windowing policy. Late chunking does not provide unlimited context or reconstruct text excluded from encoding. Record overlap between encoder windows separately from the final chunk spans, since those are different boundaries and can create different duplication behavior in the returned evidence.

## Compare the entire evidence path

Retain a retrieval fixture with pronouns, definitions and independent factual statements. The first cases may benefit from surrounding context; the last provide a check against unnecessary changes to already self-contained evidence. Inspect both source coverage and individual failures. A different final chunk count can change the amount of returned text, so hold the evidence budget visible rather than comparing only one hit limit.

## Keep late chunking separate from other techniques

Late chunking changes document encoding and pooling. It is not the same mechanism as token-level late-interaction reranking, and it does not describe generation-time cache reuse. Preserve that distinction in architecture diagrams and index metadata. A useful result states which encoder, document window and pooling spans were evaluated, including where context was unavailable, rather than labeling any context-enriched representation as a verified late-chunking pipeline.

Related reading: [Batch text embeddings on your own GPU](/journal/text-embeddings-batch/); [Evaluating embeddings for Russian language retrieval](/journal/embeddings-russian-retrieval/).