Matryoshka embeddings: truncate and normalize both sides consistently

Matryoshka embeddings can be shortened only under the selected model’s supported representation contract, with matching treatment of queries and indexed documents. Arbitrarily slicing any embedding vector is not equivalent to using a model trained to retain useful information in shorter prefixes.

Check that truncation is actually supported

The original Matryoshka paper trains nested representations. Sentence Transformers documents the corresponding training and inference approach. The important property comes from training, not from the ability of an array library to select a prefix. Read the model card for supported dimensions and processing requirements before planning a smaller index. An arbitrary checkpoint may not preserve retrieval quality under the same operation.

Define one representation identity

Store model revision, prompt roles, output dimension, truncation and normalization together. The official EmbeddingGemma card is one model-specific reference for supported representations. Do not mix its rules with a different model’s dimensions. A query and a document can have equal vector lengths yet occupy incompatible spaces because they came from different checkpoints or were normalized at different stages.

Use a controlled dimension experiment

Take the same judged question set and document corpus. Construct separate indexes for supported full and shortened representations, keeping corpus identifiers unchanged. Encode queries under the matching configuration for each index. Compare relevant-document coverage and ranking disagreements before changing any candidate counts. This isolates the representation change from a simultaneous change in retrieval policy or document parsing.

Change	Required matching action
Shorter supported prefix	Rebuild corresponding document vectors
Normalization after truncation	Apply consistently to query and document
New checkpoint revision	Evaluate a new representation identity
New query instruction	Recheck asymmetric retrieval behavior

Account for normalization and distance

A prefix of a normalized full vector is not generally unit length. When the model’s guidance requires normalized truncated embeddings, normalization must follow the truncation step. The selected distance operation must match the resulting contract. Equal ranking under cosine and inner product depends on relevant assumptions, not on their names. Validate these assumptions rather than switching index metrics to make an integration error disappear.

Migrate without mixing representations

Keep old and new indexes distinguishable until the replacement passes the held-out fixture. A payload flag alone cannot repair incompatible vectors already placed in the same index. Record which representation produced each result during comparison, and preserve rollback artifacts. Report the measured local quality tradeoff separately from reduced vector storage. Smaller vectors offer a concrete storage change, but they do not establish lower end-to-end latency or unchanged answer quality without examining the rest of the retrieval pipeline.