In this article
A ColPali API migration must preserve query processing, image processing, masking and multivector scoring before adding new features.
Migrating ColPali to Sentence Transformers requires a score-parity fixture, because replacing imports alone can silently change query processing and similarity. The migration should preserve the retrieval decision on known pages before introducing a new checkpoint, batching policy or numerical representation.
Start from the supported migration path
The author-maintained ColPali repository marks colpali-engine as deprecated and points toward Sentence Transformers. The migration guide explains the current transition. Record the old package version and model revision before changing them. A new API can load successfully while applying different preprocessing, masking or normalization, so successful initialization is only the beginning of acceptance.
Map operations rather than names
The old workflow separated query processing, image processing and multivector scoring. The new MultiVectorEncoder usage exposes query encoding, document encoding and similarity operations. Map the actual role of each old call to the new interface. Do not assume that a method with a familiar name receives the same already-processed object or returns the same shape.
| Old responsibility | Migration check |
|---|---|
| Process a query | Preserve prompt and query role |
| Process page images | Preserve page rendering and document role |
| Score multivectors | Preserve masks and aggregation |
| Select top pages | Preserve identifiers and tie handling |
Keep a small score fixture
Choose a manual page with a diagram, another page with similar vocabulary and an unrelated page. Use a question whose answer appears only in the diagram. Save the old score matrix and ranking with page identifiers. Run the replacement against exactly those images and questions. This fixture checks both relevant-page selection and whether apparently close scores changed for a systematic reason.
Inspect masks and projection placement
Padding tokens and non-image embeddings must be handled according to the selected model and migration guidance. A change in masking can add contributions that were previously excluded. Likewise, checkpoint packaging can place a projection or normalization inside the model or in a surrounding wrapper. Applying either step twice changes the representation even if its final dimension looks correct.
Separate numerical drift from semantic changes
Small floating-point differences need an explicit tolerance suitable for the implementation, rather than an invented universal threshold. Investigate ranking changes near ties separately from large systematic shifts. If every score moves after migration, inspect aggregation and preprocessing before blaming the GPU. If only selected pages move, compare page dimensions, image conversion and masks associated with those examples.
Accept the migration before extending it
The deliverable should contain the operation mapping, pinned revisions, fixture images, old and new scores, and a written explanation of any accepted difference. First reproduce the former behavior; then evaluate additional features as separate changes. This prevents a migration, a model upgrade and a document-rendering change from becoming a single unexplained relevance regression that cannot be rolled back selectively.
Related reading: Embeddings and rerankers: other tasks for an idle GPU; Reproducible inference stacks: preserving a GPU pilot’s result.