pgvector binary quantization: Hamming candidates need full-vector reranking

Binary quantization in pgvector is a candidate-generation technique, not a replacement for checking final relevance with the intended vector metric. Preserve original vectors when the design requires a Hamming shortlist followed by full-vector reranking.

Keep two different distance spaces

The official pgvector examples build an expression index over binary_quantize and then rerank candidates with original embeddings. Hamming distance compares binary representations; it is not the original cosine or Euclidean distance. The shortlist can be useful without preserving every original neighbor. Describing both stages as one exact nearest-neighbor operation would hide the approximation that determines which documents remain available.

Preserve dimensionality and operators

The SQL definitions distinguish bit-vector operations from vector and halfvec operators. Check the declared bit length and representation on both sides of the query. A matching array length alone cannot establish that query and document embeddings come from compatible models. Keep checkpoint identity and preprocessing with the SQL fixture. Do not turn a type-correct expression into evidence of semantic correctness.

Use an explicit two-stage experiment

Prepare a manageable corpus with exact original-vector judgments. First request a bounded set using the binary index; then reorder only those IDs with the declared full-vector distance. Compare shortlist coverage with the exact reference before inspecting final ranking. The fixture is illustrative and supplies no universal candidate count. Increase the shortlist deliberately and record what newly admitted evidence changes rather than assuming every wider list is worth its additional work.

Observation	What it can establish
Shortlist recall	Whether required evidence survived approximation
Final relevance	Whether rescoring orders surviving evidence usefully
End-to-end time	Cost of both stages together

Diagnose where evidence disappeared

A relevant document absent before reranking is a candidate-generation miss. A surviving document with an unexpected final position calls for checking the original metric and query representation. Filters can introduce another boundary, so preserve their exact position and condition. Do not tune final score thresholds to compensate for documents never admitted. That would change output behavior without repairing the retrieval loss that caused the missing evidence.

Separate version and footprint claims

The project changelog records feature additions and later changes affecting binary quantization. Verify the installed extension rather than assuming any published example applies to every release. The binary index may be smaller while original table data remains. Reranking reads that data, so storage and access locality still matter. A size improvement from another benchmark does not determine memory use or elapsed time for your own corpus.

Retain the original baseline

Save exact query results, candidate IDs, final results and configuration together. Compare changes with the same query set and relevance labels. If original vectors are removed to reduce storage further, the full-vector verification stage is no longer the same design. Treat that as a new quality decision. An accepted configuration should identify the approximation and measured evidence coverage explicitly instead of presenting binary search as a cost-free substitute for the original representation.