# LanceDB phrase search: preserve positions before hybrid retrieval

Phrase retrieval in LanceDB requires an index and query contract that preserves word positions.

Author: Retrivium team
Updated: 2026-11-03
Canonical URL: https://retrivium.ai/journal/lancedb-phrase-search-hybrid-candidates/
Language: en

Phrase retrieval in LanceDB requires an index and query contract that preserves word positions. A hybrid result can miss an exact phrase for lexical reasons even when its dense embeddings are correct, so inspect the FTS branch before tuning fusion.

## Distinguish phrases from independent terms

The [full-text reference](https://docs.lancedb.com/search/full-text-search) separates phrase queries from term queries in Lance-based FTS. A phrase asks about an ordered expression; a term query can find words without the same adjacency. The checked documentation also warns that boolean operators in a search string are not a supported substitute for native query configuration. Do not borrow a grammar from another search backend merely because both products expose an FTS label.

## Preserve positions at indexing time

The documented phrase route needs positional indexing and suitable stop-word handling, including `with_position=True` and `remove_stop_words=False`. Confirm those settings before judging a query. Quoting a string cannot recover positions never stored in the index. If index configuration changes, retain its generation with the fixture so a successful phrase result is not mistakenly attributed only to a reranker or a new embedding checkpoint.

## Test a meaningful word-order contrast

Prepare an illustrative corpus containing an exact technical phrase, a sentence with the same words in another order, and a semantic paraphrase without those words. Judge the task before choosing query mode. An exact-expression lookup may need the first document, while a conceptual question may legitimately need the paraphrase. One combined average should not silently redefine which of those tasks the search is supposed to solve.

| Evidence type | What a lexical test should reveal |
|---|---|
| Exact ordered phrase | Position-aware matching |
| Same terms elsewhere | Difference from phrase semantics |
| Paraphrase | Boundary of the lexical branch |

## Keep the hybrid inputs visible

The [hybrid-search guide](https://docs.lancedb.com/search/hybrid-search) describes semantic and keyword paths before their results are combined. Export the lexical candidates separately and check the query text delivered to that branch. A phrase absent there is not automatically a failure of the vector model. Conversely, a dense hit can conceal a lexical configuration error by returning a useful document through another route. Preserve both explanations when accepting the pipeline.

## Use reranking for the right problem

The [reranking documentation](https://docs.lancedb.com/reranking) describes another operation over retrieved material. It can change ordering among supplied candidates but cannot make an unsupported lexical query syntax become valid. When evaluating a new reranker, keep FTS configuration and candidates unchanged. Otherwise a ranking comparison can accidentally measure a rebuilt positional index or changed token handling instead of the intended scorer.

## Retain separate phrase and semantic fixtures

Store query mode, original text, tokenizer/index options and expected documents with each case. After a parser or schema migration, repeat phrase-specific cases as well as semantic questions. Document which query grammar and FTS implementation were tested on the installed version. That record makes an exact-phrase failure actionable and prevents a generic hybrid-search label from hiding incompatible assumptions about terms, positions and source evidence.

Related reading: [A local RAG server: placing components around the GPU](/journal/rag-server-local-architecture/); [Evaluating embeddings for Russian language retrieval](/journal/embeddings-russian-retrieval/).