In this article
Batch throughput matters only when vectors remain consistent, traceable to their source fragments and recoverable after interruption.
Batching text embeddings on your GPU is useful when preparing a document corpus for retrieval. The main goal is a consistent vector set with a reliable link to source fragments. Peak throughput for one batch matters only if results can be checked, updated and recovered after interruption.
Fix the representation recipe
Record the exact embedding model and version, chunking rules, maximum input length and normalization where used. Save source-fragment identifiers and document versions with the output. This makes it possible to distinguish a retried batch from a new representation of changed content.
Use the model’s recommended preparation rather than assuming every encoder accepts the same prompts or prefixes. Two batches with equal dimensions are not necessarily compatible if they used different models or processing rules. Keep the recipe in the index metadata and experiment record.
Distinguish query and document encoding
Semantic-search tasks can be symmetric or asymmetric. A short question and a longer source passage may require different documented encoding methods. Check the selected model’s guidance for each role instead of using a convenient generic call without evaluation.
Prepare a small set of questions with known relevant fragments before indexing the whole corpus. This gives you a quality control for later batch or numeric-setting changes. A successful vector calculation does not itself establish useful retrieval.
Tune batches on real lengths
| Variable | What to observe |
|---|---|
| Texts per batch | Peak memory and completion time |
| Fragment length | Short and long input behavior |
| Concurrent processes | Competition for GPU and RAM |
| Database writes | Whether persistence limits the full pipeline |
Compare batches containing representative long fragments as well as short ones. A batch size selected only from brief examples can fail later. Keep model and preparation constant while changing the batch parameter, and save actual completed records rather than assuming every input survived an interrupted operation.
Make indexing resumable and useful
Give each work item a stable identity and record completed outputs. Test a controlled interruption on a small dataset, then resume without missing or duplicating logical records. Define how changed documents invalidate old vectors and how a rebuilt collection becomes authoritative.
Measure the complete path from text loading through GPU execution to durable storage. Then evaluate retrieval on the control questions. A fast encoder can be idle while database writes bottleneck the task, so report both stages. That identifies a concrete indexing role without promising demand or utilization.
Official references: sbert.net — quickstart, sbert.net — README.
Related guides: Evaluating embeddings for Russian language retrieval, Embedding dimensions: calculating index memory.