In this article
Text and page-image retrieval can miss different evidence, so candidate pages and representation-specific failures must remain separately inspectable.
Nemotron 3.5 Embed should be evaluated with matched text and page-image queries, because changing the document representation changes the retrieval problem. A successful import or a larger checkpoint does not establish that a diagram, scanned table and digital paragraph will produce equally usable evidence.
Keep the variant and input contract together
The official 1B BF16 card defines a particular checkpoint, rather than a generic interchangeable embedding service. Preserve its revision, processor, query instructions and document preprocessing as one configuration. The 8B BF16 card supplies a separate comparison candidate. Do not substitute its processing steps into the smaller model merely because both belong to the same family.
Build a paired document fixture
Use a short equipment manual containing a text specification, a connection diagram and a table photographed from a printed page. Store the original page, extracted text, page identifier and human relevance judgment. Ask the same maintenance question against each representation. This separates visual evidence that disappeared during extraction from a failure to rank evidence that remains available.
| Question | Text evidence | Page-image evidence |
|---|---|---|
| Which connector is required? | Caption and reference | Diagram and labels |
| Which operating limit applies? | Extracted table row | Row with column headings |
| Which revision is current? | Revision paragraph | Header and stamp |
Preserve query and document roles
Treat a question and a document as different input roles wherever the card requires that distinction. A search wrapper might already add instructions; adding them again changes the input. Save the final processed query alongside the document processor configuration. A comparison becomes uninterpretable when one candidate receives a title and the other receives only anonymous body text.
Compare retrieval before answer generation
The NVIDIA family announcement describes a retrieval family, including multimodal capabilities. It does not establish performance for a private maintenance collection. First inspect whether judged pages enter the candidate set. Then examine whether returned text or images contain enough detail to answer. Generation can conceal a retrieval miss by producing a plausible generic maintenance instruction.
Diagnose representation-specific misses
If text retrieval finds the paragraph but misses the diagram question, inspect extraction coverage before changing the index. If image retrieval finds the right page but loses the critical label, inspect rendering, resizing and page selection. If both miss a known exact identifier, compare tokenization and query wording. These branches require different corrections and should remain separate evaluation categories.
Release a reproducible representation choice
The output is a small matrix of query, representation, checkpoint revision, relevant page and failure explanation. Keep text and image results separately inspectable even when a later system combines them. A larger model may improve some questions while increasing operational cost; neither direction follows from its name alone. Adopt a variant only after the paired fixture demonstrates which evidence it preserves and which failure remains unresolved.
Related reading: Embeddings and rerankers: other tasks for an idle GPU; Embedding dimensions: calculating index memory.