Validate ModelOpt quantized checkpoint scales and export provenance

Accept a quantized ModelOpt checkpoint together with scales, quantization configuration, and calibration provenance. A runtime dtype does not prove correct offline quantization. Preserve source revision, recipe, and export contents before validating the supported inference path. This distinguishes missing scaling tensors from unsuitable kernels or quality regressions instead of reducing every low-precision failure to VRAM pressure or another launch flag.

Separate recipe from execution

Calibration determines parameters for a selected method using particular data and settings. The ModelOpt PTQ examples describe quantization, export, and evaluation workflows. That is different from selecting a runtime dtype. Record source checkpoint, calibration-input preparation, and recipe settings. A directory named fp8 does not identify scaling granularity, transformed layers, or the procedure required to reproduce the output. Preserve those choices before inspecting results so that later behavior changes can be connected to an actual transformation rather than an ambiguous label.

Validate unified-export contents

The unified-checkpoint guide lists safetensors containing weights and scaling factors, hf_quant_config.json, and associated metadata. Accept the set together. Substituting configuration from a neighboring model merely to satisfy a loader does not establish correctness. Keep a file manifest and exporter versions. For sharded weights, verify index and every part. External checksums validate delivery integrity, not the suitability of the quantization recipe or its quality on the intended workload. Treat these as separate acceptance layers.

Create a provenance record

Include source-model commit, ModelOpt version, recipe, calibration-dataset identity, preprocessing, export format, and evaluation fixtures. Preserve a reproducible dataset description and permitted access rather than silently replacing data after an unfavorable result. Record quantized layers and expected scales where the format exposes them. This is an artifact template, not a claim about an available Retrivium checkpoint. Distinguish manually supplied parameters from tool-selected ones so that reproduction does not depend on the experiment author’s memory.

Check model and hardware support

The TensorRT-LLM quantization matrices distinguish model support from GPU support. A matching format name does not validate every architecture, recipe, and kernel combination. Run loader acceptance and short quality fixtures before benchmarking. If loading fails, locate metadata or support constraints first. If loading succeeds but quality differs, compare the source baseline and exported model on identical tasks. Successful deserialization cannot establish preserved accuracy, and a low-bit label cannot justify an invented quality-loss percentage.

Define the release boundary

Preserve artifacts, hashes, configuration, provenance, and functional-evaluation results. Runtime settings should reference that set rather than an unpinned mutable repository branch. New calibration creates a new export identity even when the public alias stays unchanged. After serving-framework updates, validate selected kernels separately from reproducibility of calibration. The deliverable connects model origin, transformation, and actual execution without collapsing them into one dtype. It supports repeatable investigation rather than promising universal acceleration or unchanged quality merely because exporting completed successfully.

LLM quantization: checking quality before choosing a GPU; Reproducible inference stacks: preserving a GPU pilot’s result.