# Kimi K2 Thinking INT4: native QAT and unpacked weights differ

Read Kimi K2 Thinking native INT4 QAT and compressed-tensors metadata correctly, distinguishing unpacked wider weights from an original master checkpoint.

Author: Retrivium team
Updated: 2026-11-03
Canonical URL: https://retrivium.ai/journal/kimi-k2-native-int4-vs-unpacked-weights/
Language: en

Kimi K2 Thinking uses native INT4 quantization in its released artifact. Unpacking that representation into a wider dtype is not equivalent to recovering a separately preserved original high-precision checkpoint.

## Understand what native INT4 means here

The [model card](https://huggingface.co/moonshotai/Kimi-K2-Thinking) describes quantization-aware training during post-training and INT4 weight-only quantization of MoE components. This is more specific than saying that every tensor in the entire model is four-bit.

The release stores the checkpoint in compressed-tensors format. Its representation includes information needed to interpret the quantized values, not just an undifferentiated array of integers. Keep the model configuration and quantization metadata with the weights.

## Separate storage precision from recovered information

Unpacking converts the stored representation into values that another implementation can consume. The destination may use FP8 or BF16, but a wider container cannot recreate information that was not preserved by quantization.

Two different original values can map to the same quantized representation. Expanding that representation later does not identify which original value existed. Quantization-aware training can make the model work well with its quantized weights; it does not turn this information boundary into reversible compression of arbitrary master weights.

## Identify the artifact being compared

| Artifact | What it represents |
| --- | --- |
| Released native INT4 checkpoint | The published quantized model representation |
| Unpacked FP8 representation | A wider representation derived from the quantized artifact |
| Unpacked BF16 representation | Another wider representation of the stored values |
| Independently preserved master weights | A separate artifact requiring its own provenance |

The [deployment guidance](https://huggingface.co/moonshotai/Kimi-K2-Thinking/blob/main/docs/deploy_guidance.md) gives checkpoint-specific examples and warns that engine implementations evolve. A larger output file after unpacking does not demonstrate a stronger model or reproduce a different reference checkpoint.

## Read quality and runtime claims separately

The model card reports evaluations under INT4. Those are publisher results for the released model conditions, not evidence that any later conversion and backend will reproduce them exactly.

The [official K2 repository](https://github.com/MoonshotAI/Kimi-K2) provides family references. For a local conversion, compare source revision, metadata and output behavior. If generated code differs, test whether both answers satisfy the same requirements before calling the difference degradation. A different token sequence can still be correct.

## Build a provenance-focused acceptance record

Record the source checkpoint, conversion implementation, destination dtype and retained quantization metadata. State whether the comparison involves unpacking the same artifact or evaluating independently obtained master weights.

Use bounded code and reasoning fixtures to assess application behavior after conversion. Keep loading compatibility and task correctness as separate outcomes. This record prevents a BF16 label from becoming an unsupported claim about restored original precision, lossless quality, universal engine support or predictable inference speed.

## Related reading

- [LLM quantization: checking quality before choosing a GPU](/journal/quantization-quality/)
- [Validate ModelOpt quantized checkpoint scales and export provenance](/journal/modelopt-quantized-checkpoint-provenance/)