# Gemma 4 QAT unquantized vs GGUF: choose the right artifact

Distinguish Gemma 4 unquantized QAT weights from ready-to-run GGUF Q4_0 artifacts, their intended uses and task-quality acceptance.

Author: Retrivium team
Updated: 2026-11-03
Canonical URL: https://retrivium.ai/journal/gemma4-qat-unquantized-vs-gguf/
Language: en

The name unquantized QAT can look contradictory. For Gemma 4 it identifies half-precision weights from a quantization-aware-training pipeline, not a ready-to-run four-bit file merely because Q4_0 appears in the repository name.

## Read the official artifact description

The [QAT card](https://huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-unquantized/raw/main/README.md) distinguishes unquantized QAT checkpoints from GGUF artifacts. It describes the former as half-precision weights intended for downstream compilation and research, while the GGUF Q4_0 versions are prepared for deployment in the corresponding ecosystem.

The [ordinary model card](https://huggingface.co/google/gemma-4-26B-A4B-it/raw/main/README.md) identifies the parent instruction model. The [family overview](https://ai.google.dev/gemma/docs/core) provides model architecture context. A QAT-derived artifact and the parent checkpoint should not be treated as identical simply because both can be represented in a wide dtype.

## Separate training method from storage format

QAT changes training or adaptation so that a model can tolerate the intended quantized execution conditions. GGUF is an artifact format used by compatible runtimes. These describe different axes.

An unquantized QAT file can contain values prepared through the QAT pipeline without already being serialized as the final low-bit GGUF. A filename alone does not prove which tensor representation will be loaded or which runtime can use it.

## Choose the artifact for the actual operation

| Intended operation | Artifact question |
| --- | --- |
| Direct compatible GGUF inference | Is this the published ready-to-run GGUF? |
| Downstream compilation | Is the unquantized QAT source appropriate? |
| Parent-model comparison | Is the original checkpoint separately identified? |
| Custom conversion | Which tool and settings produced the new output? |
| Reproducible evaluation | Are all source revisions and formats recorded? |

Availability varies across family members and artifact types. Do not infer that every listed format exists for every size merely from the family-level description.

## Validate behavior after the chosen path

Use a bounded set of text instructions and code cases with independent references. Check preserved facts, valid final answers and protected code behavior. Different wording can remain correct; similar wording can still contain a regression.

If compilation or conversion fails, record that separately from task-quality failure. If the model loads but output is malformed, inspect the tokenizer and template before attributing the issue to QAT. Artifact identity, input preparation and answer quality are separate checks.

## Preserve a useful provenance record

Keep the parent repository, selected QAT or GGUF revision, conversion or compilation path and runtime with the evaluation record. State whether the artifact was used directly or transformed locally.

The result can justify the selected format for a checked workload. It does not establish universal losslessness, equivalent memory use across wide and low-bit artifacts or automatic compatibility with any engine that recognizes the model family. Reading the official artifact purpose prevents an attractive repository name from substituting for an actual deployment format.

## Related reading

- [Choosing a GGUF file for llama.cpp](/journal/llama-cpp-gguf-selection/)
- [Validate ModelOpt quantized checkpoint scales and export provenance](/journal/modelopt-quantized-checkpoint-provenance/)