RAG citations: measure support and completeness separately

Citation quality has separate dimensions: whether attached sources support a claim and whether important claims receive citations at all. A response can cite an irrelevant page precisely formatted, or accurately cite one sentence while leaving the rest of its factual content unsupported.

Identify claims before counting references

The ALCE study treats citation evaluation as more than checking whether reference markers exist. Break an answer into factual claims and retain the source passages actually available to the generator. Citation markers alone reveal neither support nor completeness. A paragraph with a citation at its end may contain several claims, only some of which the referenced passage establishes.

Distinguish precision from completeness

The ALCE repository provides primary evaluation machinery. Citation precision concerns whether citations are useful support for associated statements; citation completeness concerns coverage of statements needing support. These should remain separate from answer correctness. A statement can be true from outside knowledge while unsupported by its attached source, and a citation can accurately support an outdated instruction that is inappropriate to the question.

Inspect a mixed-support answer

Suppose an answer says that a component requires inspection, that the interval is annual and that any solvent is acceptable. The cited passage establishes inspection and the interval, but names no solvent. Treat the last assertion separately rather than awarding support to the whole sentence. This illustrative fixture can expose an evaluator that assumes nearby citations support every surrounding claim.

Claim	Source relationship	Evaluation issue
Inspection is required	Explicit support	Supported claim
Interval is annual	Explicit support	Supported claim
Any solvent is acceptable	Not established	Unsupported addition

Preserve citation resolution

Store document revision, passage identity and the actual cited content. A source URL that now resolves to a new edition cannot reconstruct what supported the original answer. The official evaluation code makes evaluation assumptions inspectable; adapt the inspection process to the citation format rather than assuming all numbered references map cleanly to independent passages. Broken resolution should be reported separately from an entailment failure.

Review support and usefulness together

A final report should show unsupported claims, missing citations and unnecessary references alongside answer correctness. Manually inspect technical conditions, exceptions and negations because a cited fragment may support only a narrower statement. Avoid encouraging reference accumulation: adding many documents does not automatically make a claim better grounded. The useful target is a concise, resolvable trail from each material assertion to evidence that actually supports its scope, revision and applicability to the question.