# Energy per LLM request: measuring without false precision

Measure energy per useful inference task with explicit device or server boundaries, time windows, power integration and repeatable workload conditions.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/measuring-energy-per-inference/
Language: en

GPU owners need electricity costs, while operators need useful work delivered for that consumption. Energy per request helps only when its measurement is clear. A single number without workload context can suggest precision the experiment lacks.

Short answers, long documents and batches use resources differently. Treating them as identical requests is usually uninformative.

## State the measurement boundary

Distinguish GPU telemetry, whole-server consumption and site consumption, which may include cooling and infrastructure. [NVIDIA NVML](https://docs.nvidia.com/deploy/nvml-api/latest/) provides device observations whose availability depends on hardware and environment.

Without measurement at the server input, name the actual observed boundary. Do not label GPU telemetry the entire site's electricity consumption merely because it is the only available measurement.

## Separate power and energy

Power is a consumption rate; energy covers a period. Estimating energy from power samples requires known intervals and a clear integration method. One short maximum sample is not the average for the whole test.

For illustration, a selected boundary averaging 100 watts for ten seconds consumes 1,000 joules. If ten identical tasks complete successfully then, the simple ratio is 100 joules per task.

## Define useful work

Choose a successful request of a particular profile, a defined processing volume or another appropriate unit. Record errors and unfinished work separately. Lower consumption with many failures does not demonstrate better service efficiency.

Compare request counts with [vLLM metrics](https://docs.vllm.ai/en/stable/usage/metrics/), checking each counter's meaning in the installed version. Match time windows: energy over an hour divided by answers over a few minutes is not a valid ratio.

## Run equivalent series

Preserve model, weights, context and concurrency. Test idle, warmup and working traffic separately. Across configurations, keep result quality and latency constraints part of the conditions.

Repeat the series and inspect variation. A small change relative to observed variability warrants a cautious conclusion; reproducibility matters more than one favorable minimum. Define idle-baseline treatment beforehand. You may report total-period consumption and incremental estimates separately, but must not silently change method between series. Give both their own names, boundaries and calculation assumptions.

## Use energy in economics

Electricity informs one cost component, not hosting, maintenance or demand. Connect it to [unit economics](/journal/inference-unit-economics/) and [the operating budget](/journal/power-and-cooling/).

A complete laboratory report is not required for initial contact.