In this article
Compare CUDA event timing with request duration only after defining interval boundaries and completion dependencies.
Name the measured interval
HTTP request duration includes the client, network, queue, data preparation, execution and response delivery. CUDA events measure a selected interval associated with device events. Those values can differ without any fault. State the question before running: kernel duration, an operation chain, server processing or the complete request? Do not label every chart simply latency. For each measurement, identify its start, end, units and completion conditions. A small device interval then cannot become an unsupported claim that users must receive answers in the same time regardless of queueing and transport.
Measure host duration monotonically
For duration on one machine, select an appropriate monotonic time source and keep calendar timestamps separately for correlation. The clock_gettime documentation explains Linux clock distinctions. A CPU interval around an asynchronous call may measure only submission if its ending boundary does not include completion. A control example should therefore state whether the host waits for the result. Do not subtract clocks from different servers and call the result precise duration without additional synchronization conditions. A shared request identifier links events logically, but does not make separate machines’ time scales suitable for exact arithmetic by itself.
Place CUDA events correctly
The CUDA Runtime Event API describes recording events, waiting for completion and obtaining elapsed time. Place events relative to specific work and dependencies, then verify that the timing result is ready. If an end event sits in another stream without the necessary dependency, the interval may not cover the intended chain. Consider resolution and suitability for short operations as well. The measurement diagram should show streams, events and waits rather than just two timer calls. Do not automatically describe the interval as pure kernel arithmetic time: other work or waiting can occur between events according to the chosen execution order.
Investigate discrepancies by phase
The diagnostic artifact is a timeline of request acceptance, work submission, device events, result readiness and delivery completion. If device time remains stable while server duration grows, inspect queueing and host preparation. If device time changes, inspect competing operations and dependencies. If only the external request changes, add network and client observations. Host duration minus event duration is not mechanically equal to CPU time: intervals may overlap and cover different work. Decomposition helps only with consistent boundaries. For streaming, record first meaningful content separately from complete response termination, since they represent different user events.
Present comparable results
Retain versions, work sizes, warmup, concurrency and synchronization rules. See the inference benchmark method and vLLM bench serve. Compare optimizations using the same user visible interval, while showing changed internal phases separately. Event timing alone does not establish an end-to-end SLO, and a successful external answer does not reveal the GPU’s internal time distribution. Readers should be able to identify exactly which event ended each timeline. This prevents apparent chart acceleration that occurred only because the new measurement stopped waiting for work that was still required to produce the result.