In this article
Comparing drivers requires holding other factors steady and preserving compatibility. If CUDA libraries, engine and image change too, the result concerns the entire update. Design a controlled A/B experiment with recovery and correctness checks.
Comparing drivers requires holding other factors steady and preserving compatibility. If CUDA libraries, engine and image change too, the result concerns the entire update. Design a controlled A/B experiment with recovery and correctness checks.
Preserve the baseline
Record GPU UUID, device model, driver version, firmware identity, OS/kernel, image digest and runtime. Retain workload, inputs, settings and timing method. Nvidia-smi reports driver information, but its displayed CUDA version should not automatically be treated as the toolkit version inside a container. Identify application libraries separately. If the original environment cannot be restored, declare that limitation before comparing series.
Check both versions’ compatibility
Review application and container-stack requirements before installation. If version B needs another dependency, record a coupled change rather than hiding it under “driver test.” Preserve node recovery and an agreed maintenance window. This article does not prescribe a universal kernel-module replacement command: procedure depends on OS, packages and active GPU clients. Shared-node maintenance must include applications outside the benchmark.
Build matched trial series
An illustrative design runs series A on fixed inputs, then B, then A again where practical. Returning to A helps detect drift but cannot eliminate every unobserved factor. Keep warmup and cache policy consistent. CUDA execution is often asynchronous, so CPU timing needs suitable synchronization for the chosen scope. Record end-to-end latency and compute time separately rather than combining them as one speed value.
| Comparison field | Purpose |
|---|---|
| Driver and image | Identify the changed factor |
| Inputs and warmup | Preserve matched work |
| Errors and result checks | Retain correctness evidence |
| Trial order | Reveal possible time drift |
Compare more than the mean
Show individual runs, median, spread and failures. A version can affect stability, memory use or observability without changing average speed substantially. One successful generation is not a complete application test. Compare startup, steady work and graceful exit separately. Do not attribute differences to the driver when clocks, power policy or competing load also changed. Locate those changes and repeat the relevant comparison first.
State a bounded conclusion
The artifact should reconstruct A and B and permit summary calculation from run-level data. Identify isolated and confounded changes. Do not publish an invented speedup without measurement. Updating depends on verified compatibility and the intended workload rather than universal preference for newer versions. Retain the chosen baseline for future maintenance so the next comparison uses actual recorded states rather than a memory of one favorable run. If rollback was not tested, state that separately from the observed performance result.
Related operational reading: software stack reproducibility; gpu maintenance change register.