In this article
H200 provides 141 GB HBM3e per GPU in published SXM and NVL configurations. That capacity is a runtime budget, not a model-size or concurrency guarantee.
Keep the 141 GB figure local to one GPU
The H200 product specification publishes 141 GB HBM3e per accelerator. SXM and NVL configurations share that headline capacity but belong to different server arrangements. A node total is obtained by counting devices; it does not describe one contiguous allocation available to an arbitrary process. The exact server model and accelerator identity therefore belong beside the capacity figure. A specification for H100 NVL or a Grace-based platform must not be silently substituted for H200 because the surrounding architecture names look familiar.
| Variant | Local memory, GB | Memory type | Published board power, W |
|---|---|---|---|
| H200 SXM | 141 | HBM3e | Not verified |
| H200 NVL | 141 | HBM3e | Not verified |
The product or OEM specification establishes the configuration: manufacturer source.
Validate the Hopper software environment
Use the supported driver and runtime combination required by the selected inference engine. Confirm the exact model architecture, numerical format and attention implementation, not only that a GPU appears in device enumeration. A successful first request should be compared with a reference output before the service is stressed. Record any fallback kernel or CPU offload. If partitions are used, report their visible memory and resources separately from the full physical 141 GB capacity; otherwise residency comparisons become misleading.
The software boundary is documented in the inference compatibility reference.
Use additional capacity for a defined workload
A larger HBM budget can be assigned to weights, longer context, more active requests or runtime reserves, but those uses compete. First define the input-length distribution and output limits, then measure cache growth as concurrency increases. Compare an H200 configuration with another accelerator using identical service objectives. Memory bandwidth is a device characteristic rather than a token-rate formula: scheduler decisions, kernels and workload shape remain relevant. A guide cannot establish that a particular checkpoint and request mix will fit without an actual allocation test.
Test rendering without assuming RTX equivalence
The H200 is a compute-focused accelerator. Its HBM capacity can be relevant to scene residency, but the renderer still needs a documented supported backend and device path. In Cycles, test the full scene under the chosen driver and compare output quality, feature coverage and peak memory. Do not infer a workstation display pipeline or dedicated RTX feature set from the Hopper name. An LLM service benchmark is not a substitute for a headless render test with the intended engine version.
Renderer requirements are separate in the Cycles GPU manual.
Preserve the platform boundary in conclusions
SXM modules and NVL server configurations need matching host platforms, firmware, power delivery and cooling. Evaluate sustained performance only after those prerequisites are established. Include allocator failures, service recovery and whole-server power measurements if reporting operational efficiency. Keep a configuration-specific result tied to its server rather than generalising it to every H200 installation. The catalogue records public specifications and evaluation methods; it does not assert that Retrivium operates this model or that an external routing platform has approved such a deployment.
Related reading: RAM and VRAM and a reproducible inference benchmark. GPU catalogue records the scope, model variants and unresolved specifications.
Additional primary references: 1.