# NVIDIA H100: LLM inference and server rendering boundaries

NVIDIA H100: H100 PCIe, SXM and NVL are different deployment choices. The NVL pair has 188 GB across two 94 GB GPUs, not a single 188 GB allocation.

Author: Retrivium team
Updated: 2026-11-02
Canonical URL: https://retrivium.ai/journal/nvidia-h100-llm-rendering/
Language: en

## Read the complete H100 suffix

An H100 inventory needs more than the family name. The original PCIe card has 80 GB HBM2e, the SXM configuration has 80 GB HBM3, and the NVL configuration publishes 94 GB per GPU. These memory and platform differences affect residency, cooling and connectivity. The NVL marketing total of 188 GB describes a paired configuration. A program still needs an explicit execution and memory strategy across its devices; the pair does not become an ordinary single 188 GB graphics card.

| Variant | Local memory, GB | Memory type | Published board power, W |
|---|---|---|---|
| H100 PCIe | 80 | HBM2e | 350 |
| H100 SXM | 80 | HBM3 | 700 |
| H100 NVL per GPU | 94 | HBM3 | 350–400 |

The product or OEM specification establishes the configuration: [manufacturer source](https://www.nvidia.com/en-us/data-center/h100/).

## Select an actual Hopper execution path

Hopper supports modern inference stacks, but naming the architecture does not validate every model or precision. Record the vLLM build, runtime, driver, attention backend and checkpoint revision. Test the intended numerical format rather than inferring it from a peak throughput chart. A kernel available for one quantization scheme may not exist for another. Keep the selected full-device or MIG configuration with the result: software sees its assigned memory and resources, not every physical byte advertised for the server.

The software boundary is documented in the [inference compatibility reference](https://docs.vllm.ai/en/stable/getting_started/installation/gpu/).

## Measure a service across the real topology

A paired NVL deployment and an SXM server have different host and accelerator connectivity. For tensor-parallel inference, verify which devices are peers and where collective traffic actually travels. Compare the same model, input length, output length and request arrival pattern. Report errors and tail latency along with completed output tokens. A single short prompt can conceal cache growth or communication overhead. Neither NVLink presence nor the total node memory alone establishes that a larger distributed configuration will improve user-visible performance.

## Do not treat inference certification as render support

H100 is positioned for computation rather than RTX workstation graphics. A renderer must explicitly support the installed backend, driver and device environment. For Cycles, validate a representative scene, material features and output in headless execution. Memory must cover geometry, textures and renderer workspaces independently of the LLM test. Do not assume dedicated RTX ray-tracing functionality from the Hopper family name, and do not transfer a successful CUDA compute test into a claim about every renderer plugin or commercial engine.

Renderer requirements are separate in the [Cycles GPU manual](https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html).

## Record power and cooling at configuration level

The published limits in the table describe accelerator configurations rather than electricity consumed by an entire server. An SXM module requires its designed baseboard and thermal system; it is not a PCIe expansion card. Preserve OEM power settings, firmware and cooling information with the benchmark. Run sustained requests and check thermal behaviour before calling the deployment stable. This article is a configuration guide and does not claim installed Retrivium H100 capacity, availability through OpenRouter or guaranteed output for any model.

Related reading: [RAM and VRAM](/journal/ram-vs-vram/) and [a reproducible inference benchmark](/journal/inference-benchmark-method/). [GPU catalogue](/journal/nvidia-amd-gpu-catalog/) records the scope, model variants and unresolved specifications.

Additional primary references: [1](https://developer.nvidia.com/cuda/gpus), [2](https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/h100/PB-11773-001_v01.pdf), [3](https://www.nvidia.com/content/dam/en-zz/Solutions/gtcs22/data-center/h100/PB-11133-001_v01.pdf).