# NVIDIA A100: LLM inference and server rendering boundaries

NVIDIA A100: A100 is a family of PCIe and SXM accelerators with different memory budgets. Model residency, MIG allocation and renderer eligibility need separate checks.

Author: Retrivium team
Updated: 2026-11-02
Canonical URL: https://retrivium.ai/journal/nvidia-a100-llm-rendering/
Language: en

## Identify the 40 GB or 80 GB configuration

The A100 name does not specify either memory capacity or mounting format. A server inventory should record the board identifier, form factor and physical memory before any comparison. The 40 GB and 80 GB configurations also use different published memory specifications. A PCIe card and an SXM module are not mechanically interchangeable. In the table, an unspecified power field means that this guide has not established that exact variant limit; it is not a zero-power configuration.

| Variant | Local memory, GB | Memory type | Published board power, W |
|---|---|---|---|
| A100 PCIe 40 GB | 40 | HBM2 | 250 |
| A100 SXM 40 GB | 40 | HBM2 | 400 |
| A100 PCIe 80 GB | 80 | HBM2e | 300 |
| A100 SXM 80 GB | 80 | HBM2e | 400 |

The product or OEM specification establishes the configuration: [manufacturer source](https://www.nvidia.com/en-us/data-center/a100/).

## Keep full-device and MIG measurements separate

A100 supports Multi-Instance GPU, but a partition has its own resource and memory limits. A model that fits on an unpartitioned accelerator may fail in a smaller MIG instance. Record the actual profile, visible devices, framework version and numerical format alongside the benchmark. Current vLLM architecture eligibility is only an initial gate: individual attention implementations, quantization paths and model architectures can impose additional requirements. An installation that imports successfully has not yet demonstrated correct sustained inference.

The software boundary is documented in the [inference compatibility reference](https://docs.vllm.ai/en/stable/getting_started/installation/gpu/).

## Budget for requests rather than weights alone

Weights are only the starting allocation. Runtime workspaces, the KV cache, activation buffers and fragmentation consume part of the remaining device memory. Compare 40 GB and 80 GB cards using the same model revision, context distribution and concurrency target. Increasing context and active requests together can exhaust the extra capacity. If tensor parallelism is used, document the host topology and actual collective communication behaviour; the number of installed boards does not establish an efficient distributed service.

## Qualify rendering as a separate workload

A100 is a compute accelerator rather than an RTX graphics workstation product. CUDA capability does not establish parity with a graphics-oriented card or confirm every renderer feature. For Cycles, select a supported device backend and driver, load the complete scene and verify output against a reference. Include acceleration structures, textures, geometry and denoising allocations in the scene budget. A successful CUDA inference run cannot replace this render test, and a render result says nothing about a serving engine's kernel coverage.

Renderer requirements are separate in the [Cycles GPU manual](https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html).

## Retain the actual server configuration

The practical comparison unit is a qualified server configuration. Cooling, board power limits, host memory, PCIe placement and firmware belong in the record. After a sustained workload, check errors, throttling, request failures and restart recovery before interpreting throughput. Preserve the exact MIG or full-device state so another operator can reproduce it. This guide describes evaluation criteria; it does not claim that Retrivium currently owns any particular A100 variant or that a named model will fit without measurement.

Related reading: [RAM and VRAM](/journal/ram-vs-vram/) and [a reproducible inference benchmark](/journal/inference-benchmark-method/). [GPU catalogue](/journal/nvidia-amd-gpu-catalog/) records the scope, model variants and unresolved specifications.

Additional primary references: [1](https://developer.nvidia.com/cuda/gpus), [2](https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/A100-PCIE-Prduct-Brief.pdf), [3](https://images.nvidia.com/data-center/a100/a100-datasheet.pdf).