# GPU VRAM, board totals and shared memory: how to read specifications

Per-GPU VRAM, a multi-GPU board total, module memory and unified CPU/GPU memory are different budgets. Adding the numbers can create a false model-fit claim.

Author: Retrivium team
Updated: 2026-11-02
Canonical URL: https://retrivium.ai/journal/gpu-vram-board-module-shared-memory/
Language: en

## Attach every memory number to an owner

A useful hardware inventory records what physically owns each memory pool and which software device can allocate it. The words GPU, board, module, node and rack are not interchangeable units. A product page may publish a total for an entire assembly while an inference process sees only one component. Preserve the original unit, capacity scope and device count. Host RAM can support preprocessing or offload, but it is not additional dedicated VRAM with identical access behaviour or performance.

## A16 is four 16 GB devices on one board

The [NVIDIA A16 specification](https://www.nvidia.com/en-au/data-center/products/a16-gpu/) describes four GPUs with 16 GB each. Its 64 GB board total cannot be used as the memory capacity of one ordinary process device. Independent workers can use different GPUs, but a model spanning them requires explicit parallelism. The same reasoning applies to other multi-GPU boards with different counts and capacities. Inventory should list both per-GPU memory and the board total so operators cannot accidentally choose a checkpoint based only on the larger number.

## MI250 divides an OAM into separate devices

AMD's [MI250 and MI250X acceptance guide](https://instinct.docs.amd.com/projects/system-acceptance/en/latest/gpus/mi250.html) identifies two GCDs per OAM, each enumerated independently and associated with 64 GB, for 128 GB per module. Module count therefore differs from the number of devices seen by the framework. Process ranks and collective communication must follow that topology. A module with several dies is not always equivalent to several devices in other architectures either: use the actual vendor-defined software device model, not a universal rule based on package photographs.

## Shared APU memory must not be counted twice

The [AMD CDNA 3 white paper](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-3-white-paper.pdf) describes MI300A with 128 GB unified CPU/GPU HBM3. CPU-side work and GPU allocations compete within a common physical budget. It is incorrect to report that as 128 GB CPU RAM plus another 128 GB of dedicated GPU memory. Integrated Radeon and other shared-memory systems similarly need platform-specific accounting, although their exact designs differ. Shared addressing also does not prove that every framework avoids copies or reaches a particular throughput.

## Distinguish physical capacity from assigned capacity

[NVIDIA MIG profiles](https://www.nvidia.com/en-us/technologies/multi-instance-gpu/) describe assigned resources rather than necessarily every physical byte of a full accelerator. A partition, virtual machine or container may expose a smaller budget. Record what the application can actually allocate after runtime reserves, then account for weights, cache and workspaces. For rendering, include geometry, textures and acceleration structures. Multi-device memory sharing requires explicit application support; a fast interconnect is not itself a command to merge pools. This accounting prevents both false model-fit and false scene-fit conclusions.

Related articles: [RAM and VRAM: which memory matters for an AI server?](/journal/ram-vs-vram/), [VRAM for LLMs: why model size is only part of the calculation](/journal/vram-and-models/). [GPU catalogue](/journal/nvidia-amd-gpu-catalog/).