In this article
A100 memory capacity and execution form are separate choices; establish the actual resource exposed to the process before testing.
A100 configurations include 40GB and 80GB variants, but that distinction alone is not enough to choose a server. PCIe or SXM form factor, platform compatibility and the execution stack also matter. More memory can expand placement options without implying proportional LLM acceleration.
Assess the whole machine and the resource actually available to your process, especially when a research server is shared.
Read the complete specification
NVIDIA’s A100 datasheet distinguishes memory and form-factor variants. Its figures apply to the stated configuration, not every listing abbreviated to A100.
| Check | A100 40GB | A100 80GB |
|---|---|---|
| Memory variant | Confirm exact product | Confirm exact product |
| Form factor | Establish PCIe or SXM | Establish PCIe or SXM |
| Platform compatibility | Verify supported server | Verify supported server |
| LLM performance | Measure selected task | Measure the same task |
Preserve the actual host, device count and connections. A model name is not a substitute for a supported system configuration.
Separate capacity from execution structure
Begin with the exact model artifact, weight format and request limits. For mixture-of-experts models, active parameters do not replace all stored weights. Quantisation metadata and runtime structures also remain outside a simple bits-per-parameter estimate.
Test loading separately from short inference, longer input and concurrent sessions. Capacity conclusions should state which of those stages passed. A large file fitting does not validate every context window.
Establish the sharing mode
NVIDIA’s MIG documentation describes supported partitioning. Record whether the process receives a complete card, a defined partition or another agreed arrangement. The full capacity on a product page may not be exposed to the workload.
Compare system observations with the access conditions before loading. If a partition is used, report results for that profile rather than the whole physical accelerator. Keep other users’ work and maintenance responsibilities explicit.
Measure the complete server cycle
CPU, RAM, storage and networking can limit loading and recovery independently of GPU inference. Record readiness after a normal restart, any file preparation and manual work needed. Observe sustained operation under ordinary supported cooling.
For a cost discussion, keep procurement, placement and operation separate from model throughput. A successful technical result does not predict paid utilisation.
A useful resource assessment follows the tested task and real operating conditions rather than the prestige of a datacentre GPU name.
Continue with How Much VRAM Does a 70B Model Need? and Two 24GB GPUs vs One 48GB GPU for an LLM.