In this article
A sum of GPU memory is not automatically one device-local memory space; specify how each card participates in the model.
Before selecting an LLM for 48GB VRAM, determine whether that means one device or the combined capacity of several cards. The difference is fundamental for a single model. Multiple GPUs require supported placement and communication; their total does not automatically become one uniform local memory space.
An equipment proposal should describe the actual architecture, not claim equivalence to a larger card from an arithmetic sum.
Inventory each device
Record accelerator count and memory per device. NVIDIA SMI can provide supported configuration information. Keep system RAM and storage in separate fields.
If a listing reports only a 48GB total, ask for the breakdown. Two cards in one host and devices at different locations present different communication and management conditions. An initial assessment does not require private keys or serial numbers.
Choose an execution arrangement
vLLM’s optimisation guidance distinguishes ways of distributing work. Model architecture, engine support and connections determine the appropriate arrangement.
| Configuration | Required check |
|---|---|
| One 48GB GPU | Full memory of one model replica |
| Two 24GB GPUs | Supported partitioning of the model |
| Two independent replicas | Each replica fits its own device |
| Remote nodes | Network and distributed-stack suitability |
Adding independent replicas can increase the number of tasks served, but it does not increase memory available to one replica. State which objective is being tested before comparing results.
Use weight arithmetic only as a starting point
An illustrative 32 billion parameters at eight bits occupy 32 billion bytes before format metadata, buffers and request state. If the model is mixture-of-experts, active parameter count does not replace the total stored weights.
Avoid selecting the largest downloadable artifact merely to fill VRAM. A smaller model with suitable quality may leave more useful context capacity and simplify operation. Record the specific model revision and format behind each estimate.
Test and report per-device behaviour
Start with a short request, then expand input and allowed output separately. Add sessions after a stable baseline. For distributed execution, monitor each GPU rather than only the average or combined total.
Measure startup and recovery as well. A distributed model may need a different restoration procedure from two independent replicas. Keep task count and success criteria fixed when comparing one large card with several smaller devices.
A suitable workload can then be discussed from the actual resource instead of an unsupported promise based on total VRAM.
Continue with Two 24GB GPUs vs One 48GB GPU for an LLM and How Much VRAM Does a 70B Model Need?.