In this article
Tensor parallelism and replicas solve different problems; combined VRAM does not automatically establish a usable distributed model.
In vLLM, --tensor-parallel-size configures multiple GPUs participating in distributed execution of one model. This differs from launching independent copies. Two 24GB cards do not automatically become one 48GB device; architecture support and communication add requirements.
Start by identifying why the devices should be combined. Fitting one larger model and serving more independent requests are different objectives.
Define the reason for distribution
Record the exact artifact and the limit of a single-device configuration. If the model already fits and the goal is more independent work, replicas may deserve a separate comparison. Do not assume more GPUs always reduce the latency of one answer.
Preserve numerical format, context and useful quality requirements. Changing to an easier model during the experiment does not establish that distribution solved the original problem.
Check model and runtime support
The serve reference defines the parameter, while vLLM’s optimisation guidance discusses execution choices. A syntax example is vllm serve MODEL --tensor-parallel-size 2, where MODEL must be replaced by a verified compatible checkpoint.
That example does not establish that two devices exist or that any architecture divides appropriately in that arrangement. Record software version and the chosen supported path before starting a larger test.
Inspect the physical topology
Preserve device count, available memory and the actual communication path. Two consumer cards in a desktop can have different slot conditions from a specialised server. Do not infer a link or bandwidth from the product names alone.
| Arrangement | What is placed | What to verify |
|---|---|---|
| One model across two GPUs | Parts of one execution task | Support, transfer and delay |
| Two replicas | Complete model on each device | Routing, queue and aggregate work |
| One larger GPU | Model on one device | Full footprint and useful context |
These alternatives require separate results. A combined monitoring total cannot substitute for a placement description.
Measure the complete execution
Check memory per device, startup, one short request, longer input and the intended concurrency. Record errors and communication observations. A failure on one card can precede exhaustion of the combined capacity.
Compare first-token delay and successful completed work, then test restart and return to owner use. Include host CPU, RAM and storage so preparation time is not attributed solely to GPU communication.
That record makes it possible to discuss the actual resource rather than promising single-model capacity from summed VRAM.
Continue with Two 24GB GPUs vs One 48GB GPU for an LLM and 48GB VRAM for LLMs: One GPU and Multiple GPUs Differ.