# Multiple GPUs in a pool: one service or one large model?

Why a distributed GPU pool is not one graphics card: independent replicas, model distribution, memory limits and the role of connections between nodes.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/multi-gpu/
Language: en

“Pool” describes resource organization but does not explain model execution. Several machines may serve different models or independent copies of one model. Alternatively, several accelerators may jointly execute one model. These approaches have different hardware and network requirements.

For owners, the distinction matters: two separate cards do not necessarily replace one card with twice the memory.

## Independent replicas

Each node runs a model that fits its resources. Requests are distributed between nodes at service level. This adds parallel-processing capacity without automatically increasing the maximum model size on an individual node.

Different machines may need different limits. If one replica is slower, the service should account for its characteristics rather than send identical workloads without checking.

## Distributing one model

When a model does not fit on one GPU, or another execution profile is required, distributed execution methods are used. The method depends on the engine, model architecture and device interconnects.

This execution involves data exchange. Internal server topology, bandwidth and connection latency therefore matter. GPUs at separate home sites cannot be treated as equivalent to an interconnected server merely by adding their VRAM.

## What to include in an equipment description

Alongside GPU count, state whether the cards are in one server, which interfaces they use and whether nodes have a dedicated network. If topology is unknown, say so: it can be checked during technical assessment.

Do not buy interconnects before selecting a workload. First determine whether one model needs distribution or independent replicas are more useful. Measurement, rather than the desire to combine all available hardware, should decide.

## How to assess the pilot

Compare at least two available scenarios: independent GPU operation and an agreed distributed run. Record model, parameters, context and test-request mix. Measure stability when one component fails as well as speed.

Finish with a diagram showing where weights load, where requests arrive and which nodes an answer needs. It makes limitations clear to both owner and operator.

Read about [VRAM](/journal/vram-and-models/) and [networking](/journal/network-and-uptime/) to prepare.