# What hardware is suitable for an LLM inference pool?

Five areas to assess in a GPU server: video memory, software compatibility, host resources, hosting conditions and predictable availability.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/gpu-requirements/
Language: en

Replace “Is my graphics card suitable?” with “What workload can this machine handle reliably?” Identical GPUs at different sites may perform differently because of cooling, power limits and server settings.

The NVIDIA RTX, Tesla and A-series names on the home page are examples of families to discuss, rather than a promise to connect every card.

## GPU and video memory

Provide the exact variant, card count and VRAM on each card. For several GPUs, add their interconnect and placement in the server. Adding the memory figures on labels does not automatically describe memory available as one device.

Weight format, context and concurrent request count matter. Running a small model does not establish that a large model can be served under the same quality and latency constraints.

## Software environment compatibility

Check whether the chosen engine supports the GPU architecture, driver version, operating system and launch method. Compatible device lists and execution methods change, so make the final decision against a fixed environment version.

Do not update a working research server at random. First record its dependencies and define how to restore its original state. Pilot preparation must not disrupt the owner’s existing work.

## The rest of the machine

Provide CPU, system memory and free disk specifications. The host loads the model, prepares requests and runs the API and logs. Allow space for a new model version too: an update may temporarily require several sets of files.

There is no need to buy extra components at the first stage. Identify the bottleneck through measurement before discussing upgrades.

## Site and availability

Describe power, cooling, internet and physical maintenance access. Maximum figures are not enough; ordinary behavior matters, including overnight interruptions, provider restrictions and response time after a failure.

If you need the machine on weekdays, say so immediately. A predictable schedule is more useful than a promise of continuous availability that cannot be kept.

## Assessment notes

Prepare a configuration sheet and a list of limitations. Do not send serial numbers, internal IP addresses or passwords unnecessarily: they are usually not required for an initial decision.

Continue with [server preparation](/journal/server-preparation/), [network and availability](/journal/network-and-uptime/) and the [onboarding pilot](/journal/pilot-checklist/).