In this article
A free GPU does not guarantee a schedulable or ready replica. CPU, system memory, placement and model startup need separate checks.
A cluster can show a free GPU while a model process still cannot start. Host memory, CPU, node restrictions or an unprepared environment may be responsible. Owners should describe the whole server, and funders should avoid treating cluster capacity as a simple sum of cards.
Separate resource types
Kubernetes GPU scheduling uses supported device mechanisms. Container CPU and memory have their own requests and limits. They are different requirements of the same job. System RAM is not VRAM, and virtual CPU counts should not automatically be interpreted as physical host cores.
Before launch, record GPU model, host-memory need, CPU, storage and placement constraints. Saying a job needs one card does not describe a sufficient configuration.
Test an illustrative conflict
Suppose a GPU is free but other processes occupy the RAM a replica needs. Another card in that node may not increase the number of models that can start. Establish the combined requirement for each replica first.
Keep a job profile and test a dedicated node, then add agreed background work. Compare placement waiting, startup and steady operation. Identify the failing stage: scheduler waiting, weight-loading errors and failures of an already running API require different actions.
Separate container startup and model readiness
A created process may still retrieve artifacts, prepare memory or warm up. Admit user traffic through an agreed readiness signal, not merely the container’s existence. Test restart after ordinary maintenance and loss of a component, recording when the replica actually resumes processing.
A GPU model can require more recovery stages than a small web application. Readiness also does not replace answer-quality checks: a live endpoint with an incorrectly loaded model remains unsuitable for the task.
Account for heterogeneous nodes
Different GPUs, CPU architectures and RAM sizes can need different placement conditions. Do not send one configuration to any node merely because total accelerator count seems sufficient. List acceptable variants and test each before treating them as interchangeable. Add an owner’s availability schedule and withdrawal procedure.
GPU sharing needs its own allocation model; it cannot be inferred from an ordinary device-count entry without configuration and checks. For a pilot report, record waiting, assigned node, preparation, readiness and completion. Owners can then understand where their resource participated without seeing other users’ input or confusing different forms of idle time.
Assessment notes
We do not claim every offered machine can already join a ready-made cluster.
Read about RAM and VRAM, GPU sharing and containers versus VMs.