In this article
In multi-socket systems, CPU, memory and GPU paths can affect an application. Test placement with the workload rather than applying one universal setting.
A multi-socket server may have enough CPU, RAM and several GPUs, yet its result depends on more than component counts. Device connections and workload placement matter. For an equipment owner, this is a useful pilot question: inspect the existing configuration before buying additional components.
Understand the NUMA relationship
In a NUMA system, a processor can incur different costs when accessing different memory regions. Thread placement, RAM allocation and accelerator connections matter to an application. The actual picture depends on the platform, operating system and software stack.
vLLM optimization guidance includes recommendations for multi-socket placement. NVIDIA NCCL documentation directs communication troubleshooting toward topology and specific exchange paths. These are reasons to measure, rather than promises that any process affinity setting produces the same improvement.
Map the server
Record sockets, memory regions, GPUs and network interfaces. Identify connections as well as device models. If the owner does not know the topology, it can be established during technical assessment.
Separately preserve virtualization settings and resources visible to the guest. A VM’s view can differ from physical placement, so conclusions should not automatically transfer between operating modes. Avoid publishing detailed internal addresses in an initial equipment proposal; a general structure and known constraints are sufficient for the first discussion.
Compare placement under control
Choose one model and a repeatable request set. Save the baseline, then test an agreed CPU and memory placement while holding other settings constant. Measure input preparation, first-token time, complete responses and errors. Include several concurrency levels: one short request may miss restrictions that appear when processes run together.
Repeat the baseline after the series. This helps reveal temperature or background-load changes that might otherwise be attributed to NUMA configuration. Keep commands and observations with each result so the comparison can be explained rather than reconstructed from memory.
Include neighboring workloads
The owner’s server may prepare data or run research processes alongside inference. Their placement can change the result, so test both an isolated mode and the actual shared mode. Do not allocate every available thread to one process without checking: extra parallelism can increase memory contention and deprive supporting operations of resources.
Agree changes with the site operator so optimization of external work does not silently degrade the owner’s tasks. For every tested scheme, record launch commands and visible devices. Check placement again after reboot where needed. A repeatable procedure is more valuable than a one-off successful setting known only to one specialist.
Use the result
The outcome should be a working-configuration record with placement, limitations and restoration steps. It helps an operations funder distinguish a configuration task from a new hardware requirement.
Continue with memory bandwidth and a reproducible stack.