In this article
Before moving a llama.cpp CUDA binary to another GPU, check its compiled architecture targets.
Before moving a llama.cpp CUDA binary to another GPU, check its compiled architecture targets. A missing-kernel-image error can concern build targets rather than damaged weights. Account for the host CPU instruction set separately. Preserve compiler, Toolkit, CUDA architectures, and GGML_NATIVE, then validate the executable on the destination. A matching filename and successful file copy cannot establish portable inference.
Separate host and device code
The executable contains CPU code and GPU kernels. The llama.cpp build guide documents CUDA architecture controls and native optimization. A build tailored to the source CPU can be unsuitable elsewhere independently of its GPU support. Distinguish failure to start the host executable from failure during the first CUDA computation. Replacing model files cannot repair an unavailable CPU instruction or missing device kernel. Those errors arise at a different boundary and need the corresponding build evidence.
Inspect CUDA targets
The NVCC guide explains PTX, Cubin, and multiple targets within a binary. Native compilation is not a universal portability guarantee. Match destination compute capabilities against both Toolkit support and included targets. Do not invent an SM number from a marketing generation name. Preserve effective CMake values and compiler output so that an intentionally restricted build can be distinguished from accidental automatic selection on the source node. The intended target list and the emitted code are separate observations.
Create a transfer manifest
Record host CPU target, Toolkit version, compiler, CMAKE_CUDA_ARCHITECTURES, GGML_NATIVE, and intended machines. Populate the table from build evidence rather than desired configuration. For each node, add help-command startup, backend discovery, and a short functional request. This is a validation template, not a promise for every architecture. Successful help output tests only part of the host path; it does not execute all GPU kernels and therefore cannot serve as the complete inference smoke test.
Follow the failure branches
If the executable cannot start, inspect host dependencies and CPU ISA. If it starts and sees the GPU but fails during computation, examine CUDA targets and runtime compatibility. If loading succeeds but an operation is unsupported, consult the particular backend and model architecture. Avoid collapsing these cases into “CUDA is broken.” Preserve the first meaningful error and build metadata before reinstalling, because an unrelated replacement binary can hide the cause and destroy reproducibility of the investigation.
Accept the destination build
Repeat checks on each claimed device class and retain the binary checksum with its manifest. Benchmark separately: broader compatibility does not guarantee identical kernels or speed across GPUs. Recheck supported targets after Toolkit updates because toolchain support changes. State which machines were actually tested, which remain planned, and which the selected toolchain excludes. Neither VRAM capacity nor vendor branding establishes architecture compatibility. This record tells an operator when rebuilding is necessary instead of repeatedly changing quantization files or context settings unrelated to missing device code.
Related reading
Install vLLM: Check GPU, Driver and Package Compatibility; Building llama.cpp on Windows with the right backend.