In this article
If a GPU works in a rootful container but fails in rootless mode, inspect daemon configuration, user namespaces and device injection separately. Rootless is not ordinary rootful Docker with a different client user. Privileged mode does not explain the original access boundary.
If a GPU works in a rootful container but fails in rootless mode, inspect daemon configuration, user namespaces and device injection separately. Rootless is not ordinary rootful Docker with a different client user. Privileged mode does not explain the original access boundary.
Confirm the selected daemon
Record Docker context, endpoint and server mode. A rootless daemon can use its own user configuration instead of /etc/docker/daemon.json. Configuring the rootful runtime does not establish that the user daemon adopted it. Preserve runtime and versions without exposing context credentials. Confirm that comparison commands actually target the intended different environments rather than accidentally using one rootful endpoint.
Separate permission from injection
NVIDIA documents special rootless runtime configuration. CDI improves compatibility with rootless scenarios but still requires a valid specification and engine support. Record device nodes, ownership, relevant groups and container UID mapping. Seeing a /dev path does not establish permission to open it. Do not transfer no-cgroups from one documented mode to every runtime without understanding its architectural implications.
Examine configuration mismatch
Suppose nvidia-ctk configured system Docker while testing uses a user’s rootless daemon. The GPU works on the host, but the user environment lacks the intended runtime configuration. Comparing config paths and actual engine identifies the mismatch. In another test, a CDI name resolves but the process lacks device access. Changing the device list cannot replace UID/GID and permission checks. These are distinct scenarios, not a universal “use sudo” instruction.
| Scope | Evidence required |
|---|---|
| Engine | Intended rootless endpoint is selected |
| Runtime | Configuration applies to that daemon |
| CDI/device | Device is described and injected correctly |
| Process | Permissions allow the required open |
Apply a supported configuration
Follow guidance for the particular engine and NVIDIA mode. Modify the correct config path and restart the relevant user service when required. Retain prior values and assess other workloads. Do not disable SELinux, seccomp and cgroups simultaneously to obtain one successful launch: that obscures cause and broadens several boundaries at once. Permissions should remain minimal and linked to an observed device operation.
Repeat without root
Create a new test container from a pinned image, verify UUID and execute a small compute check. Preserve daemon endpoint, injection mode and process identity. Rootful success is a control observation, not proof of rootless configuration. Repeat scope checks after engine or Toolkit changes, especially CDI refresh and node access. The artifact should let the intended user repeat startup without an unnoticed dependency on interactive sudo or accidental selection of another daemon. This makes the corrected boundary explicit for later maintenance.
Related operational reading: gpu containers vs vms; hardware security.