Recommended Free Tools
To use an NVIDIA GPU in a Docker container, install and verify the NVIDIA driver on the host, install the NVIDIA Container Toolkit, configure Docker, and start the container with the –gpus option. You do not need to install the full CUDA Toolkit on the host just to run a CUDA container.
This guide covers Linux, rootless Docker, and Docker Desktop on Windows with WSL 2. NVIDIA lists version 1.19.1 as the current NVIDIA Container Toolkit release in its release notes.
What you need before starting
| Requirement | Details |
|---|---|
| NVIDIA GPU and driver | A supported GPU and a working NVIDIA driver installed on the host. |
| Docker | Docker 19.03 or later for the –gpus option. |
| NVIDIA Container Toolkit | Install the toolkit on Linux hosts using the steps below. |
First, check the host driver by running nvidia-smi. It should display the GPU and driver information. If the command is missing or cannot communicate with the driver, fix the host driver before configuring Docker.
Install the NVIDIA Container Toolkit on Debian or Ubuntu
Configure NVIDIA’s production APT repository:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg –dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed ‘s#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g’ | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
Free tools Windows power users keep installed
One-click scans. No signup required.
#1 Best Overall
- AI Performance: 767 AI TOPS
- OC mode: 2632 MHz (OC mode)/ 2602 MHz (Default mode)
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- A 2.5-slot design maximizes compatibility and cooling efficiency for superior performance in small chassis
Update the package index and install the toolkit:
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
The experimental repository is normally unnecessary. If you specifically need it, enable its entry with:
sed -i -e ‘/experimental/ s/^#//g’ /etc/apt/sources.list.d/nvidia-container-toolkit.list
Install it on RHEL, CentOS, Fedora, or Amazon Linux
Configure the NVIDIA RPM repository:
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
Install the toolkit with:
sudo dnf install -y nvidia-container-toolkit
To enable the experimental repository when required, run:
sudo dnf-config-manager –enable nvidia-container-toolkit-experimental
Configure Docker to use the NVIDIA runtime
On Linux, configure Docker with:
sudo nvidia-ctk runtime configure –runtime=docker
This updates Docker’s /etc/docker/daemon.json configuration. Restart Docker to load the change:
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →sudo systemctl restart docker
The standard setup uses nvidia-ctk runtime configure; it does not require the older nvidia-docker2 wrapper. For background, see NVIDIA’s installation guide.
Run a GPU container
Test GPU access with a tagged CUDA image:
docker run –rm –gpus ‘all,”capabilities=compute,utility”‘ nvidia/cuda:12.5.0-base-ubuntu22.04 nvidia-smi
A successful result lists the GPU from inside the container. The –rm option removes the container when it exits. For a quick, less-pinned test, NVIDIA also documents docker run –rm –gpus all nvidia/cuda nvidia-smi. Pin an image tag for repeatable scripts and deployments. See NVIDIA’s Docker GPU guide.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070 Ti
- Integrated with 16GB GDDR7 256bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Select which GPUs a container can use
Docker supports selecting a number of GPUs, specific indexes, or GPU UUIDs.
Allocate a number of GPUs
To give a container two GPUs, run docker run –rm –gpus 2 nvidia/cuda nvidia-smi. Docker selects two available GPUs; this does not specify a particular pair.
Allocate specific GPU indexes
To expose GPU indexes 1 and 2, use the documented nested quoting:
docker run –rm –gpus ‘”device=1,2″‘ nvidia/cuda nvidia-smi
The single quotes protect the double-quoted device expression from the shell.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Use a GPU UUID
Indexes can change. Query a GPU UUID on the host with nvidia-smi -i 3 –query-gpu=uuid –format=csv, then pass the returned value to Docker, for example:
docker run –rm –gpus device=GPU-18a3e86f-4c0e-cd9f-59c3-55488c4b0c24 nvidia/cuda nvidia-smi
Control GPU visibility with NVIDIA_VISIBLE_DEVICES
NVIDIA_VISIBLE_DEVICES controls which GPUs the NVIDIA runtime makes visible. It accepts indexes or UUIDs separated by commas. For example:
docker run –rm –runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=1,2 nvidia/cuda nvidia-smi
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitches| Value | Effect |
|---|---|
| 0,1,2 | Expose the listed GPU indexes. |
| GPU-fef8089b | Expose a GPU by UUID. |
| all | Expose every GPU. |
| none | Expose no GPU devices but enable requested driver capabilities. |
| void, empty, or unset | Expose neither GPUs nor NVIDIA driver capabilities; behavior is like runc. |
When using this variable, –runtime=nvidia may be needed unless the NVIDIA runtime is configured as Docker’s default. For ordinary Docker commands, –gpus all is the simpler interface.
Limit driver capabilities
NVIDIA_DRIVER_CAPABILITIES determines which NVIDIA libraries and binaries are mounted into the container.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
- compute: CUDA and OpenCL.
- compat32: 32-bit applications.
- graphics: OpenGL and Vulkan.
- utility: nvidia-smi and NVML.
- video: Video Codec SDK.
- display: X11 display support.
- all: all available capabilities.
If the variable is unset or empty, the default is utility,compute. To request CUDA and NVML capabilities for selected GPUs, run:
docker run –rm –runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=2,3 -e NVIDIA_DRIVER_CAPABILITIES=compute,utility nvidia/cuda nvidia-smi
CUDA image and host-driver compatibility
A CUDA container supplies user-space CUDA components but still uses the host’s NVIDIA kernel driver. The driver must support the CUDA runtime required by the image. Official CUDA images set NVIDIA_REQUIRE_CUDA; if the host driver does not meet the image’s requirement, the container normally refuses to start.
Constraint examples include cuda>=11.0 and driver>=450. Within one constraint variable, space-separated constraints are ORed and comma-separated constraints are ANDed. Multiple NVIDIA_REQUIRE_* variables are ANDed.
For CUDA base images older than CUDA 11.7, the requirement check can be disabled with -e NVIDIA_DISABLE_REQUIRE=true. This disables the check only; it does not make an incompatible driver work. Upgrading the driver is the proper fix.
Rootless Docker
Rootless Docker uses a per-user daemon configuration. Configure it with:
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →nvidia-ctk runtime configure –runtime=docker –config=$HOME/.config/docker/daemon.json
Restart the user Docker daemon:
systemctl –user restart docker
NVIDIA also documents disabling cgroups for the NVIDIA Container CLI:
sudo nvidia-ctk config –set nvidia-container-cli.no-cgroups –in-place
Then repeat the GPU test using the rootless Docker context. Rootless mode has additional permission and device-access constraints; if the test works as root but not as your user, check the user daemon and device permissions.
Docker Desktop on Windows
Docker Desktop GPU support on Windows requires Linux containers through the WSL 2 backend and a current NVIDIA Windows driver that supports GPU access through WSL 2. Docker describes this as NVIDIA GPU paravirtualization in its GPU support documentation.
Rank #4
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5060
- Integrated with 8GB GDDR7 128bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
- Install or update the NVIDIA Windows driver.
- In PowerShell or Windows Terminal, run wsl –update.
- In Docker Desktop, open Settings → General and enable Use WSL 2 based engine, then select Apply. On supported systems, this may already be enabled and the setting may not appear.
- Open Settings → Resources → WSL Integration, select the Linux distribution that will run Docker commands, and select Apply.
Docker’s WSL prerequisites specify WSL 2.1.5 as the minimum and recommend using the latest version. If WSL Integration is missing under Resources, Docker Desktop may be in Windows-container mode. Use the Docker taskbar menu and choose Switch to Linux containers.
Before installing Docker Desktop, remove Docker Engine or a Docker CLI installed directly inside a WSL distribution to avoid daemon and context conflicts. Validate GPU access from PowerShell with Docker’s sample:
docker run –rm -it –gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
Do not install the Linux NVIDIA Container Toolkit inside WSL merely because the container is Linux-based. Docker Desktop provides GPU integration through the Windows driver, WSL 2, and its Linux-container backend. See Docker’s WSL documentation.
Common errors and fixes
APT reports “Conflicting values set for option Signed-By”
This usually means an old NVIDIA repository file remains alongside the newer signed repository entry. Find duplicates with:
grep “nvidia.github.io” /etc/apt/sources.list.d/*
grep -l “nvidia.github.io” /etc/apt/sources.list.d/* | grep -vE “/nvidia-container-toolkit.list$”
Remove obsolete duplicate list files, such as libnvidia-container.list, nvidia-docker.list, or nvidia-container-runtime.list. Keep the current nvidia-container-toolkit.list entry, then run sudo apt-get update again. NVIDIA explains this in its troubleshooting guide.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11NVML reports “Insufficient Permissions” on RHEL or another SELinux system
Use Docker directly rather than the old wrapper:
sudo docker run –gpus=all –runtime=nvidia –rm nvcr.io/nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
If SELinux still blocks access, NVIDIA documents trying –security-opt=label=disable. This disables SELinux separation for the container, so treat it as a security trade-off rather than a routine setting.
“Failed to initialize NVML: Unknown Error” after an update
The NVIDIA runtime hook can modify a container after the low-level runtime creates it. Updating an existing container can remove injected GPU access; delete and recreate the container rather than repeatedly restarting it. On systemd-managed hosts, systemctl daemon-reload can also trigger the issue.
A Docker workaround for GPU loss after a daemon reload is to set the cgroup driver in /etc/docker/daemon.json to native.cgroupdriver=cgroupfs, then restart Docker. This does not prevent GPU loss caused by explicitly updating a container. CDI avoids that particular update behavior because device nodes are part of the container configuration.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Best Value
- Powered by the NVIDIA Blackwell architecture and DLSS 4 OC mode: 2640MHz/Default mode: 2610MHz (Boost Clock)
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.125-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
“groups: cannot find name for group ID ####”
This warning is generally cosmetic: the toolkit injects host device-node group IDs, but matching group names may not exist in the container’s /etc/group. Device access can still work. NVIDIA documents a configuration option, no-additional-gids-for-device-nodes = true, to suppress the warning for runtime injection. However, it can prevent non-root users from accessing devices whose permissions depend on group ownership, including video and rendering devices.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Toolkit version and CDI notes
NVIDIA Container Toolkit 1.18.0 changed the default runtime mode from legacy to just-in-time-generated CDI specifications. Legacy mode remains supported but is deprecated. The 1.19.1 packages are nvidia-container-toolkit, nvidia-container-toolkit-base, libnvidia-container-tools, and libnvidia-container1. Since 1.18.0, the libnvidia-container and nvidia-container-toolkit package versions must match.
Toolkit 1.19.1 generates CDI specifications using schema version 0.7.0 by default. NVIDIA lists support for that schema in containerd 1.7.16 or later, Docker 26.1.0 or later, Podman 5.1.0 or later, and CRI-O 1.30.0 or later. With an older runtime, NVIDIA recommends generating the specification with sudo nvidia-ctk cdi generate –feature-flag no-additional-gids-for-device-nodes. For a normal Docker installation, manual CDI generation is usually unnecessary; use nvidia-ctk runtime configure –runtime=docker.
FAQ
Do I need to install the CUDA Toolkit on the Docker host?
No. For running CUDA containers, the host needs a working NVIDIA driver and the NVIDIA Container Toolkit. The CUDA user-space runtime normally comes from the container image.
Why does Docker say that the –gpus option is unknown?
The Docker CLI GPU option requires Docker 19.03 or later. Check the version with docker version, then update Docker if the client or engine is older.
Should I install nvidia-docker2?
Not for the standard setup described here. Install nvidia-container-toolkit, configure Docker with sudo nvidia-ctk runtime configure –runtime=docker, restart Docker, and use ordinary docker run –gpus … commands.
Can one container use only one GPU on a multi-GPU host?
Yes. Use an index, such as –gpus ‘”device=1″‘, a GPU UUID, or set NVIDIA_VISIBLE_DEVICES to the required index or UUID.
Why does nvidia-smi work on the host but not in the container?
Check that the toolkit is installed, Docker was configured with nvidia-ctk runtime configure, and Docker was restarted. Then check CUDA and driver compatibility and test with a tagged CUDA image.
Does Docker Desktop GPU support work with Windows containers?
No. Docker Desktop GPU support requires Linux containers running through the WSL 2 backend. If WSL Integration is unavailable, switch Docker Desktop to Linux containers.
The Bottom Line
On Linux, verify nvidia-smi on the host, install nvidia-container-toolkit, run sudo nvidia-ctk runtime configure –runtime=docker, restart Docker, and test a tagged CUDA image with –gpus all. Select individual GPUs with indexes or UUIDs, and check CUDA-driver compatibility and SELinux permissions when troubleshooting.
On Windows, use an up-to-date NVIDIA driver, WSL 2, Docker Desktop’s WSL 2 backend, and Linux containers. This workflow does not require a separate full CUDA Toolkit installation on the host.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Recommended Free Tools
