NVIDIA Container Toolkit: what it does and how to install it on Ubuntu and Docker

What the NVIDIA Container Toolkit solves, and for whom

A container image carries an application and its userspace libraries. It does not carry the GPU. On a plain Docker install, a container starts with no device nodes under /dev and no driver libraries from the host, so CUDA code inside it fails at the point it tries to open the device. The NVIDIA Container Toolkit exists to close that gap. The README describes it as allowing users to "build and run GPU-accelerated containers", and says the toolkit includes a container runtime library plus utilities that automatically configure containers to use NVIDIA GPUs.

The intended user is someone running Linux hosts with NVIDIA hardware who wants container isolation without hand-writing device mounts. The README is explicit that the CUDA Toolkit is not required on the host, but the NVIDIA driver is. That distinction matters: the toolkit configures access to a driver that already exists on the machine, it does not install one.

How the toolkit configures a container at runtime

The repository is written in Go and is split into a library and command-line utilities. The go.mod file shows the module path github.com/NVIDIA/nvidia-container-toolkit and a dependency on github.com/NVIDIA/go-nvml, the NVML bindings used to query the driver and enumerate devices, and on tags.cncf.io/container-device-interface, the CNCF specification for describing devices that a runtime should hand to a container. The presence of github.com/opencontainers/runtime-spec and github.com/containerd/nri points at the two integration paths: an OCI runtime hook that edits the container spec before the container starts, and a containerd NRI plugin that does the equivalent work inside containerd.

The flow is roughly this. A runtime such as Docker or containerd prepares a container. The toolkit's hook or plugin inspects the request, asks the driver which GPUs are present, then injects the matching device nodes and the driver libraries and binaries into the container's filesystem view and its OCI spec. The command-line side, nvidia-ctk, is what writes the runtime configuration that triggers this. The README does not walk through the hook internals; that detail lives in the documentation repository it links to.

Installing the NVIDIA Container Toolkit on Ubuntu and Docker

The README does not contain install commands. It states that product documentation, including platform support and installation and usage guides, is in the documentation repository, and it links an installation guide. It also warns twice about prerequisites: install the NVIDIA driver for your Linux distribution, and note that the CUDA Toolkit is not needed on the host while the driver is. So the first step is confirming a driver is present. The README does not name a command for that check, so follow the installation guide it links before going further; if the driver is missing, the toolkit has nothing to attach to.

After the toolkit package is installed for your distribution, the runtime has to be told to use it. The nvidia-ctk command is the utility that writes that configuration, and the README points to the user guide for the configuration and command line options available when running GPU containers with Docker. Once the runtime is configured, a container started with GPU access should see the device.

The repository's Makefile shows how the project itself is built and checked: it defines targets named binaries, build, check, fmt, test, examples, cmds, coverage, generate, licenses, third-party-notices, check-third-party-notices, vendor, check-vendor and lint, and it derives a cmd- target for every directory under ./cmd/ and an example- target for every directory under ./examples/. Those targets build the toolkit from source; they are not part of installing it on a GPU host.

Where the toolkit is the wrong tool

The README scopes the project to Linux distributions and to hosts with an NVIDIA driver. That rules out a native Windows or macOS host install, and the repository does not present a Windows build. Searches for a Windows or Windows 11 version of the toolkit are therefore looking for something the README does not offer; on Windows the GPU path runs through WSL2, where the Linux toolkit applies inside the WSL distribution and the driver comes from the Windows side.

A second boundary is that the toolkit only exposes hardware it can see through the driver. If the host driver is older than the CUDA version your image expects, the container may start and then fail inside the application; the toolkit will happily pass through a driver that cannot run the workload. It also does nothing for scheduling. It makes a GPU visible to a container, and the README makes no claim about deciding which container gets which device. On a shared multi-tenant host, that allocation problem is outside this project.

How it differs from running CUDA directly on the host

The obvious alternative is to skip containers and install the CUDA Toolkit and your application directly on the host. That approach removes an entire layer: no runtime configuration, no hook, no device injection, and no version skew between the container's expected driver and the host's actual one, because there is only one environment. For a single-purpose machine that runs one workload, it is genuinely simpler and there is less to debug.

The trade-off is reproducibility. A host install couples the application's dependencies to the machine's package state, so two hosts drift apart over time and a rebuild is a manual exercise. The container path keeps the application's userspace in an image while the toolkit handles the one thing an image cannot carry, the GPU. That split is the reason the project exists: the driver stays on the host and is managed as host infrastructure, while everything above it is versioned in the image.

Maintenance cadence, licensing and upgrade cost

The repository is not archived, and the most recent push recorded is 2026-09-21, two days before the date used here. The release history shows v1.20.0 on 2026-08-13 and v1.20.1 on 2026-09-19, with a release candidate v1.20.0-rc.1 on 2026-07-09. That pattern, a minor release followed by a patch roughly a month later, is what a reader should expect from the project: fixes ship between feature releases, and release candidates appear before minor versions.

The toolkit is licensed under Apache-2.0, and the LICENSE file sits at the repository root. The repository also carries a THIRD_PARTY_NOTICES.md file and third_party/ directory, and the Makefile defines targets named licenses and third-party-notices, which indicates the project generates and checks those notices as part of its build. Because the toolkit links against the driver through NVML rather than redistributing it, the driver's own licence terms remain a separate matter. That is a description of what is in the repository, not legal advice; if you redistribute a product built on this, read the licence files yourself.

Upgrade cost is mostly host-side. The toolkit is installed as a system package, so moving between versions means updating that package on every GPU host. The container images themselves are unaffected, which keeps the blast radius small, but the driver on the host still has to satisfy whatever CUDA version the images expect. Upgrading the toolkit without checking the driver version is the common way to break a working setup.

Editorial conclusion

Adopt it if you run GPU workloads in containers on Linux and want the runtime to inject devices and driver libraries for you. Do not adopt it expecting a Windows or macOS host install: the README frames it around Linux distributions and the NVIDIA driver. Before rolling it out, confirm the NVIDIA driver is present on the host and read the linked installation guide for your distribution, since the repository itself does not carry the install commands.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. NVIDIA/nvidia-container-toolkit on GitHub
4. README
5. Releases