# ROCm-docker: building the ROCm software stack into portable container images

> ROCm-docker is a build framework for turning the layers of AMD's Radeon Open Compute Platform into Docker images, with a setup script that generates Dockerfiles from templates because Dockerfiles themselves have no preprocessor. The catch is structural: the ROCm kernel driver has to live on the host, so containers alone are not enough.

**ROCm/ROCm-docker** — Dockerfiles for the various software layers defined in the ROCm software platform

- Repository: https://github.com/ROCm/ROCm-docker
- Stars: 521 · Forks: 80
- Language: Shell
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/rocm-rocm-docker

## The host kernel requirement, which no container can get around

ROCm-docker describes itself as a framework for building the software layers defined in the Radeon Open Compute Platform into portable docker images, and the most important thing it has to say is a constraint rather than a feature.

Docker does not virtualize or package the linux kernel inside of an image or container. This is a design decision of docker to provide lightweight and fast containerization. The implication for this on the ROCm compute stack is that in order for the docker framework to function, the ROCm kernel and corresponding modules must be installed on the host machine. Containers share the host kernel, so the ROCm KFD component ROCK-Kernel-Driver functions outside of docker.

That is the whole shape of the project in one paragraph. Everything ROCm ships above the kernel, the compiler, the math libraries, the communication layer, can be packaged. The kernel driver cannot, because the container shares the host's kernel and the kernel is what talks to the GPU. So the workflow is always two-step: prepare the host, then run the images.

The README links an apt-get repository intended to automate the installation of the required kernel and kernel modules, which is the piece that has to happen first.

The host prerequisites are Docker on Ubuntu systems or Fedora systems, and Docker-Compose is described as highly recommended to simplify container management.

So the target reader is a developer or a CI system that wants the ROCm toolchain without polluting the host with a specific ROCm release, and who accepts that the host still has to carry the kernel side. It is not a packaging scheme for a laptop with no AMD GPU, and it is not a way to ship ROCm to machines you do not control.

## Docker Hub gives you a first container in one command pair

For an easy start with ROCm and Docker, the README points at the rocm/rocm-terminal image hosted on Docker Hub. The instruction is that after the ROCm kernel is installed, you pull the image from Docker Hub and create a new instance of a container.

```bash
sudo docker pull rocm/rocm-terminal
sudo docker run -it --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined --group-add video rocm/rocm-terminal
```

Those flags are the entire mechanism, and each one is load bearing. The --device=/dev/kfd exposes the Kernel Fusion Driver character device, which is how the container talks to the GPU through the host kernel. The --device=/dev/dri exposes the Direct Rendering Interface, which is how rendering and compute buffers get allocated. The --group-add video puts the container user into the video group so the device permissions line up. And --security-opt seccomp=unconfined relaxes the default seccomp profile, which is commonly needed for GPU workloads under Docker.

If any of those is missing, the container starts and then fails when it tries to use the device, which is a confusing failure mode for someone who has never run a GPU container before. The device paths are host paths, so this is not portable to a machine with a different device layout.

The README also points at a set up guide at quick-start.md along with asciicasts, for people who want the full walkthrough rather than the quick pull. The same quick-start.md is referenced for the storage driver fix described further down, so it carries more than one kind of instruction.

## The FAQ section is the practical part of the README

The FAQ is not the usual list of questions about the project. It is a list of commands and failure modes, and it is the most immediately useful part of the page for anyone actually working with these containers.

The first entry is the one that catches everyone. A new docker container typically does not house apt repository meta-data. Before trying to install new software using apt, make sure to run sudo apt update first.

That is a direct consequence of how the images are layered. Each image installs a ROCm layer on top of a base, and the package index is not carried forward from the base into the child, so apt inside a fresh container has no idea what packages exist until it refreshes its lists.

The second entry covers the most common permission failure. A message like Cannot connect to the Docker daemon. Is the docker daemon running on this host? typically means your user does not have permissions to execute docker, and the fix is to use sudo or add your user to the docker group.

Then a set of file transfer and inspection commands that are worth having in one place.

```bash
sudo docker exec -it <CONTAINER-NAME> bash -l
sudo docker cp HOST_PATH <CONTAINER-NAME>:/PATH
sudo docker cp <CONTAINER-NAME>:/PATH/TO/FILE HOST_PATH
```

The first opens another terminal into a running container. The other two copy files in both directions, which is how you get a dataset or a build tree into the container and results back out.

The storage driver note is the one that catches people pulling large images. If you receive messages about no space left on device when pulling images, the cause is that the image size limits imposed by the device mapper storage driver are a problem, and the fix is documented in the quick start guide.

And there is a section on saving work. Docker containers are typically ephemeral, and are discarded after closing the container with the --rm flag to docker run. Sometimes that is wrong, and you want to serialize the work back into an image, either as a checkpoint or to share through a registry.

```bash
sudo docker ps -a
sudo docker commit <container-name> <new-image-name>
sudo docker images
```

docker commit is the mechanism, docker ps -a finds the container to commit and docker images confirms the result.

## setup.sh writes Dockerfiles, because Dockerfiles cannot do that themselves

The build story is where this repository differs from an ordinary collection of Dockerfiles, and the reason is stated plainly.

Unfortunately, Dockerfiles do not have a preprocessor or template language, so typically build instructions are hardcoded. However, the setup script allows us to write a primitive 'template', and after running it instantiates baked dockerfiles with environment variables substituted in.

So the repository holds textual template files ending with the *.template* suffix, and setup.sh substitutes environment variables into them to produce final Dockerfiles. The README's own example is the one every project hits: you want release images and debug images, so you run the setup script to generate the release Dockerfiles and build them, then run the setup script again specifying debug Dockerfiles and build new images. The docker images should generate unique image names and not conflict with each other.

That is a small template engine written in shell, and it exists to solve a real problem. Without it, a matrix of base images times build types times ROCm versions means a Dockerfiles directory nobody can maintain.

The setup.sh parameters table shows the generic parameters, and the defaults matter because they tell you what you get if you run the script with no arguments.

| setup.sh parameters | parameter [default]| description |
|-----|-----|-----|
| --ubuntu | xx.yy [16.04] | Ubuntu version for to inherit base image |
| --install-docker-compose | | helper to install the docker-compose tool |

If no parameters are given, built images will be based off of Ubuntu 16.04 with rocm components installed from debians downloaded from repo.radeon.com. The supported parameters can be queried with ./setup --help.

The source-build parameters are separate, and they are the ones you use for a debug image.

| setup.sh parameters | parameter [default]| description |
|-----|-----|-----|
| --tag | string ['master'] | string representing a git branch name |
| --branch | string ['master'] | alias for tag |
| --debug | | build code with debug flags |

One limitation is worth flagging. The setup.sh scripts checks to make sure that it is running on an Ubuntu system, as it makes a few assumptions about the availability of tools and file locations. If running rocm on a Fedora machine, inspect the source of setup.sh and issue the appropriate commands manually. So the Fedora support is real for the resulting images but the build automation is Ubuntu-only in practice.

The other two installation routes are described at the top of the build section: install from the rocm apt/rpm repository (repo.radeon.com), or build the components from source and run install scripts. The README is direct about the trade: the repository route produces images with the smallest footprint and best building speed, because no developer tools need to be installed and downloading binaries is much faster than downloading source and invoking a build process. Building from source allows much greater flexibility on install location and the ability to step through the source with debug builds.

## The compose file encodes the layer stack as a dependency graph

The docker-compose.yml in the repository is where the layering becomes concrete, and it is worth reading as a description of the ROCm software stack rather than as a configuration file.

There are two parallel families of services. The first is built from debians downloaded from a package repository: roct-deb, then rocr-deb which depends_on roct-deb, then hcc-lc-deb which depends_on rocr-deb. The second family is built from rocm source: roct-src, rocr-src with depends_on roct-src, and hcc-lc-src with depends_on rocr-src.

The dependency chain is the ROCm stack itself. The runtime and compiler sit at the bottom, the communication layer above them, and the high performance computing layer on top, and each image is built from the previous one. The compose file then adds the application containers on top of the data-only containers, with a comment that these should be run with docker-compose run --rm <application-service>.

```yaml
  rocm:
    build:
      context: ./rocm-terminal
      dockerfile: Dockerfile
    devices:
      - "/dev/kfd"
    image: rocm/rocm-terminal
    volumes:
      - ~:/usr/local/src/host-home
```

That block shows two things the earlier services do not. The application container is the one that gets the GPU device, and it mounts the host home directory at /usr/local/src/host-home, so a ROCm application running inside the container sees your source files and build artefacts without a second copy step. The image name rocm/rocm-terminal matches the Docker Hub image, so a locally built image and the published one occupy the same tag.

The top-level repository layout matches this structure. There are framework/ and dev/ directories, a rocm-terminal/ directory for the application container, scripts/ for shared build helpers, and add-rocm.sh, build_all.sh, push_all.sh and setup.sh as the entry points. build_all.sh and push_all.sh are the scripts for the whole matrix, and setup.sh is the one that generates the Dockerfiles in the first place.

## Where ROCm-docker is the wrong tool

The first wrong-tool case is a machine you cannot modify. The whole approach depends on installing the ROCm kernel and the KFD module on the host, which means root on the machine and a kernel matched to the ROCm release. If you are on a managed GPU instance where the driver is already installed by the provider and cannot be replaced, or on a kernel you are not allowed to change, this repository does not help. The container part is the easy part; the kernel part is the blocker.

The second case is a machine with no AMD GPU at all. Nothing here is useful for building CPU-only software. The image names and the device flags make the target obvious, but it is worth saying for the reader who wandered in from a general containerisation question.

The third case is a build environment that is not Ubuntu. setup.sh checks that it is running on an Ubuntu system, and the README's fallback for Fedora is to inspect the source and issue the appropriate commands manually. That is a real gap in the automation, and it means a Fedora-based CI image needs a wrapper script the project does not supply.

There is also a staleness question the README does not answer well. The defaults described in the README are Ubuntu 16.04 as the base image, which is a long way back, and the setup script's default tag and branch are both master. The README links to the current ROCm wiki for installing from AMD's own repositories, which suggests the kernel side has moved on from what this repository documents. The last push to the repository was on 2026-10-14 and the repository has no GitHub releases, so there is no tag history to compare the documented defaults against the current state of the platform. Anyone using this should check the quick-start.md rather than trusting the defaults in the README table.

The fourth wrong-tool case is a team that wants to publish its own images. The framework builds and can push, with a push_all.sh at the top level, but the naming is tied to the generated Dockerfiles rather than to anything the project specifies, and the compose file expects the images to line up with the layer stack. Sharing the result means taking ownership of the image set and its versioning.

On licence, the repository is MIT, which is the permissive grant and imposes no obligation on a team embedding the images in a workflow. The separate question is the licensing of ROCm itself and of the underlying AMD components, which are distributed under their own terms and are not covered by this repository's licence.

## Conclusion

ROCm-docker suits a developer who wants the ROCm compiler and libraries in a reproducible container rather than installed into the system Python, since the layer images are built from a template pipeline and the compose file wires the dependency order for you. It does not remove the host requirement, because the ROCm kernel and the KFD module must be installed on the machine and the container shares the host kernel. Before building anything, install the ROCm kernel from the apt-get repository on the host, and note that setup.sh assumes Ubuntu, so on Fedora the README tells you to inspect setup.sh and issue the commands yourself.

## FAQ

### What is ROCm-docker?

It is a framework for building the software layers defined in the Radeon Open Compute Platform into portable Docker images. It contains Dockerfiles, a setup.sh that generates them from template files, and a docker-compose.yml that wires the layer stack together.

### Do I need to install ROCm on the host to use the ROCm-docker containers?

Yes, the kernel side only. Docker does not package the Linux kernel, so the ROCm kernel and corresponding modules must be installed on the host machine, and the containers share the host kernel so the ROCm KFD component functions outside of Docker.

### How do I run a ROCm container with GPU access?

Pull rocm/rocm-terminal from Docker Hub and run it with --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined --group-add video. The device flags expose the KFD and DRI character devices to the container.

### Why does apt fail inside a fresh ROCm container?

A new docker container typically does not house apt repository meta-data, so you need to run sudo apt update first before installing new software with apt.

### How do I keep the work done in a ROCm container?

Find the container with sudo docker ps -a, then use sudo docker commit <container-name> <new-image-name> to serialize it into an image and sudo docker images to confirm it exists. Containers run with the --rm flag are discarded when closed.

### What license is ROCm-docker released under?

The repository is released under the MIT license, with the LICENSE file at the root. The primary language is Shell and the default branch is master, and the repository has no GitHub releases.

## Sources

- [Issues](https://github.com/ROCm/ROCm-docker/issues)
- [License: MIT](https://github.com/ROCm/ROCm-docker/blob/master/LICENSE)
- [README](https://github.com/ROCm/ROCm-docker/blob/master/README.md)
- [ROCm/ROCm-docker on GitHub](https://github.com/ROCm/ROCm-docker)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/rocm-rocm-docker