# llamacpp-rocm ships nightly llama.cpp binaries with ROCm 7 bundled in

> llamacpp-rocm is a build pipeline that produces nightly llama.cpp packages for AMD GPUs on Windows and Ubuntu with the ROCm 7 runtime included, aimed at teams running local inference on RDNA and CDNA hardware.

**lemonade-sdk/llamacpp-rocm** — Fresh builds of llama.cpp with AMD ROCm™ 7 acceleration

- Repository: https://github.com/lemonade-sdk/llamacpp-rocm
- Stars: 788 · Forks: 81
- Language: Python
- License: MIT
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/lemonade-sdk-llamacpp-rocm

## A build pipeline, not a fork with new features

llamacpp-rocm is described as fresh builds of llama.cpp with AMD ROCm 7 acceleration, based on TheRock. Read that literally and it sets expectations correctly. There is no new inference engine here, no custom sampler and no modified model format. The project takes the upstream llama.cpp code, compiles it against ROCm 7 for a list of AMD GPU architectures, and publishes the resulting binaries as nightly releases.

The stated motivation is deployment rather than research. The README says the automated pipeline specifically targets integration with Lemonade and similar AI applications requiring high-performance GPU inference. In other words, the output is meant to be dropped into an application that expects a llama.cpp server binary, and the value of the project is the freshness of the build and the fact that the ROCm runtime travels inside the archive.

The project is also candid about its scope. A contribution and support notice says that while the project currently focuses on integrating llama.cpp with ROCm in a specific production context, the broader goal is to contribute to the wider llama.cpp and ROCm ecosystem, and that it is not set up to provide comprehensive technical support, though collaborations and contributions are welcome. Read that as a signal about what kind of dependency this is: a fast moving artifact source, not a product with a support contract.

## Seven GPU targets across RDNA, RDNA2, RDNA3, RDNA4, CDNA and the new APUs

The device matrix is the substance of the project. The builds target gfx1151, described as the STX Halo APU found in the Ryzen AI MAX+ Pro 395; gfx1150, the STX Point APU in the Ryzen AI 300; gfx120X for RDNA4 GPUs including the Radeon RX 9070 XT, 9070 GRE, 9070, RX 9060 XT and 9060; gfx110X for RDNA3, covering discrete cards such as the PRO W7900, W7800, W7700, W7600, RX 7900 XTX, XT and GRE, RX 7800 XT, RX 7700 XT and 7700, RX 7600 XT and 7600, plus integrated Radeon 780M, 760M and 740M; gfx103X for RDNA2 parts including the RX 6800 XT, 6700 XT, 6600 XT and 6500 XT; gfx90a for the Instinct MI210; and gfx908 for the Instinct MI100.

Both operating systems are covered. The workflow builds for Windows and Ubuntu, and the release matrix lists a download badge for each GPU target on each platform, so a consumer picks a target rather than a single universal binary.

The practical reading of that list is that the AMD install base is fragmented, and each architecture needs its own compiled kernels. A project that only needs one card could build upstream llama.cpp itself, but a project that has to support a mix of a Ryzen AI 300 laptop, an Instinct accelerator and an RDNA3 desktop card gets a maintained artefact for each combination instead of a matrix of local build instructions. That is the practical value here, and it is a packaging value rather than a performance one.

## ROCm 7 travels inside the archive, which removes the usual install step

The line the README emphasises most is that all builds include ROCm 7 built-in, so no separate ROCm installation is required, and that the complete runtime libraries are included. The release note repeats it as ready to run: download and go.

This is the single biggest difference from building llama.cpp yourself against ROCm, where the usual path is installing the ROCm stack for the right architecture first, then pointing the build at it. Removing that step changes what the artefact is. It becomes self contained enough to run on a machine that has a GPU driver but no ROCm installation, which is exactly the situation on many Windows desktops and on Linux workstations where installing a distribution specific ROCm package is a fight with package versions.

The README also documents the downstream consequence. Under a Linux note for gfx1150, it says that if you see an out of memory error despite free VRAM, you can add ttm.pages_limit=12582912 (48 GB) to the kernel command line, for example in GRUB, run update-grub and reboot, and it points at the TheRock FAQ for more. That is a kernel level change to the graphics memory manager, which tells you the memory allocation path on those APUs does not behave like a discrete card, and that a packaged runtime cannot fix a kernel side limit.

Because the runtime is bundled, the archive is also large and the version of ROCm inside it is fixed at build time. A machine with a newer or older ROCm elsewhere on the system is unaffected by this bundle, which is convenient, and an application that expects a different ROCm minor version is not served by it.

## The smoketest is one llama-server command

Verification is a three step procedure: download the build for your GPU target from the latest releases page, extract the archive to a directory of your choice, and test with any GGUF model from Hugging Face. The test command itself is the llama.cpp server with a model path and a layer offload switch:

```bash
llama-server -m YOUR_GGUF_MODEL_PATH -ngl 99
```

The README explains the switch: use -ngl 99 to offload all layers to the GPU for maximum acceleration, and the exact number of layers may vary by model, but 99 ensures all available layers are offloaded.

That is a genuinely small smoke test, and it is the right one for this kind of package, because it exercises the parts most likely to be wrong in a fresh build: whether the ROCm kernels load, whether the GPU is detected, and whether a model can be served at all. It does not test throughput, and the README makes no performance claims to check.

What the smoke test cannot tell you is which backend the binary picked. The README does not document the flags for listing available devices or for forcing a particular architecture, so if you want to confirm the process is using the GPU you expect rather than falling back to CPU, you should read the upstream llama.cpp documentation for the device listing options. For a first run, the command above plus watching the server log is the whole procedure.

## Lemonade integration is announced as coming soon, so plan for the raw binary

The Lemonade story is the reason this repository sits under the lemonade-sdk organisation, and it is not finished. The README mentions Lemonade twice. The first mention describes the pipeline as targeting direct integration with Lemonade and similar AI applications. The second, in the quick smoketest section, says you can test these builds directly with Lemonade for a smooth AI application experience, and marks it coming soon.

So the integration that motivates the project is announced and not yet available in the documentation. Practically, that means the immediate consumer of these builds is a user who can point at a llama-server binary themselves. If you are waiting for Lemonade to manage the model, the port selection and the runtime, you should not build a pipeline on it yet.

The repository layout backs this up in a small way. The tree contains .github/, LICENSE, README.md, docs/ and utils/, and the primary language is Python. There is no C++ source tree of its own here, because the inference code comes from upstream llama.cpp; the Python side is the tooling that drives the builds and the release packaging, and utils/ is where helper scripts live. A directory called docs/ exists, but the README does not describe what is in it, so treat any deeper documentation there as unverified.

The result is a project with a clear and narrow scope: fresh compiled binaries, packaged per GPU architecture, with the runtime inside. Everything beyond that is currently a plan.

## Nightly build tags and a MIT licence, with no support commitment attached

The release tags show a daily cadence. The recent versions are b1348 on 2026-10-17, b1349 on 2026-10-18 and b1350 on 2026-10-19, and the repository is not archived with the last push on 2026-10-19. The build numbers are continuous, which is the numbering style upstream llama.cpp uses for its own binaries, and it means you can pin an exact build if a later one misbehaves.

Nightly is both the strength and the risk of this project. You get a current upstream llama.cpp without waiting for a distribution to package ROCm for your card, and you can track upstream fixes closely. In exchange, there is no long term support branch, no backport policy and no compatibility guarantee between one night and the next. A production deployment pinning a nightly build should treat it like a rolling dependency and pin by tag.

Licensing is MIT, with the LICENSE file at the repository root. Since these are compiled binaries rather than a library you link, the practical question is redistribution: shipping the binaries inside a product means the licence terms travel with them, and again the LICENSE file is the text your compliance review should read.

One detail deserves attention before any download. The README badges that link to the latest release point at the aigdat/llamacpp-rocm releases page, while this repository is lemonade-sdk/llamacpp-rocm. Whether that is a fork relationship, a moved project or a stale link is not explained anywhere in the README, so confirm which repository is publishing the binaries you install and record that decision in your own notes.

## Conclusion

llamacpp-rocm suits teams that need llama.cpp running on AMD hardware now, particularly on the Strix Halo and Ryzen AI 300 APUs where a separate ROCm installation is the usual obstacle, because every archive already carries the ROCm 7 runtime libraries. It is the wrong tool for NVIDIA hardware, for a team that needs reproducible pinned binaries rather than nightly ones, and for anyone expecting vendor support, since the README states plainly that the project is not set up to provide comprehensive technical support. Before you rely on a build, confirm that the download badge in the README points to the repository you are actually reading, since the badge links target the aigdat/llamacpp-rocm releases page while this repository is lemonade-sdk/llamacpp-rocm, and check the kernel ttm.pages_limit note if you run on gfx1150.

## FAQ

### Do I need to install ROCm separately to use llamacpp-rocm builds?

No. The README states that all builds include ROCm 7 built-in with the complete runtime libraries included, so no separate ROCm installation is required and the archives are described as ready to run.

### Which AMD GPUs does llamacpp-rocm support?

The build targets gfx1151 (STX Halo APU), gfx1150 (STX Point APU), gfx120X (RDNA4), gfx110X (RDNA3), gfx103X (RDNA2), gfx90a (Instinct MI210) and gfx908 (Instinct MI100). Windows and Ubuntu are both built for each target.

### How do I test a llamacpp-rocm build after downloading it?

Extract the archive and run llama-server -m YOUR_GGUF_MODEL_PATH -ngl 99 with any GGUF model from Hugging Face. The README says to use -ngl 99 so that all available layers are offloaded to the GPU.

### Why do I get an out of memory error on gfx1150 despite free VRAM?

The README gives a kernel level fix for that case: add ttm.pages_limit=12582912 (48 GB) to the kernel command line, for example in GRUB, run update-grub and reboot. It points to the TheRock FAQ for further detail.

### Can I use llamacpp-rocm builds with Lemonade today?

The README says Lemonade integration is coming soon, so the documented way to use these builds today is to run the llama-server binary directly. The project does say the automated pipeline targets integration with Lemonade and similar GPU inference applications.

## Sources

- [Issues](https://github.com/lemonade-sdk/llamacpp-rocm/issues)
- [lemonade-sdk/llamacpp-rocm on GitHub](https://github.com/lemonade-sdk/llamacpp-rocm)
- [License: MIT](https://github.com/lemonade-sdk/llamacpp-rocm/blob/main/LICENSE)
- [README](https://github.com/lemonade-sdk/llamacpp-rocm/blob/main/README.md)
- [Releases](https://github.com/lemonade-sdk/llamacpp-rocm/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/lemonade-sdk-llamacpp-rocm