ZINC: a Zig inference engine aimed at AMD Radeon and Apple Silicon GPUs

The gap ZINC is trying to fill: AMD cards without a Python stack

Most local inference tooling grew up around CUDA, and the AMD path in those projects is often a port rather than the primary target. ZINC inverts that. The README describes it as fast, local GGUF inference for the GPUs people already own, and the supported backend list leads with AMD Radeon across Vulkan and ROCm/HIP. Intel Arc is Vulkan only. Apple Silicon is Metal. NVIDIA RTX is marked experimental CUDA, which is a deliberate ordering: the project is not pretending to compete on the hardware where the incumbent tooling is strongest. The audience is someone with a Radeon card or a Mac who wants to run a quantised GGUF model locally and expose it over an HTTP endpoint, without installing PyTorch or a Python environment. The packaging claim is a single Zig binary that contains the command line, browser chat, model manager and API server. That is a narrower and more concrete promise than general purpose inference, and it is the promise the rest of the repository is organised around.

One binary, four surfaces, and a managed model catalog

The architecture visible in the README is a single executable with several front ends over the same inference core. The CLI takes a prompt directly or starts a chat session. The chat subcommand additionally starts the browser chat and the OpenAI-compatible API. Health checks live at /health, and model listing plus chat completions sit under /v1, which is the shape most OpenAI client libraries expect. Model resolution has three paths: a managed catalog addressed by an identifier such as qwen35-9b-q4k-m, a direct path to a local .gguf file, and a Hugging Face repository reference in the form Qwen/Qwen3-0.6B-GGUF:Q8_0. The catalog identifiers are project-defined names, not upstream ones, so a model pulled by identifier is tied to how ZINC names and quantises it. Current tuning work is stated to cover Qwen 3.5, Qwen 3.6, Qwen 3.8, Gemma 4 and Muse Glimmer. That list is short and specific, and it implies that other architectures may load but are not tuned. Backends have native kernels and are measured separately, so a Vulkan result on one card does not transfer to the ROCm build on the same card.

Building ZINC from source: Zig 0.15.2, glslc, and ROCM_PATH

There are no retrieved releases for this repository, so the documented path is a source build. The README requires Zig 0.15.2 or newer. Linux Vulkan builds also need glslc and a Vulkan loader, and ROCm builds need a working ROCm installation. The default build is:

git clone https://github.com/zolotukhin/zinc.git cd zinc zig build -Doptimize=ReleaseFast

Then two checks and a first run:

./zig-out/bin/zinc --check ./zig-out/bin/zinc model pull qwen35-9b-q4k-m ./zig-out/bin/zinc --model-id qwen35-9b-q4k-m --prompt "Hello" --chat

The ROCm backend is a separate build with an environment variable and a device selector:

ROCM_PATH=/opt/rocm zig build -Dbackend=rocm -Doptimize=ReleaseFast ROCR_VISIBLE_DEVICES=0 ./zig-out/bin/zinc --check

Tests run through zig build test. Two details matter for anyone planning to adopt this. The first is that --check exists as a preflight step and is the cheapest way to find out whether your driver and backend combination is actually usable. The second is that the ROCm path is opt-in at build time rather than detected, so your binary is tied to the backend you compiled. Pointing at an existing file or a Hugging Face repo bypasses the catalog entirely:

./zig-out/bin/zinc -m /path/to/model.gguf --prompt "The capital of France is" ./zig-out/bin/zinc -hf Qwen/Qwen3-0.6B-GGUF:Q8_0 --prompt "Hello" --chat

What the benchmark claim covers, and what it does not

The headline result in the README is that ZINC beats the comparison llama.cpp build on prefill, decode and combined time for all six models in the current Radeon AI PRO R9700 ROCm core suite. The stated controls are meaningful: same GPU, same GGUF files, same prompts, reusable servers, warmups and measured run counts. The README also states the scope explicitly, calling it a scoped, reproducible result and not a claim about every model or GPU. Raw samples, exact prompts, build revisions and the checked-in JSON are published on the benchmark page. Two things are worth separating here. First, this is the project's own measurement of its own code against a build it selected, so it is a starting point for your own testing rather than a settled comparison. Second, the claim is anchored to one card and one backend. The repository notes that backends are measured separately, so nothing in the R9700 ROCm result tells you how the Vulkan path behaves on an Intel Arc, or how Metal behaves on an M-series chip. The README also says that where a model or GPU path is incomplete, the benchmark page leaves that result visible instead of dropping it, which is a policy worth checking directly on the page rather than taking on trust.

Limitations and the cases where ZINC is the wrong tool

The supported list is the first constraint. AMD Radeon on Vulkan and ROCm/HIP, Intel Arc on Vulkan, Apple Silicon on Metal, and experimental CUDA on NVIDIA RTX. If your card is not on the validated hardware page, you are outside the tested set. The CUDA path is labelled experimental, so anyone on an RTX card who wants a dependable daily driver has no reason to prefer ZINC over the mature CUDA tooling. The tuning list is the second constraint: Qwen 3.5, 3.6, 3.8, Gemma 4 and Muse Glimmer. A model outside that set may run, but the project does not claim it is tuned. The build requirements are the third. Zig 0.15.2 or newer is a recent compiler, and pinning to a specific Zig version is a real cost for anyone maintaining a toolchain across projects. Vulkan builds on Linux need glslc and a Vulkan loader, ROCm builds need a working ROCm install plus ROCM_PATH set at compile time. There are no retrieved releases, so installation is a source build and a compiler toolchain, not a package. Finally, the README's own framing is honest about state: ZINC is described as active engineering work. That is a reasonable description of a project whose last push is 2026-09-09 and which has no release artefacts to point at.

How it differs from llama.cpp

The obvious alternative is llama.cpp, and the README names it directly as the comparison build in the benchmark suite. The difference is not the model format: both consume GGUF. It is where the engineering effort is concentrated and what gets shipped. llama.cpp is a broad C/C++ codebase with a long list of backends and a wide set of front ends contributed over time, and it is usually the default answer when someone asks what to run locally. ZINC is a Zig codebase that picks a narrower hardware target and builds native kernels for it, with a single binary that bundles the CLI, browser chat, model manager and OpenAI-compatible server rather than leaving the serving layer to a separate project. The trade-off follows from that. ZINC offers a smaller surface and a managed catalog with project-defined model identifiers, which is convenient if the models you want are on the tuning list and awkward if they are not. llama.cpp offers breadth and a much larger set of contributors, which matters when your hardware or model architecture is unusual. If you are on a Radeon card and the ZINC benchmark page covers your exact card and backend, the comparison is worth reproducing yourself with tools/performance_suite.mjs. If you are anywhere else, llama.cpp is the lower-risk choice on the evidence available here.

Maintenance cost, licence and what to verify before adopting

ZINC is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is the permissive end of the spectrum and it is the same licence family as llama.cpp, so licence is unlikely to be the deciding factor between the two. This is not legal advice; check the LICENSE file in the repository for the exact terms. The maintenance cost that does matter is the toolchain. A source build against Zig 0.15.2 or newer, with a backend selected at compile time via -Dbackend=rocm, means upgrades are rebuilds rather than package updates. There are no retrieved releases, so there is no version number to pin and no changelog to read before upgrading. The model catalog adds a second moving part: catalog identifiers are project-defined, so a catalog entry can change independently of the binary. If you depend on a specific quantisation, reference the .gguf file by path or by Hugging Face repository and revision instead of by catalog name. The benchmark tooling is checked in at tools/performance_suite.mjs with published artefacts at site/src/data/zinc-performance.json, so the measurement method is inspectable rather than described in prose. Verify three things before you commit: that --check passes on your card with your intended backend, that the model you need is either in the tuning list or loads correctly from a direct path, and that the OpenAI-compatible endpoints under /v1 behave the way your client expects. The first two are answered by commands in this article. The third needs the API guide and a request against your own client.

Editorial conclusion

ZINC is worth building if you have an AMD Radeon card on the validated list, or an Apple Silicon machine, and you want one binary that serves an OpenAI-compatible endpoint without a Python stack. Skip it if your GPU is not on that list, if you need a mature CUDA path, or if you need a packaged release rather than a source build against Zig 0.15.2. Before committing, run ./zig-out/bin/zinc --check with your intended backend and pull one model from the managed catalog to confirm your card and driver combination is actually covered.

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. zolotukhin/zinc on GitHub