ZML picks hardware at compile time with build flags, and its only prerequisite is a build system installed under another name
Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild
At a glance
- What is it?
- zml/zml is a production inference stack written in Zig on top of MLIR and Bazel, whose pitch is that one codebase compiles directly to NVIDIA, AMD, Intel, TPU and AWS accelerator targets with no rewriting. Building it requires nothing but the Bazel launcher, the smoke test downloads a pretrained model rather than shipping one, and the repository has exactly one release, which is not the inference stack.
- Who is it for?
- ZML fits a team with one model and more than one accelerator, where the cost being paid today is a port per vendor, and where writing the model once in Zig and compiling per target is worth a compiler toolchain. It does not fit a team that wants to try inference this afternoon, because the build is the project: there is no binary to download, no prebuilt wheel, and a compiler stack that has to compile even the thirty-second example.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Zig, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- zml/zml4,259 ★ · +205
| Repository | 2026-09-16 | 2026-11-15 |
|---|---|---|
| zml/zml | 4.1k | 4.3k |
205 Stars in the last 60 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
One prerequisite, installed under a different name than the tool it runs
The prerequisites section is two sentences long and the second one contains the whole setup. The only requirement is the build system, and the recommendation is to install a version manager for it rather than the build system itself. On macOS that is one package manager command:
brew install bazeliskOn Linux the instructions do not install anything: they download a binary from the launcher release page, write it to a path under the system binary directory with the name of the build tool, and make it executable. So the file called build-tool on your machine is not the build tool, it is the launcher that picks and fetches a pinned version, which is why there is also a file in the repository recording the expected version. That arrangement is invisible if you already have the launcher installed and slightly confusing if you have not, because the installed artifact and the thing it stands in for have different names.
Hardware is chosen by a build flag, and the default includes the CPU
This is the core mechanism, and it is worth understanding as a compile-time decision rather than a runtime one. Every accelerator has a flag in the same package: a flag for NVIDIA CUDA, one for AMD's runtime, one for Intel's oneAPI toolkit, one for Google TPU, and one for AWS Trainium and Inferentia 2. There is also a flag that disables CPU compilation entirely. Each of these takes a boolean value, and the examples show them appended to the same run command, before the separator that passes arguments to the program itself. The consequence is that a forgotten flag does not produce an error; it produces a binary compiled for the default target, which is the CPU, and one that will be correct but slow. The documentation also gives three complete run commands with identical prompts differing only in the platform flags, which is the clearest possible demonstration that the only variable is the target.
The thirty-second test downloads its model
The quick start offers one command as a smoke test, and it runs the MNIST example. What that example does is worth stating precisely, because it explains what the stack is actually doing: it downloads a small pretrained MNIST model, compiles it, loads the weights, and classifies a random handwritten digit. So the first run of the project involves a network fetch, a compilation of a model you did not write, and a single inference. That is a reasonable design for a smoke test and a reasonable one to be aware of, since the download means the test cannot run in an isolated build environment, and the compile means the example is a real end-to-end test of the compiler rather than a replay of prebuilt artefacts. The larger model example behaves the same way but adds an authentication step, since the small models are freely downloadable and the gated ones are not.
The model API is tagged dimensions, a dot product, and a relu
The code sample in the readme is the best documentation here, and it is short enough to read in one pass. A model is a struct of layers, and a layer holds two tensors, a weight and a bias. Initialisation takes a view into a tensor store and creates the two tensors, each tagged with a named dimension: the weight carries an output dimension and a feature dimension, and the bias carries only the output dimension. Forward is one expression: a dot product of the weight with the input contracting over the feature dimension, plus the bias, then a rectified activation, then a call that reasserts the tags on the result. One thing follows from that: the store is the abstraction, which is what lets the same code read weights from a local directory, from the model hub, or from an object store.
A Zig build file and a Bazel module manifest sit side by side
The repository root contains both a Zig build script and Bazel's module manifest, plus a lock file for that manifest, a per-package build file at the root, a version pin, a repository configuration file, an ignore file and a shell wrapper. That is an unusual combination to see in one tree and it is worth asking which one is authoritative before you assume anything about the build. What the readme describes is the Bazel route: examples are addressed as build targets, tests are a build target, and accelerator selection is a build flag. The directory layout is organised around compilation units that match those targets: an MLIR directory, a dsl directory, a platforms directory, a virtual filesystem layer, an extension library, a foreign function interface, a runtime plug-in directory, a protobuf runtime in C, third-party sources and tools. There is also a bin directory and the examples directory holds five examples rather than one.
Editor configurations and a Nix environment are committed
Several entries in the root exist for the people doing the work rather than for the build. There is a configuration file for the Neovim editor and settings directories for two other editors, all committed, which in 2026 is a deliberate choice by a project whose contributors clearly live in those editors. There is a Nix development environment in three files: a declarative definition, a lock file pinning what it resolves to, and a YAML configuration file. There is also an assistant instructions file (AGENTS.md) and a contribution guide, and the documentation is organised into tutorials, conceptual notes and how-to guides rather than a single reference.
One release exists, and it is not the inference stack
The release list has a single entry, and it is not the package most people arrive for. It is named for a separate component, versioned with that component's name and a prefix, and it dates from April 2026, more than five months before the last push. The inference stack itself, the thing the readme spends its length on, has no release. That tells you how this project is being used: the published artefact is a narrow piece of tooling, while the compiler and runtime are consumed from source. It also tells you what to expect when you adopt it. There is no version to pin, no upgrade path to read about, and no changelog to consult, because the commit history is the only record of what changed. The last push to the default branch is 2026-11-02, so the work is current even though the releases are not.
Editorial conclusion
ZML fits a team with one model and more than one accelerator, where the cost being paid today is a port per vendor, and where writing the model once in Zig and compiling per target is worth a compiler toolchain. It does not fit a team that wants to try inference this afternoon, because the build is the project: there is no binary to download, no prebuilt wheel, and a compiler stack that has to compile even the thirty-second example. Two things to check before you commit. Which accelerator you actually have, because the target is chosen by a build flag and the default includes the CPU, so a mistake compiles a slow binary rather than failing. And where you get the model, because the smoke test downloads a small pretrained network and the gated large models need a Hugging Face login or a token in the environment, which is a network dependency your build will not have if it runs in isolation.
Frequently asked questions
What is ZML?
It is a production inference stack written in Zig, built on the MLIR compiler infrastructure and Bazel, licensed Apache-2.0. Its purpose is to decouple AI workloads from proprietary hardware: one codebase compiles directly to NVIDIA, AMD, Intel, Google TPU and AWS Trainium targets without rewriting the model, and hardware selection happens through a build flag rather than at runtime.
How do I install and build ZML?
The only prerequisite is Bazel, and the recommendation is to install its version manager rather than Bazel itself. On macOS that is a single brew command. On Linux the instructions download the launcher binary, write it under the system binary directory using the name bazel, and make it executable. Then one command runs the MNIST example, which downloads a small pretrained model, compiles it and classifies a random digit. Tests run through the same build system with a test target.
Which hardware does ZML support?
NVIDIA CUDA, AMD's runtime, Intel oneAPI, Google TPU and AWS Trainium and Inferentia 2, each selected by a boolean flag in the project's platform package, plus a flag that disables CPU compilation. The examples in the readme differ only in their platform flags, and since the CPU is compiled by default, forgetting the flag yields a correct but slow binary rather than an error.
Which models does the ZML example support?
The language model example supports Llama 3.1 and 3.2, Qwen 3.5 and LFM 2.5. Gated repositories such as the Meta Llama models require authenticating with Hugging Face through a tool in the repository, or setting the token in the environment. Weights can also be loaded from a local directory or from an object store path, and omitting the prompt argument opens an interactive chat loop.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/zml-zml)Star history chart for your README
[](https://hysenlabs.com/projects/zml-zml)Was this useful?