# ONNX-MLIR: compiling ONNX graphs to objects, shared libraries and jars through MLIR

> ONNX-MLIR takes a valid ONNX graph and lowers it to code with minimum runtime support, using LLVM and MLIR as the substrate. What it produces is selectable with a flag, from an object file to a shared library to a jar, and the project tracks the ONNX operation coverage per target on a public model zoo dashboard. The cost is a pinned LLVM commit that has to be moved by hand.

**onnx/onnx-mlir** — Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure

- Repository: https://github.com/onnx/onnx-mlir
- Stars: 1,061 · Forks: 454
- Language: MLIR
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/onnx-onnx-mlir

## What the project contributes, and what a dialect buys you

ONNX-MLIR provides compiler technology to transform a valid Open Neural Network Exchange graph into code that implements the graph with minimum runtime support. It implements the ONNX standard and is based on the underlying LLVM and MLIR compiler technology.

The contribution list in the README has four items, and they describe the whole stack.

An ONNX Dialect that can be integrated in other projects. That is the reusable core. A dialect in the MLIR sense is a typed intermediate representation for one domain, and making it a first-class citizen rather than an internal detail is what allows the rest of the ecosystem to reuse ONNX-MLIR's work.

A compiler that lowers ONNX graphs into MLIR files, LLVM bytecodes, and C and Java libraries. This is the actual compilation path, and the fact that the outputs include both a C library and a Java library is what makes it usable from either language.

An onnx-mlir driver to perform these lowering. The driver is the command line front end, and the option list in the README shows it is also a modular optimizer driver with frontend options and optimisation levels.

And a python, C, C++ and Java runtime environment. The runtime is what the compiled artefact calls into, and keeping it minimal is the point of the whole exercise, since the aim is code that implements the graph with minimum runtime support.

The primary language in the repository metadata is MLIR, the licence is Apache-2.0, and the default branch is main. The recent releases are v0.5.1.1 and v0.5.0.3 both on 2026-08-27 and v0.5.0.2 on 2026-08-11, so the patch releases are frequent.

The repository layout backs the four contributions up. There is include/ and src/ for the dialect and the compiler, test/ for the test suite, third_party/ for vendored dependencies, utils/ with the clone-mlir.sh script, docker/ for the container images, and a set of CMake files at the top level that are worth reading separately.

## The emit options, and the optimisation level default of zero

The usage output in the README is the most concrete description of what the tool does, and it is a well designed command line.

```
OVERVIEW: ONNX-MLIR modular optimizer driver

USAGE: onnx-mlir [options] <input file>
```

The emit options are the interesting part, because they are the shape of the output rather than a detail of the compilation.

--EmitONNXBasic ingests ONNX and emits the basic ONNX operations without inferred shapes. --EmitONNXIR ingests ONNX and emits the corresponding ONNX dialect, which is the dialect form of the same thing. --EmitMLIR lowers the input to the MLIR built-in transformation dialect, so you can inspect the intermediate representation. --EmitLLVMIR lowers it further to LLVM IR. --EmitObj compiles the input to an object file. --EmitLib compiles and links the input into a shared library, and that is the default. --EmitJNI compiles the input to a jar file.

So the pipeline is inspectable at every stage. A user can emit ONNX IR to check the frontend, then MLIR to see the lowering, then LLVM IR, and only then produce the artefact. For a compiler that is going to be pointed at a model that does not work, being able to stop at the stage where it breaks is worth a lot.

The optimisation levels are --O0 through --O3, and the default is O0. The README makes a point of this in the same way a compiler manual would: just as with most compilers, the default optimisation level is -O0. For an ahead of time compiler, starting at zero is defensible, since you usually want to know the lowering is right before you start tuning the code generation, and it also keeps the first build fast. But it does mean that if you take the default and ship the shared library, you are shipping unoptimised code, and the README does not draw attention to that beyond the one line.

The generic options are the LLVM standard set, with --help and --help-list in their familiar forms, and the README notes that the - and -- prefix for flags can be used interchangeably.

## The pinned LLVM commit, and why the Docker path is the recommended one

The build story in this project is unusual, and understanding it is the difference between a working build and a wasted afternoon.

The prerequisite list is:

```
python >= 3.11
clang >= 18.1.3
protobuf >= 33.5
cmake >= 3.26.0
make >= 4.3 or ninja >= 1.10.2
java >= 21 (optional)
```

The comment above the list says to keep it in sync with docs/Prerequisite.md, which is the kind of note that means it has drifted before.

The real complication is the LLVM dependency. The README states that at any point in time, ONNX-MLIR depends on a specific commit of the LLVM project that has been shown to work with the project, and that the maintainers periodically need to move to a more recent LLVM level. Among other things, this requires updating the LLVM commit string in clone-mlir.sh, and when updating ONNX-MLIR it is good practice to check that the commit string of the MLIR/LLVM is the same as the one listed in that file.

So the dependency is a commit hash, not a release. That is unavoidable for a project built on LLVM and MLIR, where the dialect APIs move quickly, but it has a real cost: the version of LLVM your build uses is determined by a string in a shell script in this repository, and a mismatch between your checkout and that string produces a build failure that will not obviously point at the cause.

That is precisely why the project recommends containers. The instructions say the preferred approach to using and developing ONNX-MLIR is to use Docker images and containers, as getting the proper code dependencies may be tricky on some systems, with the instructions in docs/Docker.md. And if you intend to develop code, the workflow document sets up your Docker environment in a way that lets you contribute code easily.

A native build is supported, on Linux, macOS and Windows, with separate instructions per OS in docs/BuildOnLinuxOSX.md and docs/BuildOnWindows.md. After installation, an onnx-mlir executable should appear in the build/Debug/bin or build/Release/bin directory. And if you have difficulties building, rebuilding or testing, there is a page for helpful hints.

The practical advice is to take the Docker path unless your environment already matches the prerequisites exactly. The container image is the only configuration where the LLVM commit is guaranteed to be the right one, and for a project whose whole value is a specific lowering pipeline, a reproducible toolchain is worth more than the flexibility of a native build.

## What minimum runtime support actually means for a consumer

The whole premise of the project is code that implements the graph with minimum runtime support, and that phrase is worth unpacking because it is where the difference from a normal ONNX runtime sits.

A conventional ONNX runtime interprets the graph. The model is data, and the runtime walks it, dispatching each operator to an implementation. The graph stays in memory, the operator implementations stay in the runtime binary, and the whole thing is a general purpose interpreter that must support every operator anyone might use, whether or not your model uses them.

A compiled approach is different. The graph is turned into code at build time, and what ships is that code plus a small set of primitives the generated code calls into. A model that uses eight operators does not drag in the implementations of the other hundred and fifty. The runtime shrinks to the subset the model actually exercises.

That is the case for the technology, and it is a real one in the deployment situations this project targets. A compiled model on an s390x machine running an inference service has a smaller memory footprint, a smaller binary, and none of the dispatch overhead of an interpreter walking a graph. The cost is that you need a compilation step in your build, that the compilation is a heavy dependency rather than a library call, and that changing the model means rebuilding.

The ONNX-MLIR contribution list also has an entry that is easy to miss because it sounds like a formality. An ONNX Dialect that can be integrated in other projects is not just about reuse of the compiler, it is about the model becoming a first class citizen in an MLIR pipeline. Once ONNX has a dialect, other passes written for MLIR can operate on it, and a project with its own MLIR compiler infrastructure can drop ONNX into it without going through ONNX-MLIR's specific driver.

The runtime variants follow the same logic. There is a C runtime, a C++ runtime, a Python driver and a Java runtime, and the Python one is deliberately thin, a pybind11 module over the compiled artefact with numpy as its only dependency. That is what you would expect from a project whose stated goal is minimum runtime support, and it is the opposite of what a general purpose framework looks like.

## The runtime, the Python driver, and the CI matrix

The runtime side of the project is where the design goal of minimum runtime support becomes concrete.

The top level CMake files are named for the runtime targets: OMCRuntime.cmake, OMPyRt.cmake and MLIR.cmake. The pyproject.toml describes the Python runtime package, named om_pyrt, with the description Python driver to run inference on onnx model compiled with onnx-mlir, authored by Tong Chen and Sunny Anand at IBM, and it requires Python 3.9 or newer. It is built with scikit-build-core and pybind11, and the scikit-build configuration sets a CMake argument of ONNX_MLIR_TARGET_TO_BUILD=OMPyRt, with the wheel packages and install components both set to the om_pyrt target.

That is a normal Python extension module built with CMake, and the numpy dependency is the only runtime requirement, so the Python side of the runtime is a thin layer over the compiled artefact rather than a framework.

The requirements.txt is the test-side dependency set, and it is a different list: lit for the LLVM integrated test framework, numpy at a pinned 2.2.6, protobuf, onnx at 1.23.0, pytest and pytest-xdist. The presence of lit is the tell that the test suite is written as LLVM style file tests rather than only as Python unit tests, and pytest-xdist means they run in parallel.

The CI matrix is broad and the badges show it. There is Jenkins for the s390x-Linux pipeline and for the amd64-Linux pipeline, each with both a build status badge and a separate model zoo status badge, and Azure Pipelines for amd64-Windows, and GitHub Actions for the macOS amd64 build. The s390x target is the notable one, because s390x is where IBM's Telum AI accelerator lives, and the project maintains a separate supported operations document for it, docs/SupportedONNXOps-NNPA.md, alongside the generic CPU one, docs/SupportedONNXOps-cpu.md.

The model zoo badges are the part most projects do not have. They report the status of a set of models run through the pipeline per configuration, so coverage is measured against real models rather than asserted. That is a genuinely useful signal for anyone deciding whether ONNX-MLIR supports their model, and it is why the supported operations documents matter less than the dashboard.

On maintenance, the repository is not archived, the last push was on 2026-10-15, and the recent releases are from August 2026. The project also carries a CII Best Practices badge, a GOVERNANCE.md, a SECURITY.md, a CODE_OF_CONDUCT.md, a CODING_PRACTICE.md and a CHANGELOG.md, and it holds informal weekly meetings on Tuesdays with agenda and notes in the wiki.

## Conclusion

ONNX-MLIR fits a team that already has ONNX models and wants them compiled ahead of time into a native artefact with a small runtime, especially on the s390x and amd64 targets the project tests, where it also tracks a model zoo of supported operations. It is the wrong tool if you need a single ONNX operator implemented today, because coverage is per target and per operation, and the wrong tool if you want a zero build dependency footprint, because the build needs a specific LLVM commit cloned in. Before adopting, read the supported operations document for your target, check which LLVM commit the project currently pins in clone-mlir.sh, and plan for the Docker path rather than a native build unless your environment already matches the prerequisites.

## FAQ

### What does ONNX-MLIR do with an ONNX model?

It compiles the ONNX graph into code that implements the graph with minimum runtime support, using LLVM and MLIR. The project contributes an ONNX dialect for reuse elsewhere, a compiler that lowers graphs into MLIR files, LLVM bytecodes and C and Java libraries, an onnx-mlir driver, and a python, C, C++ and Java runtime.

### How do I choose what ONNX-MLIR outputs?

With the emit options. --EmitONNXBasic and --EmitONNXIR stop at the ONNX level, --EmitMLIR gives the MLIR built-in transformation dialect, --EmitLLVMIR gives LLVM IR, --EmitObj gives an object file, --EmitLib gives a shared library and is the default, and --EmitJNI gives a jar file. Optimisation is selected with --O0 through --O3, defaulting to O0.

### What does ONNX-MLIR need to build?

Python 3.11 or newer, clang 18.1.3 or newer, protobuf 33.5 or newer, cmake 3.26.0 or newer, and make 4.3 or ninja 1.10.2 or newer, with Java 21 optional. It also depends on a specific pinned LLVM commit recorded as a string in utils/clone-mlir.sh, which is why the project recommends the Docker images.

### How do I know whether ONNX-MLIR supports my model?

The project lists current levels of support for ONNX operation code generation in a document for a generic CPU and a separate one for the Telum accelerator, and the CI publishes a model zoo status per target configuration, which reports coverage against real models rather than only asserting it.

## Sources

- [Issues](https://github.com/onnx/onnx-mlir/issues)
- [License: Apache-2.0](https://github.com/onnx/onnx-mlir/blob/main/LICENSE)
- [onnx/onnx-mlir on GitHub](https://github.com/onnx/onnx-mlir)
- [README](https://github.com/onnx/onnx-mlir/blob/main/README.md)
- [Releases](https://github.com/onnx/onnx-mlir/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/onnx-onnx-mlir