xboot/libonnx: a pure C99 ONNX inference engine you compile into your firmware

What libonnx solves, and who it is actually for

Most ONNX runtimes assume you have an operating system, a package manager and a few megabytes of shared libraries to spare. Libonnx takes the opposite position. It is a pure C99 inference engine whose .c and .h files are meant to be dropped into an existing project and compiled alongside it. There is no runtime to install on the device, no dynamic loader, no Python. The build output is a static library.

That constraint defines the audience. If you are writing firmware for a board where the toolchain is fixed and the deployment step is a firmware image, this is the shape you want. If you are writing a desktop service that loads models at runtime and swaps them without a rebuild, this is the wrong shape entirely.

The README also states that the library supports hardware acceleration, but only through a hook: you pass an array of struct resolver_t * when allocating the context. The acceleration itself is not shipped. You supply it.

The four-call lifecycle: context, tensor search, run, free

The whole public surface described in the README is four calls. You allocate a context from a model file, look up input and output tensors by name, write into the input, run, and read out of the output.

The context is allocated with onnx_context_alloc_from_file, which takes the model filename, an array of resolvers (NULL when you have none) and a count. The README's example passes NULL and 0:

struct onnx_context_t * ctx = onnx_context_alloc_from_file(filename, NULL, 0);

Tensors are not returned in a fixed order. You ask for them by name with onnx_tensor_search, which means the tensor names in your model file become string constants in your C code. Rename a tensor in the exporter and your firmware stops finding it:

struct onnx_tensor_t * input = onnx_tensor_search(ctx, "input-tensor-name");
struct onnx_tensor_t * output = onnx_tensor_search(ctx, "output-tensor-name");

Inference is a single call with no arguments, and the README states the result is written into the output tensor you already hold:

onnx_run(ctx);

Teardown is onnx_context_free(ctx). That is the entire data flow: no session object, no explicit input binding step, no callback. For a single-model, single-threaded firmware loop this is about as little ceremony as an inference API can have.

Building libonnx and running the hello example

The README gives one build step: type make at the repository root. The top-level Makefile confirms what that does, since it recurses into examples/hello, examples/benchmark, examples/mnist and tests. You get a static library plus binaries for each.

cd libonnx
make

Examples write their output into an output directory next to their source, so the hello example is run from examples/hello/output rather than from the example folder itself:

cd libonnx/examples/hello/output
./hello

One example has extra dependencies. The README states that the mnist example needs SDL2 and SDL2 GFX, and gives the Ubuntu command:

apt-get install libsdl2-dev libsdl2-gfx-dev

Cross compilation is handled by a single variable rather than a separate build system. The README's arm64 example is make CROSS_COMPILE=path/to/toolchains/aarch64-linux-gnu-, and you repoint CROSS_COMPILE at whichever toolchain you use. If you are targeting bare metal, that variable is the seam where your own flags have to go, and the README does not describe how to add them.

Loading a model as a byte array with xxd

Embedded firmware often cannot read a file at all, so the README offers a second loading path. On Linux you convert the .onnx file into a C array:

xxd -i <filename.onnx>

The resulting unsigned char array is then handed to onnx_context_alloc instead of onnx_context_alloc_from_file. The README states this is how the models are loaded in the hello and mnist examples, so those two examples double as working references for the pattern.

This is the detail that makes the library plausible on a microcontroller. The model is just data in your image, and the loader never touches a filesystem. The cost is that the model is fixed at build time. Swapping models means reflashing, and a large model inflates your binary image by its full size.

Operator coverage is the real boundary

The README is unusually direct here. It says that running tests on folders other than tests/model may not succeed, and that some operators have not been implemented. That sentence is the most important one in the document, because it tells you the test suite is not a general conformance claim.

The listed passing tests cover mnist_8, mobilenet_v2_7, shufflenet_v1_9, squeezenet_v11_7, super_resolution_10 and tinyyolo_v2_8, each across three test_data_set folders. Those are vision models. Nothing in the README demonstrates a transformer, an audio model, or a model with control flow.

Coverage is tracked in documents/the-supported-operator-table.md. The README states the library is based on ONNX v1.17.0 with opset 24 support, but the operator table, not that version number, determines whether your model loads. Check your exporter's operator list against that table before you write a line of integration code. If your model uses an operator that is absent, there is no fallback path described in the README.

How libonnx differs from ONNX Runtime

ONNX Runtime is the obvious alternative, and the difference is not speed. It is deployment model. ONNX Runtime is a runtime: you install it, it ships execution providers, it handles model loading, graph optimisation and provider selection at run time. That buys you broad operator coverage and provider-specific kernels you did not write.

Libonnx gives up all of that. You get a static library, a fixed operator set, and a resolver hook where hardware acceleration is something you implement rather than something you enable. In exchange, the dependency graph collapses to your own toolchain. There is no shared object to cross-compile, no provider library to match to your board, and no runtime version to keep in sync with the model file.

A second alternative worth naming is the ONNX project's own tooling, which the README links to for creating models and for pre-trained models. That is not an inference engine, but it is where the model side of this workflow comes from, and libonnx's tools folder is described as help with ONNX model files rather than as a converter.

The honest framing: choose ONNX Runtime when coverage matters more than footprint, and choose libonnx when the footprint is the constraint you cannot move.

Maintenance, licence and what an upgrade costs you

The repository is not archived, and the last push was on 2026-07-07. That is roughly two and a half months before today, so the project is current, though there is no release history to point at: the repository shows no tagged releases. That matters for the upgrade story. Without releases, you track the main branch, and the README does not document rollback, deprecation policy or API stability guarantees. The four-call API is small enough that churn is survivable, but you should treat the commit you vendor as a fork of your own.

Upgrading has a specific cost that is easy to underestimate. Because the model is compiled into the firmware as a byte array, a new model and a new library version ship together. There is no field update of the model alone. Budget for a full firmware build and flash cycle every time either side changes.

The licence is MIT, stated in the README and present as a top-level LICENSE file. MIT is permissive, so redistribution inside a proprietary firmware image is the normal use case rather than an exception. The one practical obligation is preserving the copyright notice and permission text in your distribution. That is a summary of what the repository states, not legal advice; read the LICENSE file yourself.

Editorial conclusion

Adopt libonnx if your target runs C99, your model stays inside the operator table in documents/the-supported-operator-table.md, and you want inference linked into the binary rather than deployed as a runtime. Do not adopt it if you need full ONNX operator coverage, a stable packaged release, or a build you can upgrade without recompiling your firmware. Before writing any integration code, run the test suite against your own model directory and read the operator table, because the README states that tests outside the model folder may not pass and that some operators are not implemented.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. xboot/libonnx on GitHub