libonnx: A C99 ONNX Runtime for Bare-Metal Targets

What libonnx Is For, and Who Ends Up Using It

The README describes a lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support. That sentence sets the audience: people who have an ONNX model and a device that will not run a general purpose runtime. Typical cases are firmware images where the inference code has to compile with the same toolchain as the rest of the application, and where adding a large runtime dependency is not an option. The repository topics include baremetal and embedded-systems, which matches the design. The library's .c and .h files can be dropped into a project and compiled along with it, so there is no package manager step and no build system to integrate beyond your own. The trade-off is implicit in that same sentence: a small C99 engine cannot implement every operator in the ONNX specification, and the README states plainly that some operators have not been implemented. If your model stays inside the supported set, the integration cost is low. If it does not, you are writing kernels.

The Context, Tensor and Resolver Model

The API is small and file oriented. You allocate a context from a model file with onnx_context_alloc_from_file(filename, NULL, 0), where the first argument is the path to the ONNX model and the second is an array of struct resolver_t pointers used for hardware acceleration. Passing NULL means no acceleration. Tensors are looked up by name with onnx_tensor_search(ctx, "input-tensor-name") and onnx_tensor_search(ctx, "output-tensor-name"), so the names must match what is in the model. You write into the input tensor, call onnx_run(ctx), and read the result from the output tensor. onnx_context_free(ctx) releases the context. There is a second loading path for firmware: the README notes that xxd -i <filename.onnx> converts a model into an unsigned char array, and onnx_context_alloc takes that array. This is how the hello and mnist examples load their models, and it is the pattern that makes sense when there is no filesystem. The resolver array is the only extension point mentioned for acceleration, and the README does not document what a resolver_t must implement, so treating it as a pluggable backend interface means reading the headers rather than the README.

Building It: make, CROSS_COMPILE and the SDL2 Examples

The build is a single make at the repository root. The README says you will see a static library and some binaries for the examples and tests. Cross compilation is handled by overriding one variable: make CROSS_COMPILE=path/to/toolchains/aarch64-linux-gnu- builds all libraries, tests and examples for that target. The README's example is arm64, and the instruction is to change CROSS_COMPILE to point at whichever toolchain you use. That is the whole build story, which is a point in its favour for embedded work and a limitation in the same breath, because there is no configuration step for selecting kernels or disabling operators. One example has extra dependencies: compiling the mnist example requires SDL2 and SDL2 GFX, installed on Ubuntu with apt-get install libsdl2-dev libsdl2-gfx-dev. Those libraries are for the GUI, not the engine, so a headless firmware build does not need them. Examples are run from their output directory, for instance cd libonnx/examples/hello/output followed by ./hello.

What the Test Output Actually Tells You

Tests are run from the tests output directory with ./tests ../model, and the README shows the resulting output. The passing entries cover mnist_8, mobilenet_v2_7, shufflenet_v1_9, squeezenet_v11_7, super_resolution_10 and tinyyolo_v2_8, each with three test data sets except super_resolution_10, which shows one. That list is the most useful evidence in the README, because it names the model families the author has exercised. It is also narrow. There is no transformer, no detection model beyond tiny YOLO v2, and no segmentation network in the shown output. The README adds a caution immediately after: running the test on the other folders may not succeed, and some operators have not been implemented. So the passing set is a floor, not a coverage guarantee. The reference for coverage is documents/the-supported-operator-table.md, and the README states the library is based on ONNX v1.17.0 with what it calls the newest opset 24 support. Note the inconsistency in the README itself: the text says v1.17.0 while the link points at the v1.19.0 tree. Check the operator table, not the prose.

Where libonnx Is the Wrong Tool

Operator coverage is the failure mode that matters. The README does not claim full ONNX support, and it does not list which operators are missing in the body text; it points at a separate table. A model that uses an unimplemented operator will not run, and the README gives no fallback path such as a custom operator registration API. The acceleration story has the same shape. The resolver array exists, but the README does not describe the interface, so hardware acceleration is a promise backed by headers rather than documentation. There are also no releases retrieved for this repository, which means there is no tagged version to pin in a dependency manifest. Adopters are tracking the main branch. For a firmware product with a long support window, that is a real cost: you either vendor a snapshot and own it, or you follow a moving branch. Finally, the test output in the README is the author's own run on the author's machine. It is not a benchmark, and the README contains no latency, memory or throughput numbers for any target. Anyone needing to size a model against a device's RAM has to measure that themselves.

How It Differs From ONNX Runtime and TFLite Micro

The closest comparison is ONNX Runtime, which is the reference implementation for running ONNX models and ships a much larger operator set and execution provider model. The difference in approach is structural, not incremental. ONNX Runtime is a substantial codebase with a build system and platform abstractions; libonnx is a set of .c and .h files you compile into your project. If your target can host ONNX Runtime, the operator coverage argument usually wins and libonnx has little to offer. If it cannot, the comparison shifts. TFLite Micro is the other common reference point for microcontroller inference, and the difference there is the model format: TFLite Micro consumes .tflite files and libonnx consumes ONNX. If your training pipeline already exports ONNX, libonnx avoids a conversion step and the class of bugs that comes with it. If your pipeline is TensorFlow based, the conversion runs the other way. The choice is mostly decided by what your toolchain emits and what your device can carry, not by feature checklists.

Licence, Maintenance and Upgrade Cost

The licence is MIT, stated in the README and in the LICENSE file. MIT is permissive: it allows use, modification and redistribution, including in proprietary firmware, provided the copyright notice and permission notice are retained. That is a summary of what the README states, not legal advice; if the inference code ships inside a product, have counsel review the notice placement. On maintenance, the repository is not archived and the last push is dated 2026-07-07, so it is active. The absence of releases is the practical upgrade problem. Without tags, upgrading means diffing the main branch against whatever snapshot you vendored, and the README gives no changelog or migration notes. The README also points at a tools folder for help with ONNX model files and a Chinese discussion thread on whycan.com, which is where questions about missing operators are likely to be answered. Budget for reading the operator table on every upgrade, because that table is the contract between your model and this library.

Editorial conclusion

libonnx suits firmware teams already building with a cross toolchain who need ONNX inference compiled into the same binary, and who can confirm every operator in their model appears in documents/the-supported-operator-table.md. It is the wrong choice if your model relies on operators outside that table, if you need a stable tagged release to pin, or if you want a runtime with a formal support and security process. Before committing, build the tests target for your cross target, run ./tests ../model against a model in the same family as your own, and read the operator table line by line against your graph. If an operator is missing, you are writing that kernel yourself in C99, and that cost belongs in the estimate, not in a later surprise.

Official sources

1. Issues
2. License: MIT
3. README
4. xboot/libonnx on GitHub