Library / SDK
pykeio/ort avatar
pykeio/ort

ort's default build downloads binaries, and its docs enable 28 features

Fast ML inference & training for ONNX models in Rust

2,617 stars279 forksRustApache-2.0

At a glance

What is it?
ort is a Rust interface for running and training ONNX models, primarily a wrapper for ONNX Runtime with pure-Rust alternatives. Its manifest is more informative than its README: the default feature set fetches binaries and pins an API level, the documentation build enables 28 accelerator features that a default install does not have, and the workspace excludes the very backends it advertises.
Who is it for?
Use ort when you need ONNX inference in Rust across more than one kind of hardware, and read the feature list before you build rather than after. Two decisions drive everything.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 2 days ago.
What is it written in?
Mainly Rust, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 13, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • pykeio/ort2,617 ★ · +111
Star history
Repository2026-09-112026-11-13
pykeio/ort2.5k2.6k

111 Stars in the last 63 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

A wrapper first, with pure Rust alternatives behind it

The mechanism is stated in three sentences near the top of the README.

`ort` is a Rust interface for hardware-accelerated inference and training on models in the ONNX format. It is based on a crate that is now inactive, and it is primarily a wrapper for Microsoft's ONNX Runtime, with support offered for other pure-Rust runtimes as well.

The lineage matters, because the author list in the manifest names both the project and the person behind the crate it grew out of. So this is a maintained fork of an earlier binding rather than a new design, and the description in the manifest is specific about what it wraps: ONNX Runtime 1.30.

The deployment claim is the one that decides whether you want this at all. When you need to run a model from PyTorch, TensorFlow, Keras, scikit-learn or PaddlePaddle, on device or in a datacentre, ort is offered as the way to do it. And the README says it is light enough to run on a user's device, which is a different constraint from a datacentre library and explains why alternative backends exist.

The backends live in a `backends/` directory at the root, which is where the pure-Rust runtimes are kept.

The default feature set downloads binaries

The default feature list is the line in the manifest most worth reading before anything else.

It is seven entries: `std`, `ndarray`, `tracing`, `download-binaries`, `tls-native`, `copy-dylibs` and `api-30`. Two of those decide how a build behaves.

`download-binaries` means the default build fetches prebuilt runtime binaries rather than compiling one from source. That is the fast path, and it is also a build step that needs network access and that decides which runtime you get from a remote location.

`api-30` pins the runtime API version the bindings target, which matches the ONNX Runtime 1.30 in the package description. Anyone upgrading their runtime and expecting the bindings to follow has to know that the API level is a compile-time choice here, not something detected at load time.

The other entries are ordinary. `copy-dylibs` handles shared library placement, `tls-native` picks the TLS backend, `ndarray` is the tensor type bridge, and `tracing` is instrumentation.

What is not in the default set is as informative as what is. `load-dynamic` is not enabled, so the default build binds a runtime at build time rather than loading one at run time, and no accelerator feature is enabled at all.

Twenty-eight feature flags, and they are all hardware

The documentation metadata in the manifest lists the features the generated API reference is built with, and reading that list tells you what the project considers its real surface.

There are 28 of them. Two are data handling, `half` for half precision and `num-complex` for complex numbers. One is `fetch-models`. One is `training`, which is the feature that turns the crate from inference-only into what its title claims. One is `load-dynamic`, and one is `copy-dylibs`.

The remaining 22 are accelerator names: CUDA, TensorRT, OpenVINO, oneDNN, DirectML, NNAPI, CoreML, XNNPACK, ROCm, ArmNN, TVM, MIGraphX, RKNPU, VITIS, CANN, QNN, WebGPU, Azure, NVRTX and VSINPU, which covers NVIDIA, AMD, Intel, Apple, ARM, Qualcomm, RISC-V and the browser.

The README's claim that it supports almost any hardware accelerator you can think of is not marketing, it is this list. Each entry is a real integration with a real vendor runtime, and each is opt-in.

The two targets for the documentation build are an x86_64 Linux target and WebAssembly, with a custom rustdoc flag set for the documentation build. That is why WebAssembly appears in a list dominated by accelerators: it is the one non-hardware backend that ships in the same list.

The documentation is built with more than you will have

There is a divergence between the crate you install and the documentation you read, and it is deliberate.

The documentation build enables the full feature list above. The default feature list enables seven entries, none of which are accelerators, and does not include dynamic loading or training.

So the API reference shows types, methods and behaviour that come from code you will not have compiled unless you turn the flag on. Reading the reference is therefore a map of the maximum rather than of your build, and the practical consequence is that you will meet methods in the documentation that fail to compile until you enable the feature that produces them.

This is the standard trade for a project with a large optional surface: one reference for everyone, at the cost of a mismatch between what is documented and what is compiled. The migration guide from v1.x to v2.0 exists partly to smooth that, and it is linked from the README alongside the guide, the API reference and the examples.

Two other links in the same block are worth noting: the API reference URL is pinned to the current release candidate rather than to latest, so the documentation you land on matches the newest tag.

The workspace excludes the backends it advertises

The workspace configuration is where this project stops making sense, and it is worth reading as an arrangement rather than a bug.

There is one member, the FFI bridge crate, and the default member is the root package. Everything else is on an exclude list of seventeen entries: three backends by name, fourteen example directories, and a leak check test.

Excluded means outside the workspace, not broken. So the backends, named for the candle, tract and web pure-Rust runtimes, are separate crates that a consumer pulls in explicitly, and they are not built when you build this workspace. The same applies to most examples. Each example is its own crate with its own dependencies, which is why a vision example that pulls a model does not slow down the default build.

Two entries in the exclude list are worth noticing for a different reason. One example directory is excluded but does not appear in the repository's example listing, and one example directory that does appear is not excluded, which means it is part of the workspace build.

The practical takeaway is that the repository root builds one thing, the bindings, and everything interesting about the alternatives lives in directories you have to enter separately.

Two licence files, and the package ships only the source directory

The licensing is dual and unambiguous in the files, and less tidy in the metadata.

The manifest declares the licence as MIT or Apache-2.0, and both licence texts sit at the repository root as separate files. The license recorded for the repository is Apache-2.0 alone, so anyone reading only the repository metadata sees the narrower of the two options.

The package include list is short and unusual: the source directory and the two licence files. Nothing else ships. Tests, examples, documentation, the backends and the tools in the repository tree are all excluded from the published artifact, which is consistent with the workspace arrangement where none of them are members either.

The metadata block also carries a maintenance status field, which is the value crates.io renders as a badge rather than anything the repository can enforce.

One more thing in the manifest that tells you about the project: the authors list names the commercial entity behind it alongside the individual who wrote the crate it replaces. A dual licence and a commercial steward on a bindings library is a normal combination, and it is the reason the adopter list in the README is filtered.

Version 2.0 is still a release candidate

The three most recent tags are all release candidates: v2.0.0-rc.13 on 2026-07-28, v2.0.0-rc.12 on 2026-03-05 and v2.0.0-rc.11 on 2026-01-07.

Two observations. The manifest version matches the newest tag exactly, so the crate you get from a registry and the source in the repository agree. And the candidate series has been running for at least a year, with roughly four to five months between the first three of them, which is a normal pace for a large release with a documented migration.

The README treats the migration as the main event for existing users: a guide from v1.x to v2.0 is one of four links in the documentation block, next to the guide itself, the API reference and the examples. For anyone on 1.x, that link is the reason to stay or move.

Meanwhile the last push to the default branch was on 2026-10-02, more than two months after that tag, so the branch is ahead of every published candidate.

The adopter list is filtered by licence, not by size

The README lists free software projects that use ort, and it is the longest part of the file. It also has a policy stated in a comment above it.

The comment says the section only showcases projects with OSI-approved licences, and that businesses which sponsor the project appear at the top of the README instead. That is an unusual and specific rule, and it means the list is curated by licence rather than by size.

What is on it covers a wide range: HuggingFace's text embeddings inference, Google's Magika for file type detection, a desktop and WebAssembly tool for PaddleOCR models, accelerated transformer inference at the edge, a Style-BERT-VITS2 speech synthesiser, a biodiversity deployment tool, a trail camera detector that finds animals and people, Elixir bindings, a Rust OCR library, local embedding generation with reranking, a semantic search proxy, a desktop image gallery with OCR and semantic search, two local speech-to-text projects built on an NVIDIA model, a runtime for language, speech and text-to-speech models across mobile and game engines, a Rust inference library and command line tool for YOLO models, and a drawing app that runs a hand pose model.

Support is a Discord channel and GitHub Discussions, funding goes through OpenCollective, and the root also carries a security policy, a contributing guide and an agent instruction file.

Editorial conclusion

Use ort when you need ONNX inference in Rust across more than one kind of hardware, and read the feature list before you build rather than after. Two decisions drive everything. Whether you accept a default feature set that downloads prebuilt runtime binaries at build time, since that changes what your build does on a machine without network access. And whether you are on 2.0, which is still a release candidate across its three most recent tags, so treat the migration guide from v1.x as required reading rather than optional. If you need a stable pin today, the older line exists and the dual MIT or Apache-2.0 licence makes either choice safe for commercial use.

Frequently asked questions

What is pykeio ort?

A Rust interface for hardware-accelerated inference and training on machine learning models in the ONNX format. It is primarily a wrapper for Microsoft's ONNX Runtime, which the manifest names as version 1.30, and it also supports other pure-Rust runtimes.

Which hardware backends does ort support?

The manifest lists feature flags for CUDA, TensorRT, OpenVINO, oneDNN, DirectML, NNAPI, CoreML, XNNPACK, ROCm, ArmNN, TVM, MIGraphX, RKNPU, VITIS, CANN, QNN, WebGPU, Azure, NVRTX and VSINPU. None of them is in the default feature set, so each has to be enabled explicitly.

Is ort 2.0 a stable release yet?

No. The three most recent tags are release candidates, v2.0.0-rc.13 from 2026-07-28, rc.12 from 2026-03-05 and rc.11 from 2026-01-07, and the README links a migration guide for moving from v1.x to v2.0.

What Rust version and licence does ort need?

Rust 1.92 or newer on edition 2024. The crate is published under MIT or Apache-2.0 with both licence files at the repository root, and the package itself ships only the source directory plus those two files.

Which free software projects use ort?

The README lists text embeddings inference from HuggingFace, Google's Magika for file type detection, a trail camera detector, Elixir bindings, a semantic search proxy, two local speech-to-text projects built on an NVIDIA model, a runtime for language and speech models on mobile and in games, and a Rust library and command line tool for YOLO models. The list is restricted to OSI-approved licences.

Official sources

  1. License: Apache-2.0
  2. Project website
  3. pykeio/ort on GitHub
  4. README
  5. Releases
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/pykeio-ort.svg)](https://hysenlabs.com/projects/pykeio-ort)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/pykeio-ort.svg)](https://hysenlabs.com/projects/pykeio-ort)

Was this useful?