Apache TVM FFI: One C ABI for Kernel Libraries, DSLs and Frameworks
Open ABI and FFI for Machine Learning Systems
At a glance
- What is it?
- TVM FFI defines a stable, minimal C ABI plus Python, C++ and Rust bindings so a kernel library can ship a single wheel that PyTorch, JAX, CuPy, PaddlePaddle and NumPy can all call. It is an interface convention, not a runtime, and the project is still in an RFC stage where minor version bumps may break the C ABI.
- Who is it for?
- Adopt TVM FFI if you maintain a kernel library or DSL and the matrix of frameworks and Python versions you must support has become the expensive part of your release process, or if you are building a runtime that wants one extension point for third-party kernels. Do not adopt it for a single-framework extension with one Python target: pybind11 plus the framework's own extension API is less machinery for the same result.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly C++, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The wheel matrix problem TVM FFI is aimed at
A kernel library that wants to serve PyTorch, JAX and CuPy today faces a build matrix rather than a coding problem. Each framework has its own extension mechanism, its own tensor type, and its own supported Python versions, and free-threaded Python adds another axis. The README frames the goal plainly: ship one wheel to support multiple frameworks, Python versions, and different languages, and it names FlashInfer as the kernel-library case. The same pitch is extended to kernel DSLs (TileLang, cuteDSL), to frameworks and runtimes that want a uniform extension point (PyTorch, JAX, PaddlePaddle, NumPy and CuPy), and to ML infrastructure that needs bindings across Python, C++, Rust and projects such as XGrammar. The audience is therefore library and compiler authors, not application developers. If you are writing model code in PyTorch, this project is not in your path.
A stable C ABI, DLPack tensors and a compact call convention
The mechanism described in the README has three parts. First, a stable, minimal C ABI that kernels, DSLs and runtimes extend through, which is the layer the project says it cares most about keeping fixed. Second, zero-copy interop across PyTorch, JAX and CuPy using the DLPack protocol, so tensors cross the boundary without a copy and without a framework-specific tensor class. Third, a compact value and call convention covering common data types, which the README presents as targeting ultra low-overhead ML applications. Around those sit multi-language bindings for Python, C++ and Rust. The documentation index fills in the object model that the README only gestures at: separate concept pages for Any, Object and Class, Tensor, Function and Module, and Exception Handling. That set tells you the ABI is not a bare function-pointer table. Values are boxed into a tagged representation, functions and modules are first-class, and errors are carried across the boundary as structured exceptions rather than return codes. The exact wire layout is not in the README, so treat the Concepts pages as the place to confirm it before you commit to a binary interface.
Install and first contact
The README gives a two-line install: pip install apache-tvm-ffi, followed by pip install torch-c-dlpack-ext, which it labels a compatibility package for torch <= 2.9. If you are on a newer PyTorch, that second line is likely unnecessary, but the README does not state a cutoff beyond that version note, so check the quickstart page rather than assuming. uv is listed as an alternative installer, and building from source is covered under the Developer Manual at dev/source_build.html, with CI reproduction at dev/ci_cd.html and the release process at dev/release_process.html. For publishing, the Packaging section covers Python packaging, stub generation (stubgen) and C++ tooling, which is the part that matters if you intend to ship one wheel across frameworks. The two guides that map most directly onto the README's use cases are Export Functions & Classes and the Kernel Library Guide. The README does not print a minimal code sample, so the first runnable example lives in the quickstart, not here.
The RFC stage is the constraint that shapes adoption
The README is unusually direct about status: Status: RFC, with main features complete and ABI stable, but with an explicit acknowledgment that evolution may be needed and an invitation to shape it with the community. The versioning rule during that stage is the part to read twice. Releases are 0.X.Y, where a bump in X indicates C ABI-breaking changes and Y indicates other changes. So v0.1.13 to v0.2.0 is a break, and the release list shows how fast the patch line moves: v0.1.13-post2, v0.1.13-post3 and a v0.1.14-rc4 all within roughly six weeks. The README says the RFC stage is planned to last three months from v0.1.0, after which the project intends to follow Semantic Versioning. The practical consequence is that anything you compile against this ABI, including a wheel you distribute, needs a pinned range and a rebuild plan for the next X bump. If your users install your wheel and their framework upgrades independently, the ABI version is a compatibility surface you now own.
Where TVM FFI is the wrong tool
Three cases stand out. A single-framework, single-Python-version extension gets little from an open ABI: the framework's own extension API already handles the tensor type, and adding a second convention means maintaining bindings on both sides of a boundary that only one consumer crosses. Second, code that must exchange rich, framework-specific objects rather than tensors and common scalar types will find the compact value convention working against it; the README describes coverage of common data types, and the Concepts pages define Any, Object and Class precisely because the type system is deliberately narrow. Third, teams that cannot absorb ABI churn should not start during the RFC window, since the versioning rule explicitly permits breaking C ABI changes in a minor bump. None of this is a flaw in the design. It is the cost of being an interface that many parties implement independently.
TVM FFI versus a pybind11 extension module
The obvious alternative is a hand-written pybind11 (or nanobind) extension that links the framework's C++ library directly. The difference is what each one is coupled to. A pybind11 module is coupled to CPython's C API and to one framework's headers, which is fine until you need a second framework or a free-threaded interpreter, at which point you either build a second module or introduce an abstraction yourself. TVM FFI inverts that: the kernel side targets one C ABI, and the framework side is the project's problem, with DLPack handling tensor interchange so no framework header enters your build. The cost is a dependency and an ABI version to track. The README also lists PyTorch, JAX, PaddlePaddle and NumPy/CuPy as frameworks that expose this as a uniform extension point, which is the claim to verify for your specific framework version before you restructure a build around it. A second alternative, staying on the framework's own custom-op path, keeps you supported by that framework but leaves the multi-framework wheel problem exactly where it started.
Licence, releases and the cost of keeping up
The project is Apache-2.0, and the README carries the standard ASF header with the licence and NOTICE files governing use and redistribution. That is permissive and includes a patent grant, which matters for a component you redistribute inside your own wheel. This is not legal advice; read the LICENSE and NOTICE files in the repository for the terms that bind you. On maintenance, the visible signal is cadence rather than volume: a post-release, another post-release, and a release candidate inside about six weeks, with the last push to main in September 2026. Fast patch cadence during an RFC stage usually means either active bug fixing or a moving interface, and the versioning rule says both are possible. Budget for re-reading the Stable C ABI page and the Concepts pages at each X bump, and for rebuilding any compiled artifact you ship. The documentation is organised into Get Started, Guides, Concepts, Packaging and Developer Manual sections, which is enough structure to find an answer, but the README itself is a pointer document rather than a tutorial. Plan your first integration around the quickstart and the Kernel Library Guide rather than the repository front page.
Editorial conclusion
Adopt TVM FFI if you maintain a kernel library or DSL and the matrix of frameworks and Python versions you must support has become the expensive part of your release process, or if you are building a runtime that wants one extension point for third-party kernels. Do not adopt it for a single-framework extension with one Python target: pybind11 plus the framework's own extension API is less machinery for the same result. Before committing, verify the exact v0.1.x release you pin, whether torch-c-dlpack-ext is needed for your PyTorch version, and whether the object, Any and exception semantics documented under Concepts match the value types your kernels actually exchange.
Community notes