# huggingface/kernels: loading compiled compute kernels from the Hub

> The Kernel Hub lets a Python application fetch a compiled kernel at runtime from the Hugging Face Hub instead of linking against a locally built extension module. The trade is a set of hard constraints on kernel authors, and a hardware support matrix with three tiers, the lowest of which is experimental and not yet published.

**huggingface/kernels** — Build compute kernels and load them from the Hub.

- Repository: https://github.com/huggingface/kernels
- Website: https://huggingface.co/docs/kernels/
- Stars: 775 · Forks: 137
- Language: Python
- License: Apache-2.0
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/huggingface-kernels

## What the Kernel Hub actually changes for a library author

The Kernel Hub is a way for Python libraries and applications to load compute kernels directly from the Hugging Face Hub. Instead of shipping a wheel that contains a compiled extension built against one specific PyTorch and CUDA combination, a library asks for a kernel by repository name and version at runtime, and the Hub serves the build that fits the machine.

The problem this solves is concrete. A PyTorch extension has to be compiled against a particular torch version, a particular CUDA version and a particular C++ ABI. A library that wants to run well on several hardware backends either publishes a wheel per combination, which is a long build matrix, or it publishes one build and it breaks somewhere.

The README sets out three properties that a Hub kernel has to have, and they are the design constraints of the whole project. A kernel is portable, meaning it can be loaded from paths outside PYTHONPATH. It is unique, meaning multiple versions of the same kernel can be loaded in the same Python process. And it is compatible, meaning it must support all recent versions of Python and the different PyTorch build configurations, including various CUDA versions and C++ ABIs, while older C library versions must still be supported.

That last constraint is the demanding one, and it is why the project is not simply a file downloader. The compatibility requirement is what turns the Hub into a distribution channel with rules attached, and those rules are what the kernel-builder and the ABI check exist to enforce.

## Three components: the loader, the builder and the community kernels

The repository splits into three pieces and they serve different audiences.

The kernels package is the Python library that loads kernels from the Hub, and it is the only part most users install. The kernel-builder is for kernel authors, and it is a Rust command line tool, hf-kernel-builder, that builds the kernels. The third piece is the kernels-community organisation on the Hub, where Hugging Face maintains a set of kernels that other people can use as-is.

The Rust side of the repository is a Cargo workspace with five members: kernel-builder, kernels, kernels-common, kernel-abi-check and kernel-port. The names describe the division of labour. kernels-common holds shared code, kernel-abi-check is the tool that verifies the ABI compatibility requirement, and kernel-port is the part that speaks to the platform-specific bits. The workspace also pins a dependency on hf-hub, the Rust client for the Hub, at a specific git revision, and the Makefile refers to the CLI as hf-kernel-builder in its cargo build command even though the project documentation calls it kernel-builder.

The Python side lives under kernels/ with its own dependency lock file, and the Makefile keeps a file in kernels/src/kernels in sync with a copy in kernels-common/src by copying it during the build, so the two packages cannot drift apart on their Python dependency list.

## Installing the package and running your first Hub kernel

Installation is a single pip command, and the README states the requirements next to it: the kernels package needs torch>=2.5 and CUDA.

```bash
pip install kernels
```

The usage example in the README downloads the activation kernels from the kernels-community organisation and calls one of them:

```python
import torch
from kernels import get_kernel

activation = get_kernel("kernels-community/activation", version=1)

x = torch.randn((10, 10), dtype=torch.float16, device="cuda")

y = torch.empty_like(x)
activation.gelu_fast(y, x)
```

Three details in that snippet carry the design. The kernel is identified by a Hub repository and a version, so the caller decides which build to fetch rather than inheriting whatever the installed wheel happened to contain. The function writes into an output tensor the caller allocates, so the call signature follows the usual PyTorch convention of explicit buffers. And the dtype and device are chosen by the caller, which is what allows one binary to serve several configurations.

The Hub itself is the catalogue, at huggingface.co/kernels, and the README links it for browsing what is available.

For authors, the builder is a Rust binary that the Makefile builds and then uses to generate the command line documentation:

```bash
cargo build -p hf-kernel-builder
```

The generated reference lands in docs/source/builder-cli.md, and the Makefile target that produces it rewrites the binary name from hf-kernel-builder to kernel-builder in the output, which is why the two names appear interchangeably in the project.

## The hardware tiers and the gaps in them

The README publishes a table with three tiers, and the tier is a promise about how well the combination is tested rather than a marketing label.

CUDA and CPU are tier 1. Kernels are supported, the kernel-builder supports them, and the kernels are validated in CI. Those are the backends where a broken build would be caught before release.

ROCm, XPU, Metal and TPU are tier 2. Kernels and the builder work for them, but they are not validated in CI. The projects also carry a warning that Neuron and TPU support are experimental and that the packages are not public yet, so a TPU user is relying on a path that is not finished.

Huawei NPU is tier 3, where kernels are supported but the kernel-builder is not, which means there is no supported way to build a new kernel for that backend. Neuron is also tier 3 and is marked as neither supported by the builder nor validated, with the README's warning about experimental status on top.

The practical reading is that the system is only as trustworthy as its weakest tier for the backend you care about. On CUDA and CPU, a kernel published through this system has been through continuous integration. On everything else, the guarantee is much weaker, and for Huawei NPU you need a kernel from elsewhere.

## Version bumps, lock files and the release script

The project takes version management seriously enough to have a Makefile target for each case. There are targets to bump every version site to the next patch dev release, to the next minor dev release, to strip the dev suffix before a patch release, and dry-run variants of each.

The scripts do the work, calling python scripts/bump_version.py with a dev flag, a major flag or a dry-run flag, then running cargo check --workspace and refreshing the lock file with uv lock inside the kernels directory. The comment above the target explains the version format, giving examples such as 0.10.1 becoming 0.10.2.dev0 and 0.10.1.dev0 becoming 0.10.1.dev1, and notes that the Cargo.lock and the kernels/uv.lock are refreshed so every site stays consistent.

That last point is the real substance. A project that distributes both a Rust tool and a Python package, and that publishes prebuilt binaries for several hardware backends, has version numbers in several places, and a mismatch between the loader and the builder is a confusing failure rather than an obvious one. Automating the bump is the cheap way to avoid it.

The remaining Makefile targets are ordinary hygiene: style and quality run ruff format and ruff check over kernels/src and kernels/tests, with the style target applying fixes, and pin-actions runs pinact to pin the GitHub Actions versions used by the workflows.

## Licence, documentation and the practical checks before you rely on it

The licence is Apache-2.0, with the text in the LICENSE file at the repository root. The documentation is published at huggingface.co/docs/kernels, and the README points there for both the loader and the builder.

Two operational details follow from the architecture and are worth checking before you build on it. The first is that the kernel is fetched at runtime, so a first use needs the Hub to be reachable and the requested build to exist for the calling machine. The second is that the version argument in get_kernel is a real dependency pin. The project is on a 0.x version line, with the recent releases v0.16.2, v0.17.1 and v0.17.2, and the loader and the builder are versioned in step, so an application should request a specific kernel version and treat the loader package as a normal pinned dependency rather than something to float.

The examples directory makes the intended usage concrete, with a basic.py file, a cookbook directory and a kernels directory of test cases, and the workflows in the repository include a test_kernels job that the README links through a status badge. A developer who wants to see how a kernel is fetched and called in a realistic setting has that starting point in the tree.

The community catalogue is worth a look before writing anything yourself, since the activation kernels used in the quick start are maintained by Hugging Face itself and the same organisation hosts more.

## Conclusion

Adopt huggingface/kernels if you want to ship a Python library that picks up an optimised kernel for the user's hardware without asking the user to install a compiled extension first, and stay with conventional pip-installed extensions if your deployment must work offline or on a runtime the Hub does not cover. Before you depend on it, check the hardware tier of your target in the README table, since ROCm, XPU, Metal and TPU kernels are not validated in CI and TPU and Neuron packages are not public, and pin the kernel version you call get_kernel with, because the package is still on a 0.x version line.

## FAQ

### What does the huggingface kernels package do?

It lets a Python application load a compiled compute kernel directly from the Hugging Face Hub at runtime, so a library can use an optimised kernel for the user's hardware without the user installing a locally compiled extension first.

### What are the requirements for using the kernels package?

The README states that installing it with pip requires torch>=2.5 and CUDA. The package is installed with pip install kernels.

### How does a Hub kernel differ from a normal pip-installed kernel package?

The README lists three differences: a Hub kernel is portable because it can be loaded from paths outside PYTHONPATH, unique because several versions of the same kernel can be loaded in one Python process, and compatible because it must work with recent Python versions and the different PyTorch build configurations, including various CUDA versions and C++ ABIs.

### Which hardware backends are fully supported by the Kernel Hub?

CUDA and CPU are tier 1, meaning kernels are supported, the kernel-builder supports them and they are validated in CI. ROCm, XPU, Metal and TPU are tier 2 with no CI validation, and the README warns that Neuron and TPU support is experimental with packages not public yet.

### How do I build a kernel with kernel-builder?

The builder is a Rust command line tool, hf-kernel-builder, built in the repository with cargo build -p hf-kernel-builder, and its reference documentation is generated into docs/source/builder-cli.md. The Makefile also provides style and quality targets that run ruff over kernels/src and kernels/tests.

## Sources

- [huggingface/kernels on GitHub](https://github.com/huggingface/kernels)
- [License: Apache-2.0](https://github.com/huggingface/kernels/blob/main/LICENSE)
- [Project website](https://huggingface.co/docs/kernels/)
- [README](https://github.com/huggingface/kernels/blob/main/README.md)
- [Releases](https://github.com/huggingface/kernels/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/huggingface-kernels