QuACK: memory bound GPU kernels written in Python through the CuTe DSL

Kernels in Python, not in CUDA C++

QuACK stands for a Quirky Assortment of CuTe Kernels, and the one technical fact that defines the project is in the second line of the README: the kernels are written in the CuTe-DSL, NVIDIA's Python DSL for the CuTe layout algebra.

That matters because of what it implies. Writing a fast GPU kernel usually means writing CUDA or C++ and compiling it, which means a toolchain, a build step and a debugging loop that is far from the model code. The CuTe DSL lets the kernel be expressed in Python while still reasoning about the low level layout of the data in memory, and the project's own write-up claims the result of getting memory bound kernels to speed of light in the comfort of Python.

The scope is deliberately narrow. The listed kernels are RMSNorm forward and backward, softmax forward and backward, cross entropy forward and backward, layernorm forward and backward, a Hopper GEMM with epilogue, a Blackwell GEMM with epilogue, and a Blackwell GeForce GEMM with epilogue.

The forward and backward pairs are the ones a training loop actually needs, and their presence tells you the project is aimed at people who train models rather than people who only serve them.

The package is quack-kernels on PyPI, licensed Apache-2.0, and the last push to the repository was on 2026-10-15, with the most recent release v0.6.5 from 2026-09-11.

The hardware list that decides whether you can use it

The requirements section is three lines and it is the most important part of the README for anyone deciding whether to install.

You need an H100, a B200 or B300, or an RTX 50 GPU. You need the CUDA toolkit 12.9 or later. You need Python 3.12.

Those are not preferences. The Hopper and Blackwell GEMM kernels and the Blackwell GeForce variant are written against specific architectures, and the point of a hand written kernel is that it exploits what the hardware does, so a kernel that targets one generation of tensor cores is not portable to the previous one.

The RTX 50 line appears separately because consumer Blackwell differs from datacenter Blackwell in ways that matter to a GEMM kernel, which is presumably why there is a third GEMM variant for it rather than one kernel with a flag.

There is also a small inconsistency worth knowing about if you are reading the metadata rather than the README. The README states Python 3.12 as the requirement, while the package metadata in pyproject.toml declares requires-python of 3.10 or later. The README is the stricter of the two, and it is the one the author wrote for users, so treat 3.12 as the supported version.

Installation: pip, extras, and the uv warning

The install section is short and it is split by CUDA version, because the package on PyPI is built for CUDA 12.9 by default.

# For CUDA 12.9:
pip install quack-kernels

For CUDA 13.x, the extra is different and an extra index URL is needed, since the matching builds come from the PyTorch wheel index.

# For CUDA 13.x:
pip install 'quack-kernels[cu13]' --extra-index-url https://download.pytorch.org/whl/cu132

Two warnings come with that. The first is about the installer rather than the package: do not use uv for CUDA 13.x installs yet, because it can race and install nvidia-cutlass-dsl with the 13.x extra in the wrong order, and the README links the upstream NVIDIA/cutlass issue. The second is an optional extra for NVIDIA matmul heuristics, described as giving better untuned GEMM configurations.

# Optional: install NVIDIA matmul heuristics for better untuned GEMM configs
pip install 'quack-kernels[heuristics]'

The optional JAX binding is a third extra, and the README notes that it pulls in jax and jax-tvm-ffi.

The development setup follows the same pattern, with a dev extra and a separate dev plus cu13 variant, and it adds pre-commit install after the editable install.

Using the kernels, and the JAX route for some of them

The public surface is a module import, and the three names in the README are the ones you reach for first.

from quack import rmsnorm, softmax, cross_entropy

JAX bindings are also available for some kernels, documented separately in the docs directory, and the example given is a module under the quack package with a _jax suffix.

from quack.softmax_jax import softmax

The qualifier some matters. The JAX interface is described as optional bindings that require jax and jax-tvm-ffi, and the documentation points at a specific file in the package as an example. So the JAX path covers part of the library rather than all of it, and the README is careful to say so.

The mechanism behind the bindings is visible in the dependencies. The package depends on apache-tvm-ffi, the Apache TVM foreign function interface, and on nvidia-cutlass-dsl at 4.7 or later for the DSL itself. The JAX extra adds jax-tvm-ffi on top of jax, so the same FFI path serves both frameworks.

Where the performance numbers come from

The README has a performance section that is a single figure, an SVG chart named for bf16 kernel benchmarks on a single row, and a pointer to the blog post for the details.

The blog post is dated 2025-07-10 and lives in the media directory of the repository, and the README describes it as comprehensive and as explaining how to get memory bound kernels to speed of light, right in the comfort of Python thanks to the CuTe DSL.

The repository layout backs up the idea that performance work is a first class activity here. There is a benchmarks directory, a microbenchmarks directory, a tools directory, an examples directory and a scripts directory, and the examples include a GEMM trace script and an IKET trace script, both of which sound like instrumentation rather than usage examples.

The optional bench extra, which pulls in pandas and tyro, is the other signal. Pulling in a command line argument library and a data frame library suggests a benchmarking setup that runs kernels over many shapes and collects the results.

For an adopter the practical consequence is that the performance claims are tied to a specific shape regime, since the chart is labelled for a single row. A kernel that is at bandwidth for one row is not necessarily at bandwidth for a batch of a thousand, and the blog post and the benchmark scripts are where to look for the details of what was measured.

The alternative: a compiled extension, and what the DSL is buying

The conventional way to get a kernel faster than the one your framework ships is to write it in CUDA C++, compile it, and call it from Python through a custom operator. The CuTe DSL approach exists to avoid that pipeline, and understanding the trade-off is what tells you whether QuACK is interesting to you.

What you give up with a compiled extension is iteration speed. Every change to the kernel means an edit, a compile, and a load, and the code you are editing is not the code you are profiling. With a Python DSL, the kernel sits next to the model code, the profiling loop is short, and the layout algebra is expressed in the same language as the rest of the experiment.

What you accept is that the DSL is a layer above the hardware with its own abstractions.

For most teams the right first move is not to adopt QuACK at all but to establish whether the framework's own implementation of the operator is the bottleneck. QuACK becomes interesting once that question is answered, because its kernels are specific enough that they will not be a drop-in improvement for every shape and every dtype.

The repository also carries AGENTS.md, CLAUDE.md and an AI directory, which tells you that the project is developed with AI assistance in the open. Whether that matters to you is a question about your own review process.

Editorial conclusion

Adopt QuACK if you are profiling a model on an H100, a B200 or B300 or an RTX 50 card and the bottleneck is a memory bound operator that the framework already provides but does not run at bandwidth for your shapes. Do not adopt it as a general library, since the hardware list, the CUDA 12.9 requirement and the Python 3.12 guidance narrow the audience to one class of machine. Before you install, check whether you are on CUDA 12.9 or 13.x, because the 13.x path needs a different extra and the project warns that uv must not be used for it.

Frequently asked questions

Official sources

1. Dao-AILab/quack on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases