AutoKernel: an overnight agent that edits one kernel at a time

Edit one kernel, benchmark, keep or revert

AutoKernel turns kernel tuning into a loop a coding agent can run unattended. The design is borrowed from karpathy/autoresearch, which put autonomous agents on LLM training research; AutoKernel moves the same idea to GPU kernels, where an agent modifies one file, runs a fixed evaluation, keeps or reverts, and repeats indefinitely. The agent reads program.md, which the project calls its research org code, then edits kernel.py for one kernel at a time, runs bench.py, and keeps or reverts the edit. A fixed evaluation decides every outcome, so no human sits between a change and its verdict. orchestrate.py is the scheduler: it picks which kernel to optimize next with Amdahl's law and tracks aggregate progress, which puts cheap wins on hot kernels ahead of expensive ones on cold kernels. Costs are stated plainly, at about 90 seconds per experiment, roughly 40 experiments an hour, and something near 320 across all kernels in one overnight run. program.md is written for long unattended sessions, carrying a 6-tier optimization playbook, a decision framework, crash handling and Amdahl reasoning built in, aimed at stretches of 10+ hours.

Four scripts frame the loop around a model

profile.py runs torch.profiler over a model, ranks kernels by GPU time, and classifies each one as compute bound or memory bound. extract.py turns the top N of those into standalone Triton or CUDA C++ kernel files. verify.py plugs the finished kernels back into the model and reports the total speedup end to end. Nine kernel types ship, and the key metric column splits them: matmul, flash_attention and fused_mlp are graded in TFLOPS, while softmax, layernorm, rmsnorm, cross_entropy, rotary_embedding and reduce are graded in GB/s. That split is a constraint rather than a label, since a memory bound kernel and a compute bound kernel cannot be improved by the same edit. Nine types is also the ceiling on coverage, because an operator outside that list has no starter file in kernels/ and no PyTorch reference in reference.py. One detail only shows up on a close read of the quick start: the command it gives profiles models/llama_7b.py with `--class-name LlamaModel`, which the model table lists as the 160M compact model, while the 7B weights in the same file need `--class-name LlamaModel7B` and `--input-shape 1,2048`. The bundled definitions are self-contained, so no transformers library is needed for GPT-2 Small at 124M, the compact LLaMA, a 7B LLaMA or BERT-base at 110M, and models/custom.py is a template for your own class.

Five correctness stages guard every reported speedup

An edit earns a speedup number only after passing five stages: smoke, shape sweep, numerical stability, determinism and edge cases. bench.py adds performance measurement and roofline analysis on top of those checks, and the agent may keep a kernel only when the whole set holds. reference.py provides the PyTorch version of each of the nine kernels, so correctness is judged against a fixed implementation instead of a stored output file. KernelBench applies a different contract, with kernelbench/bench_kb.py running 5-trial correctness plus CUDA event timing plus stability plus determinism, while kernelbench/scorer.py folds a whole difficulty level into a single fast_p figure. Two correctness definitions in one repository means a kernel that survives the main loop has not automatically passed the KernelBench grader, and program_kb.md exists partly to bridge that gap.

uv sync resolves torch against the pytorch-cu128 index

Requirements are an NVIDIA GPU (tested on H100/A100/RTX 4090), Python 3.10+ and uv. The install is four commands, and the last one builds the test data and baselines that nothing downstream reads until it has run.

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/RightNow-AI/autokernel.git
cd autokernel
uv sync
uv run prepare.py

The base environment is deliberately small, and it resolves through an explicit wheel index rather than PyPI.

[project]
name = "autokernel"
version = "1.0.0"
requires-python = ">=3.10"
dependencies = [
    "matplotlib>=3.10.0",
    "numpy>=2.2.0",
    "pandas>=2.2.0",
    "torch>=2.4.0",
    "triton>=3.3.0",
]

uv honours the [tool.uv.sources] section, which pins torch to the pytorch-cu128 index at https://download.pytorch.org/whl/cu128 and marks it explicit, so a machine that needs a different CUDA build has to edit pyproject.toml, a case the README does not document. First use profiles a bundled model, passing the model file with `--model`, the class with `--class-name`, a token count with `--input-shape 1,512` and a precision with `--dtype float16`, then pulls out the top five kernels and confirms the harness runs.

uv run extract.py --top 5
uv run bench.py

bench.py should pass on a fresh checkout. If it does not, every keep or revert decision that follows is built on a broken reference.

The CUDA C++ backend needs an extra the quick start omits

The README headline promises Triton or CUDA C++ kernels, and extract.py exposes a backend flag whose documented values are `triton` and `cuda`. A plain sync installs neither compiler helper nor build tool for the second option. pyproject.toml keeps the CUDA C++ backend inside an optional extra named cuda whose only dependency is ninja>=1.11.0, carrying a comment that it speeds the build from about 30s to about 5s. Nothing in the quick start installs it, and the documentation never says that choosing the cuda backend without it will fail. The same narrowing shows up in the one-line repository description, which mentions Triton only. Four more extras widen the surface.

cuda = [
    "ninja>=1.11.0",
]

The models extra pulls transformers>=4.50.0, accelerate>=1.5.0 and safetensors>=0.5.0 for HuggingFace ingestion, kernelbench pulls datasets>=2.16.0, profiling pulls HolisticTraceAnalysis>=0.2.0, and hf-kernels pulls kernels>=0.4.0 with huggingface-hub>=0.20.0. A team pointing the profiler at a HuggingFace checkpoint has to run `uv sync --extra models` first, and a team that wants to publish optimized kernels instead of leaving them in a branch needs the hf-kernels extra alongside export_hf.py. The project is MIT licensed, and the repository lists a single LICENSE file with no separate terms covering what an agent writes into kernel.py.

KernelBench exposes the gap against one-shot generation

One-shot generation is the real alternative here, and the project's own comparison is explicit. Most KernelBench evaluations prompt a model once for a ModelNew class, while AutoKernel runs 50-300+ iterative refinement experiments per problem against the same fast_p metric, exploring a space instead of guessing. KernelBench supplies 250+ standardized problems across 4 difficulty levels, and bridge.py is the piece that turns one of them into a working starter. bridge.py fetches problems from HuggingFace or a local repo, caches them and writes a starter kernel.py, and program_kb.md tells the agent how to write ModelNew, when to reach for CUDA C++ instead of Triton, and which fusion strategy fits each problem level.

uv sync --extra kernelbench
uv run kernelbench/bridge.py fetch --source hf --level 1
uv run kernelbench/bridge.py setup --level 1 --problem 1 --source hf
uv run kernelbench/bench_kb.py
uv run kernelbench/scorer.py --level 1

At about 90 seconds an experiment, the top of that range spends a full overnight budget on one problem, while the ~320 figure covers every kernel of a single model. The pipeline table also names fewer tools than the repository ships: analysis.py and export_hf.py appear in the tree and in no table, and examples/hf_kernels_test/ sits beside them. kernel.py, the file the agent edits, is a checked-in source file rather than something the pipeline generates.

The operator supplies the agent, not the harness

Nothing in the repository runs the loop by itself. The operator opens Claude, Codex or any coding agent inside the project directory and hands it one line.

Read program.md and let's kick off a new experiment. Start with setup.

That sentence is the whole operator interface. From there the agent profiles the model, presents a plan, creates a branch such as autokernel/mar10-llama7b, works through the kernels in priority order and verifies the result end to end. The failure modes follow from that shape. With no coding agent available, the loop never starts, and the harness cannot spot a stalled edit on its own. A kernel can clear all five stages and still be wrong for a real workload, because bench.py compares against reference.py rather than against production traffic. Crash handling is delegated to program.md, and no rollback command is documented for a branch the agent has already pushed. Nothing has been pushed to the repository since 2026-03-19, and there are no tagged releases to install instead.

Editorial conclusion

AutoKernel suits teams that already run a PyTorch model on an NVIDIA GPU and want a repeatable kernel search instead of hand tuning. It does not suit a single kernel a competent engineer can write in an afternoon, or any pipeline with no coding agent ready to drive the loop. Before adopting it, check three things: that your model is built from the nine supported kernel types, that uv can resolve torch from the pytorch-cu128 index on your machine, and that your correctness bar matches the five stages in bench.py rather than the 5-trial KernelBench contract.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. RightNow-AI/autokernel on GitHub