# fast.cu is a notebook of CUDA kernels written to beat cuBLAS

> The pranjalssh fast.cu repository collects hand written GPU kernels and the numbers they produced on H100 and GB300 hardware, including a bf16 matrix multiply that the README reports as faster than cuBLAS and a ten rung NVFP4 optimisation ladder. It is a research log rather than a library, and the measurements are the author's own.

**pranjalssh/fast.cu** — Fastest kernels written from scratch

- Repository: https://github.com/pranjalssh/fast.cu
- Stars: 629 · Forks: 82
- Language: Cuda
- License: MIT
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/pranjalssh-fast-cu

## A repository whose output is a table of numbers

fast.cu describes itself in one line: fastest kernels written from scratch. That is the entire project description, and the repository is correspondingly small in structure. The top level contains a .gitignore, a LICENSE, a Makefile, the README, and two directories, gb300/ and h100/, which is where the kernels live.

The primary language is recorded as Cuda, the licence is MIT, and there are no GitHub releases. The last push was on 2026-08-15.

What makes it more than a code dump is that every kernel comes with a comparison. Each example in the README shows the kernel's own result next to a vendor library's result on the same hardware, which is the only way to say anything useful about a hand written kernel. The comparisons are against cuBLAS for the matrix multiplies and the reduction, and against cuBLASLt for the NVFP4 example, which is the right baseline in each case because those are the libraries NVIDIA ships for those operations.

So the repository's value is two things. It is a set of implementations you can read, in a language where reading a good kernel teaches you something, and it is a set of claims with numbers attached, which you can argue with or try to reproduce. The README also links to a longer explanation at cudaforfun.substack.com, titled as a worklog on outperforming cuBLAS on H100, which is where the reasoning behind the optimisation lives rather than in the README itself.

## The H100 bf16 matmul and its cuBLAS baseline

The first example is matrix multiplication of square bf16 matrices accumulated in fp32 on an H100, and the README gives results at two sizes:

```
N=4096
Kernel: 763 TFLOPs
cuBLAS: 716 TFLOPs

N=8192
Kernel: 808 TFLOPs
cuBLAS: 795 TFLOPs
```

Read those numbers for what they are. At N=4096 the kernel is ahead by 47 TFLOPs, which is about 6.6 percent over the cuBLAS figure. At N=8192 the gap narrows to 13 TFLOPs, about 1.6 percent. That is a meaningful detail, because a claim of being faster than a vendor library usually means something very different at one size than at another, and the larger size is where the vendor library is at its best.

The accumulation detail matters too. Accumulating in fp32 from bf16 inputs is the standard mixed precision pattern, and it means the comparison is not against a different numerical contract, it is against the same contract with a different implementation.

Running it is two commands, one to build and one to run:

```
make matmul && out/matmul
```

The example kernels live in the h100/matmul/ directory, and the orchestration is in h100/matmul.cu. So the kernels and the harness that drives them are separate files, which is the arrangement you want if you intend to read the difference between them.

The Makefile confirms the architecture this is aimed at. The base compile adds -gencode arch=compute_90a,code=sm_90a, which is the Hopper generation target, and links against cuBLAS with -lcublas -lcuda. The compile flags are aggressive: -O3, -DNDEBUG, --use_fast_math, --expt-relaxed-constexpr, --expt-extended-lambda, plus -fno-strict-aliasing and -fPIE on the host compiler side.

## The GB300 NVFP4 example is a ladder of ten rungs

The second example is the one that shows how the project works as a discipline rather than as a deliverable.

The NVFP4 example contains ten complete kernel snapshots, a shared correctness and benchmark runner, and a cuBLASLt baseline. The Makefile shows what that means in practice. There is a variable NVFP4_RUNGS set to 0 through 9, and the targets are built from the same main.cu with a different gemm header selected per rung using a define that names the header file:

```bash
make nvfp4
./out/nvfp4 8192 8192 8192 --rounds 5 --cooldown-sec 8

make nvfp4-ladder
./out/nvfp4-r0 8192 8192 8192 --rounds 5 --cooldown-sec 8
./out/nvfp4-r9 8192 8192 8192 --rounds 5 --cooldown-sec 8
```

The ladder build produces all ten binaries at once, so you can run rung 0 and rung 9 side by side and see what the nine intervening optimisations bought. Each rung is a separate header file, gemm0.cuh through gemm9.cuh, compiled against the same main.cu, which means the comparison between two rungs is not confounded by changes in the surrounding code.

The runtime flags are worth noting too. --rounds 5 asks for five measurement rounds, and --cooldown-sec 8 puts a pause between them, which reads as an attempt to let the GPU cool between runs rather than reading a thermally throttled peak.

The hardware requirement is stated plainly: this needs a GB300 or B300 and CUDA 13.1, and the compile flags confirm it, with -gencode arch=compute_103a,code=sm_103a and a separate nvcc path at /usr/local/cuda/bin/nvcc, linked against cuBLASLt. The README points at gb300/nvfp4/README.md for every optimisation rung, the scheduler controls, the correctness-only runs and the benchmark modes.

## The sum reduction, and a gap of about 1.5 percent

The third example is a sum of 2^30 elements on the H100, and its numbers are closer than the matrix multiply's:

```
Kernel: 3240.11 GB/s
cub Library: 3193 GB/s
```

That is a bandwidth bound problem rather than a compute bound one, so the comparison with the CUB library is the meaningful one, and the gap is about 47 GB/s, roughly 1.5 percent. It is a much smaller win than the matmul at N=4096, and it is the kind of number that tells you the ceiling on this class of optimisation is low. A reduction that already runs at close to memory bandwidth has very little left to give.

The comparison here is against the CUB library rather than cuBLAS, which is the right choice, and the two libraries are linked differently in the Makefile, with -lcublas for the matmul and sum path and -lcublasLt for the NVFP4 path.

Building and running it is the same two step pattern:

```
make sum && out/sum
```

and the kernel itself is a single file, h100/sum.cu.

Taken together, the three examples tell a more honest story than the headline suggests. The largest win is on a compute bound operation at a specific size, the reduction win is marginal, and the matmul win shrinks as the problem grows. That pattern is what a real optimisation log looks like, and it is more useful to a reader than a single flattering number would be.

## The Makefile also tells you how to profile the kernels

The build file is short, but two details in it are worth reading closely.

The first is the profiling targets. For the sum and matmul binaries there are matching profile targets, sumprofile and matmulprofile, and they are built from a variable that resolves ncu, the Nsight Compute profiler, from the path. The command it runs is:

```
NCU_COMMAND = sudo $(NCU_PATH) --set full --import-source yes
```

So profiling a kernel here means running Nsight Compute with the full metric set, with source import enabled, and it means doing it under sudo, because the profiler needs privileges that a normal user does not have. The profile target writes its output to a file named after the target, so you get sumprofile and matmulprofile as artefacts.

The second detail is the warning flags. The host compiler is given -fno-strict-aliasing, which disables an optimisation that assumes pointer types are not aliased, and -Wno-psabi, which suppresses a warning about the ABI. Both are the kind of flag you see in performance sensitive CUDA code, and both are also flags that can hide a real bug. The nvcc side uses -w, which suppresses all warnings from the device compiler entirely.

Combined with --use_fast_math, this is a build configured for throughput rather than for correctness by default. That is a reasonable choice for a benchmark harness, and it is exactly the kind of thing to check before you copy a pattern from one of these kernels into production code.

The clean target is rm $(OUT_DIR)/*, so the build directory is not preserved between runs.

## A worklog, not a library

The honest framing of fast.cu is that it is a public notebook with compilable contents, not a package.

There is no API to import, no installation, no versioned release, and no compatibility promise. The repository ships kernels for two specific architectures, on hardware you are unlikely to have, and the numbers attached to them are measurements the author took on that hardware. There is no CI badge, no coverage report and no test suite mentioned in the README, although the NVFP4 example does include a shared correctness runner, which is the one part of the project that speaks to verification rather than to speed.

That combination makes it valuable in two specific ways. As a reading exercise, the kernels are a way to see how a hand written implementation approaches a problem that a vendor library already solves well, and the ladder structure of the NVFP4 example makes the intermediate steps legible rather than presenting only the final version. As a benchmark harness, the presence of the cuBLAS and cuBLASLt baselines means the claims are checkable, which is rarer than it should be.

What it is not is a drop in replacement for cuBLAS. There is no packaging, no dispatch layer, no fallback, and the comparison shows how close the margin is. If you need a faster matmul in a product, this repository is where you would look for ideas and where you would validate an idea, not what you would depend on.

The licence is MIT, so the code can be reused, modified and shipped without obligations beyond keeping the notice, which for kernel code you might actually incorporate is the most practically useful thing about it.

## Conclusion

Adopt fast.cu as a reference and a benchmark, not as a dependency, because it is a set of kernels and a worklog rather than a packaged library with an API you can call. Do not use it as evidence that hand written kernels beat vendor libraries in general, since the published comparisons are the author's own measurements on one architecture each. Before you draw a conclusion from it, read the linked worklog, check which GPU and CUDA version each result needs, and reproduce the baseline yourself with the cuBLAS and cuBLASLt comparisons the repository ships.

## FAQ

### What is in the fast.cu repository?

Hand written CUDA kernels for two architectures, with the author's own comparisons against vendor libraries. The examples are an H100 bf16 matrix multiply with fp32 accumulation, a GB300 NVFP4 matrix multiply built as a ladder of ten kernel snapshots, and an H100 sum reduction over 2^30 elements.

### How do I build and run the fast.cu H100 matmul example?

Run `make matmul && out/matmul`. The example kernels are in the h100/matmul/ directory and the orchestration code is in h100/matmul.cu, and the build compiles for the sm_90a architecture with nvcc.

### What hardware does the NVFP4 example in fast.cu need?

The README states it requires a GB300 or B300 and CUDA 13.1. The Makefile compiles it for compute_103a and sm_103a with a separate nvcc at /usr/local/cuda/bin/nvcc, and links against cuBLASLt.

### How do I profile the fast.cu kernels?

The Makefile provides sumprofile and matmulprofile targets that run the Nsight Compute profiler with the full metric set and source import enabled, under sudo, writing the output to a file named after the target.

## Sources

- [Issues](https://github.com/pranjalssh/fast.cu/issues)
- [License: MIT](https://github.com/pranjalssh/fast.cu/blob/main/LICENSE)
- [pranjalssh/fast.cu on GitHub](https://github.com/pranjalssh/fast.cu)
- [README](https://github.com/pranjalssh/fast.cu/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/pranjalssh-fast-cu