cpufp: measuring the peak of floating point and matrix instructions per CPU

A peak number, not an application number

cpufp is described as a CPU tool for benchmarking the peak performance of floating points and AI instruction sets, and it can automatically sense the local SIMD and DSA instruction set architectures while compiling. That word peak is the key to reading its results correctly. The tool measures what the hardware can do in the ideal case, with the instruction stream arranged to hit the widest possible execution port, and it is not a general purpose benchmark of real workloads. That makes it useful for a specific question, what is the ceiling on this processor for a given data type, and nearly useless for another, which application runs fastest on this chip. The repository is written primarily in assembly, with a directory per architecture, which tells you the design intent: the tool has to emit and time instructions directly rather than go through a compiler's abstraction, and the code that measures an instruction is close to the instruction itself.

Nine architectures, and a support table that rules out Windows

The support table in the README is the first thing to read, because it decides whether the tool can run on your machine at all. arm64 is supported on Linux and MacOS but not on Windows. e2k, loongarch64, riscv64 and x86-64 are listed as supported on Linux only. In other words, the only configuration where a Windows user could expect the tool to work is not covered at all, and the macOS support is limited to arm64, which in practice means Apple silicon. The repository layout matches the table exactly, with an x64 directory, an arm64 directory, a riscv64 directory, a loongarch64 directory, an e2k directory and a common directory for shared code. There are no GitHub releases for the project, so the binaries come from running the build scripts yourself, which the README documents per architecture.

The x86-64 instruction set, from SSE to AMX

The x86-64 table is the most detailed, and it reads like a map of the last two decades of Intel and AMD. The SIMD entries start at SSE and SSE2 for fp32 and fp64 before Sandy Bridge, move to AVX and then FMA for fp32 and fp64 from Haswell and Zen, then to AVX512f from Skylake X and Zen4. The int8 and int16 vector instructions are covered by AVX512_VNNI from Ice Lake and AVX_VNNI from Alder Lake. The fp16 entry is AVX512_FP16 from Intel Sapphire Rapids and the bf16 entry is AVX512_BF16 from AMD Zen4. The matrix rows are the AMX extensions, with AMX_INT8 and AMX_BF16 from Intel Sapphire Rapids and AMX_FP16 from Intel Granite Rapids, plus the newer AVX_VNNI_INT8 and AVX_VNNI_INT16 from Intel Lunar Lake. For a reader building an AI inference stack, that last group is the interesting part, because it is where the density of new instructions is highest and where published peak numbers are least available.

The other four architectures have their own ISA stories

arm64 has asimd for fp32 and fp64 from the Cortex-A57 and A53, asimd_hp for fp16 and asimd_dp for int8 from the Cortex-A75 and A55, and then two matrix instructions, bf16 and i8mm, from the Cortex-X2, A710 and A510. The riscv64 entry is the RISC-V V vector extension at version 1.0 for fp16, fp32 and fp64, plus a matrix instruction named ime for int8 from the SpacemiT X60, and the README flags that ime is a SpacemiT custom vendor extension rather than a standard part of the ratified set. loongarch64 covers LASX and LSX for fp32 and fp64 from the Loongson 3A5000, plus a scalar FP entry. The e2k table is different in kind, because Elbrus supports instructions that perform two independent operations, like FMA but with additional rounding since the operations are independent, and the README works through a concrete example, fmul_addd, with its operation and its latency and throughput on elbrus-v4 and elbrus-v1.

Building per architecture with the shell scripts

The build is a set of small shell scripts, one per target, and the README lists them in order:

`./build_x64.sh`

The same pattern applies to the other four, with ./build_arm64.sh, ./build_riscv64.sh, ./build_loongarch64.sh and ./build_e2k.sh, and a ./clean.sh to remove the built output. Each script exists because the tool compiles the instruction set it is testing, and that differs per architecture, which is also why the ISA detection happens at compile time rather than at run time. A consequence worth noting is that the machine you build on and the machine you benchmark on are not necessarily the same, and a cross build would produce a binary for the ISA the compiler detected, not necessarily for the ISA of the host.

Running the benchmark with a thread pool and an idle gap

The benchmark command takes two parameters:

`./cpufp --thread_pool=[xxx] --idle_time=yyy`

The thread_pool value is the list of CPU threads to benchmark, taken from setting affinities, and the README says to reference the result of the lstopo command, giving the example format [0,3,5-8,13-15]. The idle_time value is the interval in seconds between any two adjacent benchmarks, and it defaults to 0. The reason to set it is thermal and frequency behaviour, since a sustained peak measurement on a single core is a different number once the core has heated and the clock has moved, and the gap between runs is the simplest way to get comparable figures. The tool is therefore best read as a laboratory instrument that you configure deliberately, not a command you run and read.

Benchmark results are published per processor model

The README carries a large results table linking to a markdown file for each processor that has been measured, and the breadth of that list is itself information. On x86-64 there are entries for the AMD Ryzen7 9700X, the Ryzen7 8845HS, the Ryzen9 6900HX, Intel Xeon Gold 6455B, Xeon W9-3495X, Ultra7 255H, Core i5-1340P, Core i7-8809G, Core i3-8121U, Intel N150 and a ZHAOXIN KX-6640MA. On arm64 there are Apple Silicon M4 Max and M2 Max, Qualcomm Snapdragon 8 Gen5 and X Elite X1E80100, AWS Graviton 3E, Broadcom BCM2712 and BCM2711, CIX P1 CD8180, HUAWEI Kunpeng 920 7260 and 920 2249K, Phytium D2000/8, and RockChip RK3588 and RK3399. riscv64 has SpacemiT K1 and Kendryte K230, and loongarch64 has Loongson 3A6000. So if you care about a chip that is not on this list, running the tool is the only way to get the numbers, which is the clearest argument for the project existing at all.

Editorial conclusion

cpufp fits a kernel or compiler engineer who needs to know what a specific processor can actually do in floating point, especially a chip with a matrix extension nobody has benchmarked yet, since the tool detects the local ISAs while compiling and reports per-instruction peaks. It does not fit a general performance comparison, because the numbers are theoretical peaks rather than application throughput, and it does not run on Windows at all according to the support table. Before running it, check that your architecture and operating system are in the supported list, read the result files for a comparable processor to understand the output format, and set the thread pool and idle time so the run does not disturb the machine you are benchmarking.

Frequently asked questions

Official sources

1. Issues
2. License: GPL-3.0
3. pigirons/cpufp on GitHub
4. README