NumKong widens the accumulator so an Int8 dot product stops overflowing

The accumulator decision is the whole pitch

The README opens with a failure mode rather than a feature list. Most libraries return a dot product in the same type as the input: Float16 times Float16 gives Float16, Int8 times Int8 gives Int8. A 2048 dimensional i8 dot product can reach plus or minus ten million, and i8 tops out at 127.

NumKong promotes instead. Float16 accumulates into Float32, BFloat16 into Float32, Int8 into Int32, Float32 into Float64. Overflow stops being a silent wrong answer and the intermediate range stops constraining what the kernel can compute.

That choice propagates into everything else, including the retrieval kernels the project is aimed at. A distance or similarity function over bf16 vectors has to survive the accumulation, not just the storage.

The accuracy story is built on 118-bit extended-precision baselines with per-type ULP budgets, over log-normal, uniform and Cauchy input distributions. Tests check the triangle inequality, Cauchy-Schwarz bounds, NaN propagation, overflow detection and probability-simplex constraints for each ISA variant, and results are cross-validated against OpenBLAS, Intel MKL and Apple Accelerate. Those are the project's claims about its own test suite, which is a different thing from an independent measurement.

Where the project's own latency table claims a win

For single call latency, the README reports a 2048 dimensional dot product on Intel Sapphire Rapids, single-threaded, measuring gso/s and mean relative error against a higher precision reference. The listed versions are NumPy 2.4, PyTorch 2.10 and JAX 0.9.

The f64 row reads 5.8 gso/s at 1e-16 error for NumKong against 2.0 gso/s at 1e-15 for NumPy with OpenBLAS. At f32 the gap widens to 7.1 against 1.5 gso/s, with error 2e-7 versus 2e-6.

The narrow precision rows are where the comparison changes shape. At bf16 NumKong reports 9.7 gso/s against 0.5 for both PyTorch and JAX, at f16 11.5 against 0.5 and 0.4, and at i8 14.8 gso/s with zero relative error where the other three are listed as overflow. NumPy has no entry at all for bf16 and e5m2, marked as absent in the table rather than slow.

The accuracy note underneath is specific: for f64, compensated Dot2 summation reduces error by 10 to 50 times against naive Float64 accumulation depending on vector length, and for f32, widening to Float64 gives 5 to 10 times lower error.

The same comparison on matmul, where NumKong is behind

The README anticipates the obvious objection, that PyTorch and JAX are built for throughput rather than single call latency, and then shows matrix multiplication anyway: 2048 by 2048 times 2048 by 2048, same machine, same single-threaded setting.

The ordering reverses. At f64, NumKong reports 8.6 gso/s against 65.5 for NumPy with OpenBLAS and 68.2 for PyTorch with MKL. At f32, 37.7 against 140 and 145. Those are the configurations where an established BLAS has had decades of tuning.

Narrow types tell a different story. At bf16, NumKong reports 458 gso/s against 851 for PyTorch with MKL, with a slightly worse error of 3.6 percent against 1.8. At f16, 103 against 140. At i8, 1279 gso/s where the other columns are listed as overflow. At e5m2, 398 gso/s at zero error.

So the pattern in the project's own data is consistent: NumKong competes on latency and on types the big stacks handle badly, and loses on wide-precision throughput matmul. A throughput-oriented pipeline on f32 is not where this library wins.

Five megabytes against 705, and your own threads

The size and parallelism table in the README draws a line that matters more than any gso/s figure. PyTorch with MKL is listed at 705 MB, JAX with jaxlib at 357 MB, NumPy with OpenBLAS at 30 MB, mathjs at 9 MB and NumKong at 5 MB.

The parallelism column is the substantive difference. PyTorch gets vector and tile SIMD, OpenMP threads and hidden allocations. JAX gets vector SIMD, XLA threads and hidden allocations. NumPy gets vector SIMD, built-in threads and hidden allocations. mathjs gets no SIMD, no threads and many allocations. NumKong is listed as vector and tile SIMD, your threads, your allocations, across seven languages.

Hidden allocation is the practical part. A library that allocates its own buffers inside a hot loop will show up in your memory profile and will not share your allocator, so a dependency that hands you kernels with no allocation and no thread pool of its own is a different kind of component.

The README points to a separate NumWars repository for a broader throughput comparison than the one it prints.

Installing per language, and what each binding costs you

The quick start table maps languages to install mechanisms. C and C++ go through CMake with headers and prebuilt binaries for Linux, macOS, Windows and Android. Python uses pip on Linux, macOS and Windows. Rust uses cargo add. JavaScript uses npm install, working on Node.js, Bun, Deno and browsers. Swift goes through Swift Package Manager on Apple platforms and Linux. Go uses go get, on Linux, macOS and Windows through cGo.

For Python, a plain install is the documented path:

The wheel matrix in pyproject.toml is broader than the README table implies: manylinux and musllinux wheels for Linux on x86, aarch64 and riscv64, additional Linux wheels for i686, ppc64le and s390x, macOS wheels for x86_64 and arm64, Windows wheels for AMD64 and ARM64, and pyodide wheels for wasm32, covering Python 3.10 to 3.14 plus free-threading 3.14. Cross-compilation uses environment variable overrides such as NK_TARGET_X8664_ and NK_TARGET_ARM64_.

On the Rust side, the crate is numkong, the source path is rust/numkong.rs, rust-version is 1.64 and the note says that version is where Core C FFI landed in stable Rust. Cargo features are minimal: std, parallel, which pulls in fork_union, and wasm-runtime. There is a no-std category, which is unusual for a library that also wants AMX and SME.

The JavaScript package runs a capability probe before install, since the package.json preinstall script executes node probes/probe.js before node-gyp-build runs.

17 numeric types across 30 SIMD backends, with hardware-aware defaults

The coverage claim is 17 numeric types from 6-bit floats to 128-bit complex numbers, across dozens of operations and 30 or more SIMD backends, with more than 2000 kernels. Architectures named include x86, Arm, RISC-V, LoongArch, Power and WebAssembly.

The tags and README reference two dimensional registers, AMX and SME, which is how integer matmul gets its throughput rather than through vector units.

Defaults are chosen per architecture: Arm prioritises f16, x86 prioritises bf16. That is a sensible heuristic and also a trap, because it means the same source produces different precision behaviour on an Apple machine and on an Intel one unless you pin the type.

The test scripts in package.json show how broad the runtime matrix is: separate targets for native, emscripten, emscripten64, wasi-node and browser via playwright, plus Bun and Deno. The Rust crate tests wasmtime and wasmtime-wasi. Hardware aware defaults plus per-target tests is the shape of a library that expects to be dropped in without configuration, and also the shape of one where you should read the capabilities output on your target rather than assume.

Adoption risks and what to check before committing

The first risk is the single maintainer shape. The Cargo.toml and package.json both name one author, the release cadence is fast, at v7.8.5 on 2026-10-05 with v7.8.4 three days earlier, and the version number is already at 7 with no 1.0 in sight. Fast releases with a major-version-as-integer scheme mean interface churn is possible across releases.

The second is that this is a project that competes with heavily optimised vendor libraries on their home turf. The f64 and f32 matmul rows are the honest signal there.

The third is scope. Seventeen numeric types, thirty backends and seven bindings is a large surface to hold stable, and the README does not document a compatibility policy between releases or a deprecation path.

Licensing is Apache-2.0, stated in the LICENSE file, in Cargo.toml and in package.json, which carries a patent grant alongside the usual terms. That is the permissive end of the scale and is a point in its favour for embedding in a product.

Before adopting, print the capabilities of the built extension on your target machine and confirm the numeric type and backend you expected are the ones selected, then benchmark the one operation you actually care about rather than the table in the README.

Editorial conclusion

Adopt NumKong for vector search, distance kernels and small dimensional numerics on one CPU where you care about tail latency and want a few megabytes of dependency instead of hundreds. Do not adopt it to replace a tuned BLAS inside a PyTorch training loop, since the README's own matrix multiplication table puts NumKong below NumPy and MKL for f64 and f32. Verify first which numeric type your data actually is, because the library sets hardware aware defaults with Arm prioritising f16 and x86 prioritising bf16, and a wrong default is silently a different model of your arithmetic.

Frequently asked questions

Official sources

1. ashvardanian/NumKong on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases