warp renamed itself from WASTE but kept the old name in the CLI, and a bigger expert cache made it eight times slower
Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.
At a glance
- What is it?
- warp is a dependency-free C inference engine that keeps a mixture-of-experts model trunk in RAM and streams selected experts off NVMe, so a 2.78 trillion parameter model opens on a laptop. Its README carries unusually honest measurements, including two tables that document how performance collapses when the obvious tuning knob is turned the wrong way.
- Who is it for?
- Read this engine if your workload tolerates token rates in the single digits and you have fast internal NVMe, because the design makes storage bandwidth the only thing that matters and the README is honest about where that bandwidth goes. Do not adopt it for interactive use on Kimi K3, where one run reads 4.6 TB and takes ten minutes, and do not assume the new name is complete, because the binary, the warning prefix and the shard environment variable are all still WASTE.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 3 days ago.
- What is it written in?
- Mainly C, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- sqliteai/warp2,559 ★ · +132
| Repository | 2026-09-17 | 2026-11-15 |
|---|---|---|
| sqliteai/warp | 2.4k | 2.6k |
132 Stars in the last 59 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
The rename reached the title but not the command line
The heading names WARP as Weight-Aware Runtime and Paging and notes in parentheses that it was formerly WASTE. Nothing else in the file agrees. The quickstart still invokes the old binary and the old prefix:
$ waste run ~/models/k3.waste 'What is the capital of Italy?'
waste: no --budget, using 46.39 GB of 64.00 GB (expert cache 17.56 GB)
[16 tokens, 26.87 s, 0.60 tok/s | experts 9000 hit / 14552 miss = 38%]The container keeps the old extension, the message prefix is the old name, and further down the shard environment variable is still `WASTE_BANK_SHARDS`. The Makefile header comment reads WASTE, embeddable MoE inference engine plus CLI, and the accelerator build option is `make WASTE_ENABLE_METAL=1`. So a rename reached the title and the npm-style prose while every identifier a user types kept the previous name.
A larger expert cache made decoding eight times slower
Four cache sizes were measured inside one process, and the last two rows are the reason to read the whole table:
| expert cache | hit rate | decode | | 3.32 GB | 29.1% | 0.56 to 0.58 tok/s | | 17.32 GB | 36.2% | 0.63 tok/s | | 23.32 GB | 38.4% | 0.07 to 0.09 tok/s | | 29.32 GB | 41.3% | 0.07 to 0.08 tok/s |
Hit rate improves monotonically across all four rows. Throughput does not: it peaks at the default and then falls by roughly a factor of eight, twice in a row, while the bytes read keep dropping. The stated mechanism is that the engine stays inside its own budget while the machine does not, so a cache hit turns into a page fault. The README's conclusion from that is blunt, that giving the process more memory is not always faster. Most of the memory requirement is the resident trunk rather than the cache, and shrinking the cache from 17.32 GB to 3.32 GB costs about 10% of throughput while enlarging it past the default costs everything.
Fewer experts per token is the speed knob, and it has a price
Experts routed per token is a manifest field, `num_experts_per_token`, and Kimi K3 ships at 16. One load with the settings interleaved produced:
| experts per token | decode | KL from top-16 | working set | | 16 | 0.59 tok/s | none | 17.01 GiB | | 12 | 0.70 tok/s | 0.007 | 12.76 GiB | | 8 | 0.89 tok/s | 0.037 | 8.50 GiB | | 4 | 1.06 tok/s | 0.118 | 4.25 GiB |
The README calls top-8 a 1.49x gain for a divergence twice that of a quantization this project rejects elsewhere, and notes that it reproduces the top-16 greedy continuation on the prompts tested. Top-4 does not: its next-token distribution still looks close, and it stops following the prompt within a few tokens. That is why the gate is a continuation check rather than a KL threshold, and why the default stays at 16 despite every row below it being faster.
One row refuses to climb, and the cache curve explains it
Across the four measured models the 200-token column is usually faster than the 64-token one, because the expert cache is still filling during the first few dozen tokens. DeepSeek-V4.1-Flash is the exception: 3.77 tok/s over 64 tokens and 3.71 over 200, flat rather than climbing. The stated reason is that its cache already sits at 93% by the 64th token, so the longer run has nothing left to win. The cache-size curve behind that number puts the knee at 9.6 GB, while the automatic budget allocates 41.74 GB, slightly past the top of the curve, buying hit rate that no longer buys time. This is the same lesson as the oversized-cache table, arriving from the other direction: on this machine a budget past the knee is not free, and the engine's default cannot be tuned upward on the strength of a hit-rate number alone.
Storage bandwidth is the whole budget
A cold Kimi K3 token reads about 17 GB of experts. The internal SSD on the test machine sustains 12.78 GB/s and a tested USB enclosure managed 0.94 GB/s, so the container belongs on internal NVMe and the ratio between the two numbers is the ratio between the two experiences. The reason the full model fits at all is layout and compression: the container is arranged so one expert needs one aligned read, those reads overlap with computation, and unused RAM becomes a bounded cache. Experts use 3-bit residual vector quantization while the more sensitive shared weights stay at 4 or 8 bits, and K3's linear attention with a compressed latent KV cache holds 4K context in about 0.21 GB instead of 11.25 GB. Published weights occupy 1.42 TB and the converted container is 982 GB.
One run of K3 reads 4.6 TB and takes ten minutes
The performance table has four models and one empty cell. Kimi K3 fills the 64-token column at 0.45 to 0.62 tok/s and its 200-token cell holds a dash, because one run of it takes ten minutes and reads 4.6 TB. That single omission explains the shape of everything else in the file: why 64 GB is called the practical minimum, why a 32 GB machine is said to open the model but page heavily, and why the tuning discussions concentrate on the smaller models. The two models that do have both columns are the ones a person would actually sit and wait for, DeepSeek-V4.1-Flash climbing to 3.71, GLM-5.3-Flash at 3.32 to 3.86, and Kimi-Linear at 14.29 to 17.22 tok/s from a 19 GB container. The engine's headline is a demonstration, not a throughput target.
The build asks for gnu11 and probes for a real python3
The Makefile explains two decisions at length. The compiler standard is `-std=gnu11` rather than `c11` because under a strict ANSI setting glibc hides every POSIX extension, which would leave pread, fcntl, posix_memalign and the pthread family implicitly declared on Linux, with only model.c defining _GNU_SOURCE itself. The interpreter for the Python recipes is found by running candidate commands instead of looking one up, because on Windows the name `python3` on PATH is often the Microsoft Store alias: a zero-byte reparse point that exits 49 and prints an advert, so a recipe would otherwise report itself broken. The probe is written recursively rather than immediately so that it runs only when a recipe actually expands the variable, since an immediate assignment would cost two process spawns on every make invocation, including clean.
The release titles name models the measurement table never covers
Three recent tags read like a changelog of capabilities: v0.8.1 for Qwen3.8-Flash-Next, v0.8.0 for DeepSeek-V4.1-Flash, and v0.7.2 for bank striping, K2 tool calling and three oracles. None of those three tag titles is explained anywhere in the README, and the Qwen variant named by the newest release appears in no measured row. The four models that are measured are Kimi K3, DeepSeek-V4.1-Flash, GLM-5.3-Flash and Kimi-Linear. The numbers live in separate documents rather than in the README, with the full design and measurements in ENGINE.md and EFFICIENCY.md, the on-disk layout in FORMAT.md, Kimi Delta Attention in KDA.md, the DeepSeek cache curve in DS41.md, and two numbered entries in LEARNED.md. The repository also carries example programs split by capability, api_plan.c, api_text.c and api_vision.c, one chat JSON per model, and a serve directory that the README never mentions.
Editorial conclusion
Read this engine if your workload tolerates token rates in the single digits and you have fast internal NVMe, because the design makes storage bandwidth the only thing that matters and the README is honest about where that bandwidth goes. Do not adopt it for interactive use on Kimi K3, where one run reads 4.6 TB and takes ten minutes, and do not assume the new name is complete, because the binary, the warning prefix and the shard environment variable are all still WASTE. Before you tune anything, leave `num_experts_per_token` at the shipped value of 16 and leave the cache at or below its default, since the four-row table shows throughput falling by a factor of eight while the hit rate climbs. And check the release titles against your model, because the newest tag names a Qwen variant that appears in none of the measured rows.
Frequently asked questions
What is warp, the C inference engine published as sqliteai/warp?
An embeddable inference engine written in C with no third-party runtime dependencies. It keeps the model trunk in memory, streams selected mixture-of-experts weights directly from disk, and uses the remaining RAM as a bounded expert cache. It was formerly named WASTE.
How fast does warp run Kimi K3 on a laptop?
On a 64 GB MacBook Pro with an M5 Pro and the container on the internal SSD, the table reports 0.45 to 0.62 tok/s over 64 tokens, a 982 GB container and a minimum of 29.19 GB RAM. There is no 200-token figure because one run takes ten minutes and reads 4.6 TB.
Why does warp get slower when the expert cache is enlarged?
Measured hit rate rises from 29.1% at a 3.32 GB cache to 41.3% at 29.32 GB, but decode peaks at 0.63 tok/s with a 17.32 GB cache and then falls to 0.07 to 0.09 tok/s in the two larger configurations. The README attributes this to a cache hit turning into a page fault.
Can warp spread expert weights across several drives?
Yes, since 0.7.2. Setting `WASTE_BANK_SHARDS=/mnt/a,/mnt/b` reads expert e from shard e modulo N, so the experts one token routes to land on different devices. The README names `tools/split_banks.py` as the tool that writes the shards.
What does num_experts_per_token change in warp?
It sets how many experts are routed per token, and Kimi K3 ships at 16. Measured on the test machine, 8 experts per token decodes at 0.89 tok/s with a KL of 0.037 from top-16, while 4 reaches 1.06 tok/s but stops following the prompt within a few tokens, so the default stays at 16.
Official sources
Where developers are discussing it
Posts on Hacker News, dev.to and Lobsters that link to this repository, found by our scan of those communities. 0 in the last 30 days; newest first.
- Run DeepSeek v4.1 Flash on your MacHacker News · Sep 16, 2026 · 2 points · 1 comments
- Show HN: Warp – Run DeepSeek v4.1 Flash with 5 GB of RAM at 3.77 tok/sHacker News · Sep 15, 2026 · 16 points · 6 comments
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/sqliteai-warp)Star history chart for your README
[](https://hysenlabs.com/projects/sqliteai-warp)Was this useful?