Running a 2.78-trillion-parameter model on a laptop: inside kimi-k3-in-c

A model that should not fit, on a machine that cannot hold it

Kimi K3 is a 2.78-trillion-parameter mixture-of-experts model. Its checkpoint on disk is 1.56 TB. A single GPU has 80 GB of memory at best. The author of kimi-k3-in-c decided that the model should run on an ordinary laptop with 8 GB of RAM, and wrote a C99 inference engine to prove it. The README reports a peak resident set size of 8.24 GB during a run that generated 8 tokens at 26.5 seconds per token on a laptop. The same model, the same prompt, and byte-identical output at every memory budget between 8 GB and 224 GB. The project is for people who want to run a specific huge model on hardware they already own, not for people who want a general-purpose LLM runtime.

The four reductions that make it possible

The README describes four decisions about where bytes live. The first is that the model already ships in a compressed form: the 1.45 TB of routed experts are stored in packed 4-bit format, which the README calls half a byte per weight. The second is KDA attention, an attention variant with a memory footprint that never grows with sequence length. The third is MLA, multi-head latent attention, which replaces 96 attention heads with one latent vector. The fourth is the streaming architecture itself: the dense trunk of the model stays resident in memory to a configurable depth, and the rest streams from disk on every token. The routed experts are never fully resident; they are multiplied directly from their packed form. These reductions together let a 1.56 TB checkpoint run in 8 GB of RAM, at the cost of reading from disk continuously when the model does not fit in memory.

How streaming and memory presets change the clock

The README gives a table of memory budgets and their effect on time per token. On an ordinary laptop with 8 GB of RAM, the whole model streams off the disk on every step, and each token takes 26.5 seconds. With 32 GB, some of the model sits in memory and the time drops to 24.2 seconds. At 64 GB it is 19.8 seconds. Only at 128 GB or more does the model fit entirely in memory, and then the disk wait disappears, giving 5.6 seconds per token. The README notes that a machine with 124 cores and a fast NVMe drive still reads from disk each step for the first three rows, so a slower drive makes those rows slower. The output is byte-identical regardless of memory size. More memory only buys speed, never a different answer.

Building and running: commands from the README

The README shows a quick start that clones, builds, and verifies in about a minute with no model. The full setup requires a checkpoint and a trunk file. The run command is: ./bin/k3 ~/k3model --trunk ~/k3trunk --preset laptop --tok ~/k3model --prompt "The capital of France is" --gen 8 --incremental. The --preset option selects a memory profile; the README mentions presets named laptop and server. The server preset, run on a machine with 128 GB or more, produced 28 tokens at 10.69 seconds per token and a peak RSS of 127.92 GB. The laptop preset produced 8 tokens at 32.69 seconds per token and 8.24 GB peak RSS. There are also prompt options, memory options, generation options, diagnostic options, exit codes, and environment variables documented in the README, though the cleaned text does not list them all.

The config reader that refuses to guess

One of the documented invariants is a config reader that refuses to guess. That means the engine will not silently assume a wrong model dimension or a wrong number of experts. If the checkpoint does not match the expected format, the program should fail rather than produce garbage. This is a deliberate design choice, and it has a consequence: the engine is tied to the exact checkpoint layout of Kimi K3. It is not a general model loader. The tokenizer is also handled byte for byte, which matters for reproducing the exact same output across machines. The README emphasizes that the answer is byte-identical from the smallest machine to the largest, which implies a strict floating-point contract in the kernels as well. The section on kernels with a floating point contract suggests that arithmetic is done in a way that is deterministic regardless of memory layout or disk speed.

What the engine does not do

The README is clear that this is a base model, not a chat model. The example prompt "The capital of France is" generates " Paris." and then a continuation, not a conversational reply. There is no chat template. That means you cannot use this as a drop-in assistant; you would need to build your own prompting and post-processing around it. The engine is also single-model: it is built for Kimi K3 specifically, not for arbitrary transformer checkpoints. The README does not mention support for other model families. The speed is a real limitation: 26 seconds per token on a laptop is not interactive. Generating a 100-token answer would take over 40 minutes on 8 GB of RAM. This is a tool for batch generation or for demonstrating that the model can run at all, not for real-time use.

Alternatives and what they do differently

The obvious alternative is llama.cpp, which also runs LLMs on CPU with quantization and no GPU. The difference in approach is that llama.cpp is a general-purpose inference engine that supports many model architectures and uses GGUF quantization, typically 4-bit or lower, but it does not attempt to run a 2.78T-parameter model in 8 GB of RAM. llama.cpp would require the model to be converted to its format, and a 2.78T model would still be far too large for an 8 GB machine because it does not stream the entire model from disk in the same way. Another alternative is to run the model on a GPU or a cluster with more memory, which is the conventional route. kimi-k3-in-c is unique in its focus on one model and on making the disk the primary storage for weights, with a resident trunk that is configurable. That is a fundamentally different trade-off: it accepts extreme slowness in exchange for an extreme memory footprint.

Maintenance, license, and what to verify

The project is released under Apache-2.0, which permits commercial use, modification, and redistribution with attribution. The repository has a CHANGELOG.md and recent releases: v0.1.0 on 2026-08-02 and v1.0.0 on 2026-08-07. The last push was 2026-08-26, so it is actively maintained as of that date. The README mentions CI and a Makefile, so building is likely straightforward on Linux x86-64, which is the stated platform. The engine is C99 and zero-dependency, which helps portability, but it uses AVX2 instructions, so you need a CPU that supports them. There is no BLAS, no framework, and no GPU. The main maintenance cost is that this is a single-model project; if Kimi K3 updates its checkpoint format, the engine will need changes. Before adopting it, verify that you can obtain the model checkpoint legally and that your disk has enough free space for 1.56 TB, since the model streams from disk on every token when you have less than 128 GB of RAM.

Editorial conclusion

Adopt kimi-k3-in-c if you need to run a specific 2.78T-parameter base model on hardware without a GPU and can accept 10 to 30 seconds per token, or if you want to study how extreme quantization and memory-mapped streaming can be done in portable C. Skip it if you need chat templating, multi-model support, or interactive speeds; the repository is a single-model proof of concept. Before relying on it, verify that your CPU supports the required AVX2 instructions, that your disk can stream 1.56 TB repeatedly if you have less than 128 GB of RAM, and that you have the exact checkpoint format the code expects, since the config reader refuses to guess.

Official sources

1. FareedKhan-dev/kimi-k3-in-c on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases