Deltafin runs all 2.8T parameters of Kimi K3 on one machine

What Deltafin actually is, and who it is for

Deltafin is a single native binary that runs Kimi K3, the 2.8-trillion-parameter mixture-of-experts model from Moonshot, without pruning. The README is explicit about the constraint it refuses to break: all 16 routed experts stay in place, and "K3 itself decides every token." Nothing about the model is reduced to make it fit.

The intended audience is narrow. The README frames the project as an experiment rather than a product, aimed at people who want to see how far consumer hardware can be pushed. Kimi K3 is designed for infrastructure on the scale of 16 nodes and roughly 4.8 TB of aggregate VRAM. Deltafin targets the same weights on a home machine. The README puts the comparison at a $15,000 home setup against a $2,000,000 infrastructure estimate.

If you need a fast local assistant, this is not it. If you want to run the actual model, unmodified, and are willing to wait, the project is built precisely for that.

How the K3 target and the draft models divide the work

The architecture separates two roles. K3 is the target: it validates every token before it reaches the user. Small draft models are allowed to guess ahead, and the README credits that guessing with much of the speed. The guesses are checked, and rejected guesses do not change the output.

Two draft components exist. DSpark, from Inferact's Kimi-K3-DSpark, is installed by the normal setup and takes 6.635 GiB on disk and approximately 4.49 GiB when admitted at runtime. Deltafin avoids materializing DSpark's redundant copy of K3's embedding. Chat and server requests use DSpark automatically when it is beneficial. If it fails, if there is insufficient headroom, or if the live economics look bad, full K3 runs by itself.

Qwen is a separate add-on for raw text continuation. The README states it speeds up raw completion only: the small models guess what comes next, K3 checks the guess, and the output is identical. The README reports one measured 17-token completion running 2.7 times faster with the same output IDs. Qwen adds 4.337 GiB on disk and does not improve chat speed, which is why it is optional.

The workspace layout reflects this split. native/deltafin is the default member, with separate crates for bootstrap, a direct curl-sys replacement, native build support, and xtask. The release profile sets codegen-units to 1, lto to "fat", panic to "abort", and strips symbols, with a comment noting that the binary is latency-sensitive and that unwinding across C, ATen, Metal, or CUDA boundaries is undesirable.

Installing Deltafin and running a first prompt

The README gives a three-step install: clone, build with Cargo, then run setup. Building requires Rust 1.85 or newer, which Cargo.toml sets as the workspace rust-version.

git clone https://github.com/gavamedia/deltafin.git
cd deltafin
cargo build --locked --release
./target/release/deltafin setup --full

The third command downloads the full 1.7 TB K3 model to disk, which the README calls optional but the fastest option. If disk space is short, substitute the streaming variant. It installs the resident model and fetches exact experts on demand, starting at 215 GB. The README warns it runs far more slowly at first when routes have no local cache, and that a cache builds up over time.

./target/release/deltafin setup --stream

DSpark is included in the normal setup. Qwen is not. To add it during a fresh install, pass the flag; to add it later, run the dedicated command.

./target/release/deltafin setup --full --include-qwen
./target/release/deltafin setup-qwen

After that, the README points at two entry points: an OpenAI-compatible API server for local chat and coding agents, and a command-line prompt path. The README shows the CLI form as `deltafin run --prompt ...`, and notes that Qwen accelerates `/v1/completions` traffic. The README does not document server startup flags, ports, or environment variables, so check the repository files before assuming defaults.

The speed ceiling is the real specification

The README publishes its own benchmark on an M1 Max laptop: 0.2901 token/s, or 3.447 seconds per token. That is the headline number, and it should be read as a hard constraint rather than a footnote. A 200-token answer at that rate takes over eleven minutes.

The historical series shows how much work each gain required. On July 27, 2026 the figure was 0.0141 token/s. Two days later it was 0.1311. By July 30 it reached 0.2660, and on August 2 it was 0.2847. The current 0.2901 is described as 1.9 percent higher throughput than the previous update. Most of the improvement happened early; the recent steps are small.

This matters for adoption decisions. The project's value is fidelity, not latency. A quantized build that re-encodes the expert bank to roughly 3 bits will fit and respond sooner, but the README's position is that those weights are no longer the ones Moonshot released, and that nobody has measured what the compromises cost. That is a fair statement of the trade, and it cuts both ways: Deltafin has not measured the cost of its own slowness either, in the sense that 3.447 s/token limits what you can practically do with the output.

Upgrading, and where the upgrade path stops

Upgrades run from the Deltafin folder with a single command. Models, converted weights, and caches are left alone, and the README states it never re-runs setup or re-downloads K3.

./target/release/deltafin upgrade

The command needs a clean, non-diverged branch. It remembers how the binary was built, so a CUDA build stays a CUDA build instead of falling back to CPU. Anything unexpected stops the upgrade. One limitation is stated plainly: `upgrade` ignores build environment variables and reuses whatever the binary was already built with. To switch configuration, such as CPU to CUDA or a moved LibTorch tree, you must run `cargo build --locked --release` yourself once with the new variables set.

For anyone arriving from the older Python version, the README requires an inspection step first. Run `git status --short` and continue only when it returns nothing. If it lists files, preserve or commit that work yourself rather than letting an upgrade procedure guess. Existing model data stays in the same repository-root directories.

Why llama.cpp and quantized K3 builds are a different experiment

The obvious alternative for local model serving is llama.cpp, which covers a wide range of quantized formats and runs comfortably on ordinary hardware. The difference is not just speed. Quantization changes the weights: a 4-bit or 3-bit representation is a different numerical object from the one the model author published, and llama.cpp's design assumes you accept that trade in exchange for fitting on smaller machines.

Deltafin rejects that trade by construction. Every expert byte is kept exactly as shipped. The README describes other projects as running full K3 faster by re-encoding the expert bank to roughly 3 bits, and characterizes that as clever engineering toward a different goal: the smallest K3 that fits and is close enough. The two projects are not competing on the same axis. One optimizes for usable throughput on modest hardware; the other optimizes for fidelity and treats throughput as whatever remains.

A second alternative is simply renting capacity. Kimi K3 is designed for 16-node infrastructure with roughly 4.8 TB of aggregate VRAM. If your goal is to use K3 rather than to study it, remote inference avoids the 1.7 TB download and the 3.447 s/token wait entirely. Deltafin's case rests on locality and on running the unmodified weights, not on cost per token.

Licence status and what to verify before you commit

There is a discrepancy worth resolving before adoption. Cargo.toml declares `license = "MIT"` for the workspace, and the README's badge row also shows MIT. The repository metadata reports NOASSERTION, which typically means an automated licence detector could not map the LICENSE file to a known identifier. The LICENSE file exists at the repository root, so the answer is available, but it requires reading the file rather than trusting either label. This is a factual gap, not a legal opinion, and anyone shipping Deltafin inside a product should read the LICENSE text directly.

Separately, the model weights are a different artefact from the code. The README links to the Kimi K3 model card on Hugging Face, and the streaming setup downloads experts on demand. Whatever terms apply to K3 apply independently of the MIT claim on the Rust workspace.

On cost, the README is concrete about disk: 1.7 TB for the full download, 215 GB to start with streaming, 6.635 GiB for DSpark, and 4.337 GiB for Qwen. The README does not quantify the time required for the full download, nor the cache growth rate under streaming, so budget for both before starting.

Editorial conclusion

Adopt Deltafin if you want to study or serve the exact Kimi K3 weights Moonshot shipped and can accept roughly 0.29 token/s on an M1 Max, or if you have the budget for a high-memory workstation and want the full 16-expert routing with a drop-in /v1 endpoint. Do not adopt it if you need interactive latency, since 3.447 s/token is not a chat experience, and do not compare it against quantized K3 builds without accepting that those run different weights. Before committing, check docs/REQUIREMENTS.md against your accelerator, decide between setup --full and setup --stream based on disk space, and confirm the licence question: Cargo.toml declares MIT while the repository metadata reports NOASSERTION.

Frequently asked questions

Official sources

1. gavamedia/deltafin on GitHub
2. Issues
3. README