kimi-k3-mlx: a faithful MLX port of a model nothing can hold

The deployment warning is the first thing the README says

The opening note is blunt: nothing in this repo runs on a single Mac. The smallest deployable tier is around 870 GB, measured against a 512 GB memory ceiling on the largest Apple Silicon machine that exists. Anyone planning around this project should read that section before sizing anything.

The weight explains the arithmetic. K3 is 2.78 trillion parameters total with 104 billion active, and the bf16 checkpoint is 5.6 TB. Routed experts alone account for 97.94 percent of parameters at 2722.7 billion, so almost the entire footprint is the part of the model that is sparse.

The accounting is checked against reality: a predicted size of 1.561 TB matches an actual 1.561 TB. That agreement is what validates the parameter budget table, which breaks the remainder down as KDA attention across 69 layers at 30.6 B, shared experts at 12.2 B, MLA attention across 24 layers at 5.6 B, latent-MoE up and down projections at 4.7 B, embeddings plus the language model head at 2.4 B, and everything else at 1.8 B, for a total of 2780.0 B.

The converter streams because mlx_lm convert materialises the whole model

`scripts/convert.py` exists as a streaming converter for a specific reason: `mlx_lm convert` cannot be used here because it materialises the entire model, and K3 in bf16 is 5.6 TB. Streaming is not an optimisation in this case, it is the only workable path.

The storage format in the source checkpoint is itself unusual. Routed experts ship as MXFP4, meaning a `weight_packed` unsigned 8-bit tensor plus an e8m0 `weight_scale` with group size 32, while everything else stays bf16. A converter therefore has to handle two different numeric formats across one model, with the 4-bit path applied to the 97.94 percent that matters most for size.

The repository has no tagged releases, so there is no versioned artifact to install. What exists is source: `kimi_k3.py` for the text tower, `kimi_k3_vision.py` for the vision tower and projector, the `kimi_k3_vl/` package that joins them, plus `reference/`, `scripts/`, and `tests/` directories.

Stable LatentMoE runs routed experts in 3584 dimensions, not 7168

The mixture-of-Experts layer is not the DeepSeek-style sparse MoE that `mlx-lm` already implements for Kimi-Linear-48B-A3B. Routed experts operate in a 3584-dimensional latent rather than the 7168-dimensional residual stream, with the path going `down_proj` into the experts, then `RMSNorm`, then `up_proj`. Shared experts do not get this treatment and stay in the wider space.

The routing itself is conventional in outline: 896 routed experts with top-16 selection, `moe_intermediate_size` of 3072, a sigmoid router with an auxiliary-loss-free `e_score_correction_bias`, and two shared experts applied on every token. Layer 0 is the only dense layer, configured through `first_k_dense_replace=1` with an `intermediate_size` of 33792.

One structural detail worth flagging: the routed experts are sourced in MXFP4 while the rest of the model is bf16, so any comparison against the reference implementation has to account for that format boundary rather than treating the port as a uniform bf16 translation.

The A_log shape mismatch is a silent-garbage bug the tests assert

This is the sharpest of the five porting notes, and the one that costs the most time when missed. The reference `modeling_kimi_linear.py` allocates `A_log` with shape `[num_heads]`, which is `[96]`. Every shipped shard instead has `[128]`, which is `head_dim`, while `b_proj` is `[96, H]` and `dt_bias` is `[12288]`.

The reason that matters is the broadcast. The `fla` kernel broadcasts `A_log` against a tensor `g` of shape `(B,T,96,128)`, so a length-128 vector can only align with the trailing axis. The decay is therefore per-channel and shared across heads, which is not what the reference init code implies. The shapes are authoritative here; the reference initialisation is stale with respect to the released weights.

Assuming the Kimi-Linear layout produces silent garbage rather than a shape error, which is the worst failure mode for a port. `tests/test_kimi_k3.py` asserts the correct layout. The other four divergence markers, `[K3-1]` through `[K3-5]`, cover SiTU-GLU activation computed in fp32 because the tanh saturation points sit far enough out that bf16 rounding becomes visible, the AttnRes residual stack, the LatentMoE latent, and q-LoRA with an output-gated MLA.

Attention is 93 layers split between KDA and a NoPE gated MLA

The layer budget is 93 total: 69 Kimi Delta Attention layers and 24 gated Multi-head Latent Attention layers, with 96 heads and `head_dim` of 128. The two attention types differ enough that they are not variants of one another.

KDA applies a short convolution with kernel size 4 on the query, key, and value projections, uses a low-rank forget gate through `f_a` and `f_b`, and a full-rank output gate, with `gate_lower_bound` set to -5.0. MLA uses q-LoRA at rank 1536 and kv-LoRA at rank 512, applies NoPE so there is no rotary embedding at all, and gates the output with a sigmoid `g_proj`.

AttnRes is the third mechanism layered on top: every 12 layers the residual is pushed onto a stack, and each sub-layer input becomes a softmax mixture over that stack, scored by a rank-1 direction of shape `[1, 7168]`. The activation function is SiTU-GLU, computed as `β·tanh(g/β)·σ(g)` multiplied by `β_lin·tanh(u/β_lin)` with β of 4 and β_lin of 25. Vocabulary is 163,840 tokens on tiktoken, with `tie_word_embeddings` set to false.

The vision tower diverges from Kimi-VL in six named places

`kimi_k3_vision.py` holds a 3D, video-capable MoonViT with 27 layers and 447 M parameters, plus the patchmergerv2 projector. mlx-vlm's existing `kimi_vl/vision.py` is the nearest code it builds on, sharing the complex64 2D-rope approach, and the differences are enumerated rather than left for you to discover.

The six markers cover position handling and shape conventions. `[K3-V1]` uses 3D `(t,h,w)` grids with a 1D sin-cos time embedding on top of the learnable 2D grid and rope frequencies repeated per frame. `[K3-V2]` uses bilinear position-embedding interpolation where Kimi-VL used bicubic, written against torch's `align_corners=False` half-pixel convention because mlx-vlm ships only bicubic and nearest kernels. `[K3-V3]` is the non-square projection: `qkv_hidden_size` of 1536 against hidden 1024 means `wo` is `[1024, 1536]`. Then `[K3-V4]` switches to RMSNorm, and `[K3-V5]` drops biases entirely.

`[K3-V6]` is the merge: `sd2_tpool` mean-pools over frames, does a 2x2 spatial merge, then a two-layer MLP projector with exact erf GELU and a post-RMSNorm. Note that the ViT blocks use tanh-approximate GELU while the projector uses exact GELU. The reference uses both on purpose, and they are different functions.

A norm epsilon difference of five orders of magnitude, chosen deliberately

One judgement call is called out because it changes numerical behaviour. The reference builds `nn.RMSNorm(dim)` with no `eps` argument, so torch falls back to `finfo(dtype).eps`, which is 1.19e-7 in fp32 but 7.8e-3 in bf16. Those are materially different normalizers, not a rounding detail.

The port sets `VIT_NORM_EPS` to the fp32 value, matching an upcast implementation. Separately, `projector_ln_eps` at 1e-5 is given explicitly by the config and applies only to the projector's post-norm.

Unlike the text tower, the vision tower is small enough to validate against the reference for real. `tests/test_vision_parity.py` runs Moonshot's own torch code on CPU against the MLX port at full K3 dimensions across all 27 layers, and the two agree to 1.5e-6 relative error end to end, which is float32 round-off. That parity test is the strongest evidence in the repository that the port is faithful in the part it can actually be checked.

The vision-language merge has to expand, not scatter

`kimi_k3_vl/` follows the mlx-vlm package shape with `Model` as the glue, `LanguageModel`, `VisionModel`, and the config plumbing. The piece that carries the weight is the merge, and it is expanding rather than scattering.

K3's processor rewrites each image placeholder in the raw text into this sequence:

<|media_begin|>image {W}x{H}<|media_content|><|media_pad|><|media_end|>

So the tokenized prompt carries exactly one `<|media_pad|>` token, id 163605, per image, and that single token has to expand into the image's entire feature block. Most LLaVA-style models do not work this way, and neither does mlx-vlm's Kimi-VL glue, where the processor has already emitted one placeholder per image token and merging is a same-length scatter.

Applying that scatter here would keep one token per image and silently discard the rest. There is no crash and no error: the model writes fluent prose while being effectively blind to most of the picture. `tests/test_vl_wrapper.py` asserts the expansion directly against hand-built expected sequences, and rows are assembled by concatenating the spans between placeholders, which is linear in the number of images and easier to check than the reference's index arithmetic. With a batch larger than one, rows are left-padded.

Editorial conclusion

This repository is worth reading as a porting reference rather than as something to install. It fits someone implementing K3 elsewhere, or someone who needs the precise set of divergences between K3 and Kimi-Linear spelled out with test coverage. Verify three things before planning any deployment: that you have multi-terabyte storage and roughly 870 GB of memory for even the smallest tier, that your hardware is not a single Mac, since the project states the largest Apple Silicon machine has a 512 GB ceiling, and that you read the streaming converter rather than reaching for `mlx_lm convert`, which materialises the whole 5.6 TB model. The last push on the repository is dated 2026-08-01, there are no tagged releases, and no license is declared.

Frequently asked questions

Official sources

1. Issues
2. PipeNetwork/kimi-k3-mlx on GitHub
3. README