REAP prunes mixture-of-experts models instead of merging them

The argument against merging experts

REAP comes from the paper REAP the Experts: Why Pruning Prevails for One-Shot MoE compression, and the first of the three stated contributions is a theoretical claim about why merging is the wrong tool. The authors prove that expert merging introduces irreducible error because of the loss of the router's independent, input-dependent modulation of the expert outputs, which results in functional subspace collapse and substantially reduces the functional output space of the compressed sparse layer. In a pruned model, by contrast, the router keeps independent control over the remaining experts, because each surviving expert is still the expert that was trained. The practical implication is that merging compresses by producing a new function, while pruning compresses by deleting part of an existing one. The second contribution is REAP itself, a saliency criterion that selects experts contributing least to the layer output by considering both the router gate values and the average activation norm of the experts.

What the results claim and on which models

The headline claim is that across a diverse set of sparse mixture-of-experts models ranging from 20B to 1T parameters, REAP consistently outperforms merging and other pruning methods on generative benchmarks, especially at 50% compression. The most striking result is near-lossless compression on code generation and tool-calling tasks with Qwen3-Coder-480B and Kimi-K2, even after pruning 50 percent of experts. The evaluation surface named in the README is broad, covering agentic tasks, non-agentic coding, tool-use tasks, mathematical reasoning, creative writing and multiple choice benchmarks, and the figures compare compressed GLM-4.5 Air and Qwen3-30B-A3B against a variety of compression methods, with a second figure plotting accuracy against parameters across all models and methods. The released checkpoints on Hugging Face cover pruned versions of GLM4.6, GLM4.5-Air, Qwen3-Coder-480B, Qwen3-Coder-30B, MiniMax-M2, Kimi-Linear and DeepSeek-V3.2, so you can test the method without running the pruning pipeline yourself.

The calibration mix, and why it is published

To reproduce the Hugging Face checkpoints, the project releases the calibration data mix used for agentic reasoner models, and the four components are named with their sample counts. The general coding dataset is theblackcat102/evol-codealpaca-v1 at 4096 samples. The reasoning dataset is open-r1/Mixture-of-Thoughts at 12288 samples, equal parts from its code, math and science subsets. The single-turn tool calling dataset is Salesforce/xlam-function-calling-60k at 4096 samples, and the agentic coding and multi-turn tool calling dataset is SWE-bench/SWE-smith-trajectories at 4096 samples from its tool split. The full set is 24576 samples at a maximum sequence length of 16384 tokens, and the composite dataset is given on the command line:

bash experiments/pruning-cli.sh 0 Qwen/Qwen3-30B-A3B reap 42 0.25 "theblackcat102/evol-codealpaca-v1:4096,Salesforce/xlam-function-calling-60k:4096,open-r1/Mixture-of-Thoughts[code]:4096,open-r1/Mixture-of-Thoughts[math]:4096,open-r1/Mixture-of-Thoughts[science]:4096,SWE-bench/SWE-smith-trajectories(tool):4096" true true false false false

The dataset subsets are specified in square brackets and the split name in curly brackets, which is a detail worth noting if you write your own invocation. Publishing the mix matters because calibration data is usually the least reproducible part of a compression pipeline, and here it determines which behaviours survive the compression.

Installing it, two ways

The virtual environment route asks for uv to be installed and then runs the build script:

bash scripts/build.sh

The Docker route is a compose file with a GPU service:

docker compose up --build -d
docker compose exec app bash

The compose file is worth reading before you use it, because the resource settings are aggressive. It builds from a Dockerfile in the repository, mounts the working directory at /app read-write, mounts the Hugging Face cache from ~/.cache/huggingface, and keeps a named volume for the virtual environment at /app/.venv. It reserves all NVIDIA GPUs, sets shm_size to 16g and mem_limit to 1t, and runs with network_mode host so the container reaches the vLLM server the evaluation starts. The base image is nvidia/cuda:12.2.2-devel-ubuntu22.04 with Python 3.12 added from the deadsnakes PPA, uv installed system-wide, and a non-root dev user created with your own user and group IDs passed as build arguments. The README notes that if you use a different cache directory you should update the mount path to avoid excessive container storage.

Configuration and adding a model

Configuration starts by copying .env.template to .env and filling the empty fields. The evaluation setup for WildBench is more fiddly and the README is precise about it. You copy config/wildbench_prod_env_XXXX.example, where the XXXX is the port used to launch vLLM, which defaults to 800X where X is the rank of the first GPU used to run the evaluation script, so wildbench_prod_env_8000 when running with CUDA_VISIBLE_DEVICES=0,1,2,3. In the copied directory you update credentials.conf with your OpenAI API key and substitute base_url:XXXX in model_deployments.yaml with the port you selected, for example turning http://0.0.0.0:XXXX/v1/ into http://0.0.0.0:8000/v1/ for the example above. Adding a new model means adding attribute names to MODEL_ATTRS in src/reap/model_util.py, where each entry is keyed by the class name of the model, model.__class__.__name__, and the values include moe_block, the attribute name of the sparse mixture-of-experts submodule in the decoder.

The dependency set tells you what this is for

The pyproject is a research evaluation stack, and the pins are strict. Python 3.12 or newer is required, torch is pinned to 2.7.1 and transformers to 4.55.0, with a commented line noting that transformers 4.53.3 is the latest version working for ernie. vllm is pinned to 0.10.0, which is the serving stack the evaluation depends on, and the project registers itself as a vLLM general plugin with an entry point named llama_4_text_only pointing at reap.model_util:register_llama_with_vllm. On top of that sit accelerate, datasets, lm-eval with the vllm and api extras, evalplus with vllm, deepspeed, trl, wandb, matplotlib, seaborn, umap-learn, crfm-helm and evalscope, plus livecodebench. Four of those are pulled from vendored source in third-party/ as editable local paths rather than from an index, with evalplus, livecodebench, helm and evalscope, which is what lets the project pin evaluation behaviour it cares about. The dev group adds pytest, pandas, plotly, kaleido and Google API client libraries.

Where REAP is the wrong choice

Three cases. The first is a dense model or a model whose expert structure the pipeline does not know, because the whole method is defined in terms of a sparse mixture-of-experts block, and registering an unfamiliar architecture is a code change rather than a config change. The second is a workload that is neither code generation nor tool calling, since the near-lossless results are specific to those tasks and the general benchmark numbers show a mean accuracy decrease rather than a loss. The third is a team that needs a reproducible, versioned artifact, because the repository publishes no GitHub releases, the packaging version is 0.1.0, and the last push was on 2026-04-17, so the code moves faster than its tags. Against merging, the difference in approach is exactly the paper's claim: merging produces a new blended expert set and loses the router's per-input modulation, while pruning keeps the surviving experts intact and lets the router keep routing to them, which is why the pruning result degrades more gracefully as compression increases.

Editorial conclusion

REAP is a good fit for a team serving a large mixture-of-experts model where the memory cost of the expert weights is the problem and the workload is code generation or tool calling, since the near-lossless results the authors report for those tasks are exactly where the method was aimed. It is a poor fit for a model family the calibration pipeline does not support, and it is a poor fit for a team that cannot run multi-GPU calibration and evaluation, because the tooling assumes vLLM serving, an evaluation stack and a Hugging Face cache. Check three things before you plan a run: that your model class is registered in MODEL_ATTRS in src/reap/model_util.py, since adding a model means telling the code which attribute holds the sparse block, that you can calibrate on a 24576-sample mix at a maximum sequence length of 16384 tokens, and that you read the router normalisation change, because the top-k router logits are now correctly renormalized to sum to 1, which the authors say should give a modest improvement for most models and datasets.

Frequently asked questions

Official sources

1. CerebrasResearch/reap on GitHub
2. Issues
3. License: Apache-2.0
4. README