tinyrouter: a 10K-parameter head that decides which open-source model answers your question

Two decisions per question: which model, and in what role

tinyrouter is a coordinator, not a solver. For each question it makes two choices: which of three open-source language models should answer, and what role that model should play, either Thinker, Worker or Verifier. The coordinator itself is deliberately small. A frozen 0.6B encoder reads the question into a single vector, and a head of roughly 10,000 parameters turns that vector into the routing decision.

The method follows TRINITY, given in the README as Xu et al., ICLR 2026, arXiv:2512.04695, rebuilt from scratch with an all open-source model pool served through Fireworks AI. The pool assigns deepseek-v4-pro to knowledge, glm-5p2 to math and kimi-k2p6 to general use.

How a routing decision is made and scored

The loop runs in five steps. The frozen 0.6B encoder turns the question into one 1024-dimension vector. The 10K head reads that vector and picks a model and a role. The chosen model answers in that role and its output is appended to the transcript. The first three steps repeat for up to five turns, and a Verifier turn can accept and stop early. The final answer is graded right or wrong, and that reward drives training.

Training is separable CMA-ES, a derivative-free evolution strategy. The README describes it as breeding thousands of candidate heads, keeping the ones that route best, and repeating. Correctness comes from an automatic grader, exact-match for math and letter-match for MMLU, so the reward is binary.

The hardware split is worth noting if you plan to reproduce it. The 0.6B encoder and the evolution loop run on a single NVIDIA H200, while the three models are called over HTTP.

The results, and the honest reading of them

Evaluation used 120 held-out questions per task, with every single-model baseline averaged over three runs to remove run-to-run noise.

On math, glm-5p2 scored 0.794 as the best single model, TinyRouter 0.792, random routing 0.792, deepseek-v4-pro 0.747 and kimi-k2p6 0.742. On MMLU, TinyRouter scored 0.925, deepseek-v4-pro 0.922 as the best single, random routing 0.875, glm-5p2 0.783 and kimi-k2p6 0.539. Averaged across both tasks, TinyRouter reaches 0.858 against 0.835 for the best single model and 0.833 for random routing.

The README reads those numbers correctly, which is rarer than it should be. The win is across tasks, not within one. On MMLU the three models range from 0.539 to 0.922, so there is something to route around and the router beats random by five points. On math all three sit near 0.79, so there is nothing to route around and the router merely ties the best single model and random routing.

The oracle ceiling, and the headroom the router misses

Rather than stopping at the headline, the project built an oracle-ceiling diagnostic to ask whether the pool leaves room for routing to help at all, and used it to decide where to spend effort.

The answer overturned the easy reading of math as a no-benefit case. On math500 the best single model scores 0.808 and a perfect router would score 0.856, giving real headroom of 4.9 points with a 95 percent interval of 0.005 to 0.085, a verdict the README labels ROUTER_BOUND. On MMLU the best single scores 0.939 and the perfect router is at least 0.939, so headroom is 2.5 points with an interval starting at zero, which the README calls inconclusive and near-ceiling.

So there are about 4.9 points of real, achievable headroom on math, and the trained router captures none of it. On that task the limit is the router, not the pool.

Warm-start, shaped fitness, and what the experiment can prove

The diagnostic pointed effort at math, so two upgrades were tried: warm-starting the head with a supervised fit against per-question, per-model correctness labels instead of starting evolution from a blank head, and shaping the training reward with a format bonus, a turn penalty and variance reweighting while keeping the evaluation purely right or wrong.

On the held-out 120 math questions the best single model scored 0.817, the upgraded TinyRouter 0.808, the prior router on the same test 0.792, and random routing 0.733.

The README refuses to call that a win, and the reason is sound. The evaluation samples each model once per question, so sampling noise is large: random routing alone swung from 0.792 to 0.733 between runs with nothing changed, and a swing that size swamps a 1.6-point router delta. No clean control was run, meaning blank-init, pure-binary, same settings, so there is no causal claim that either upgrade moved the number. The result is still below the best single model and below the 0.856 ceiling.

What it costs to reproduce

Costs are tracked from token ledgers at real Fireworks prices. Core replication plus rigorous eval came to 20.89 dollars, split across the three models at 6.56 for deepseek, 6.70 for glm and 7.64 for kimi. The oracle-ceiling diagnostic added about 14 dollars, and the warm-start and shaped-fitness experiment, covering label collection, retraining and evaluation, cost 27.22 dollars.

That total is the most useful number in the repository for anyone deciding whether to try this at all. The whole investigation, including the experiments that did not pay off, came to roughly 62 dollars, which is not true of most model-routing work.

Setting it up, and the state of the repository

Configuration is deliberately kept out of the tree. The .env.example tells you to copy it to a file outside the repo, fill it in, restrict it to mode 600, and source it before running:

cp .env.example ~/.config/trinity/secrets.env
source ~/.config/trinity/secrets.env

Three variables are expected: FIREWORKS_API_KEY for the three models, TRINITY_GPU_HOST for the remote GPU box, and TRINITY_GPU_INDEX for which physical GPU is allowed, with every remote command pinning CUDA_VISIBLE_DEVICES to it.

The package is named trinity-coordinator at version 0.0.1 and requires Python 3.10 or newer. Dependencies include torch 2.3 or newer, transformers 4.44 or newer, the cma package for the separable CMA-ES variant, httpx for the async Fireworks client and tenacity for retry and backoff. The last push was on 2026-07-06 and the repository declares no licence.

Editorial conclusion

Read tinyrouter if you want a small, fully costed replication study of model routing with the negative results left in, and take the oracle-ceiling diagnostic as the transferable idea. Do not adopt it as a routing component on the strength of the 0.858 average, because on math the router ties random routing and the diagnostic says the missing 4.9 points are the router fault. Run the diagnostic on your own pool first: if it shows no real headroom, no amount of training on the head will produce a win.

Frequently asked questions

Official sources

1. harrrshall/tinyrouter on GitHub
2. Issues
3. README