# decider: a model that answers typed questions without generating a word

> decider is a family of models fine-tuned from Qwen3.5 that take a state and a set of typed questions and return, from a single forward pass, a probability distribution for every question. The answer set is fixed by the caller, so there is no decoding and no parsing, and the family spans a 2B model that runs on a CPU to a 31B model served through vLLM.

**Mapika/decider** — A family of System One-style models fine-tuned from Qwen3.5, designed for one-pass typed decisions with calibrated probabilities.

- Repository: https://github.com/Mapika/decider
- Stars: 1,152 · Forks: 51
- Language: Python
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/mapika-decider

## The output space is defined by the caller, not the model

The opening paragraph of the README is the whole design in three sentences. A language model that does not generate text. It reads a state and a set of typed questions and returns, from one forward pass, a probability distribution for every question.

That changes what a model call is. There is no decoding step, no parsing step, and no output outside the options you defined. The caller supplies the answer set, and the model returns a distribution over exactly that set.

Three question types are defined. Choice is over 2 to 255 options. Score is over 2 to 10 described levels. Noul is the probability of yes.

The example in the README shows how compact this is.

```python
from decider.infer import Decider
d = Decider("Mapika/decider-2b")        # downloads the weights on first use; CUDA, else MPS, else CPU
d.decide("My card was charged twice.", [{"question": "Which team?", "options": ["billing", "technical", "sales"]}])
# [{"choice": "billing", "confidence": 0.77, "probs": {"billing": 0.77, "technical": 0.19, "sales": 0.04}}]
```

The response carries a chosen option, a confidence, and the full probability map. For a triage tool, a router, or a labelling pipeline, that is a better-shaped answer than a string you have to parse and defend against a downstream parser.

The trade is that the model cannot express anything outside the answer set. If the right answer is not among the options you supplied, the model has no way to say so, and the distribution will still be confident.

## A family sized by hardware, not by one flagship

The table titled which model to use is organised by what you have, which is the honest way to present a model family that spans four orders of magnitude in hardware.

With a smaller GPU, decider-4b at 8.4 GB in bf16, with a held-out accuracy of 0.784 on the regression set and a JevBench public hard tier of 0.649.

On a CPU or a laptop, the GGUF builds of the 2B and 4B models through llama.cpp, which are Q4_K_M files of 1.3 GB and 2.7 GB, or the 2B model in PyTorch. The README gives the timing directly: on 8 server CPU threads a request of 40 to 120 tokens takes 0.12 to 0.31 s with the 2B and 0.3 to 0.7 s with the 4B.

With a GPU and hard decisions, meaning long policy texts and multi-hop reasoning, decider-12b at 24 GB in bf16, with a JevBench public hard tier of 0.712, and built as Gemma-4-12B-it with a state-tracking LoRA merged in.

With 65 GB of GPU memory, or 19.6 GB with NVFP4 in vLLM, decider-35b-a3b, with a held-out accuracy of 0.810 and a JevBench public hard tier of 0.676.

With a Blackwell GPU at NVFP4, 31 GB, and vLLM, decider-31b, served by decider.serve_vllm, with a Decision Index 0.3 public suite score of 64.27 and a median of 39 ms.

And there is a chat variant, decider-chat-gemma4-31b, which is stock Gemma-4-31B-it with an option-count temperature, reported as #2 of 70 on the Decision Index with a score of 57.33 and an ECE of 0.047.

The two chat models are described as stock checkpoints plus a config, and the decider-12b v2 is a merged LoRA of rank 32. That pattern, a fine-tune expressed as a small merged adapter over a larger base, is what makes it practical to publish and maintain a family this wide.

## Option-count temperature, and why a fixed temperature is wrong

One configuration detail is worth pulling out of the changelog, because it is a design idea rather than a bug fix.

temperature_by_options in decider_config.json sets T(n) = max(min, a + b ln n) for a question with n options.

So the sampling temperature is a function of how many options the question has. That is the correct shape for the problem. A question with two options and a question with two hundred and fifty-five options are not the same classification task, and a fixed temperature treats them as if they were. More options means a flatter distribution is appropriate, and a logarithmic form gives you that without hand-tuning per question.

Both chat repositories use it, decider-chat-gemma4-31b and decider-chat-qwen3.6-27b, with the Qwen one using a single temperature instead.

The changelog also records a regression test for the feature. Replaying 300 stored index rows through decider.serve gives the stored answers on 1,412 of 1,412 questions for Gemma and 1,431 of 1,438 for Qwen. That is a specific and checkable claim about determinism: the same model, the same config and the same questions give the same answers, which is what you need if the model is going to be a component in a system that is tested.

Earlier entries in the changelog show the same pattern of narrow, verifiable fixes. The 12b v2 entry lists new temperatures for each type, Choice 1.5, Noul 0.05 and Score 1.0, and reports a fresh held-out yes/no set score alongside the older one, plus a JevBench public hard tier result that went down, 0.712 against 0.730, and a top-label ECE that went up, 0.147 against 0.098. Reporting a regression in a headline metric while shipping the change is worth noticing, since it tells you the evaluation is not being cherry-picked.

## An independent reproduction, not a distillation

The README has an Independence section, and it is unusually specific about provenance, which matters for anyone deciding whether to trust the numbers.

The project is independent and not affiliated with or endorsed by TypeSafe AI. It is an open reproduction of the System One model class, which is TypeSafe AI's Jev. The 2B model is built on Qwen/Qwen3.5-2B-Base, the 4B on Qwen/Qwen3.5-4B-Base, and the 35B mixture of experts on Qwen/Qwen3.5-35B-A3B-Base.

On the training data, the README says the mixture is public datasets plus data labelled by a local Qwen3.5-27B teacher, pointing at teacher_data/ and decider/data/mixture.py, and then states plainly: nothing was distilled from Jev.

That last sentence is the one to check if you care about whether the reproduction is independent. A reproduction that distilled from the original model's outputs would inherit the original's behaviour including its errors, and any comparison against JevBench would be partly circular. The stated alternative, training against a local teacher on public data, is a different and more defensible claim.

The repository layout backs the claim up. There is a teacher_data/ directory, a moe/ directory for the mixture of experts, a decider/ package, docs/, examples/, scripts/ and tests/, and a set of model cards at the root, including MODEL_CARD.md, MODEL_CARD_4B.md, MODEL_CARD_35B.md, MODEL_CARD_35B_NVFP4.md, MODEL_CARD_0.8B.md and MODEL_CARD_VISION.md.

The licence is Apache-2.0, and the weights are published on Hugging Face under the Mapika organisation.

## Games as the demonstration, not the claim

The showcase image is a set of recorded episodes, and the caption under it explains the setup in a way that is worth reading carefully.

Every move is one forward pass, and the bars are the served probabilities. The Tetris harness shortlists 8 placements with a hand-tuned heuristic and states their consequences, and the model picks one, 20 lines per game against 0.6 for a random pick from the same 8. Pong uses an unreleased games-RL overlay, and sources, seeds and windows are in docs/DEMOS.md.

The honest framing here is the shortlist. The model does not choose among all legal Tetris placements; it chooses among 8 that a heuristic has already filtered, and the comparison is against a random pick from those same 8. That is a narrower claim than a game-playing system, and the number 20 against 0.6 only means something when you know the shortlist size.

The same pattern appears in the other examples. The pyproject.toml has a games extra pulling in gymnasium, gym-super-mario-bros, nes-py, minigrid, ale-py and imageio, and the examples directory holds composite_scoring.py, hierarchical_beam.py and routing_with_confidence.py. The last of those names is the most relevant to a production use, since routing with confidence is what a decision model in a pipeline actually looks like.

The games exist to show that the probabilities are usable in a loop where every action matters. They are a demonstration that the output format holds up under repeated use, and a reader should not take them as a benchmark against other game agents.

## Serving, GGUF and the packaging shape

The installation is a single pip command, with a naming detail worth catching.

```bash
pip install decider-ai                  # the import name is decider
```

The package is decider-ai and the import is decider, and the README flags that difference explicitly. The distribution name on PyPI and the module name differ, which is a small trap for anyone writing an automated install step.

The pyproject.toml requires Python 3.11 or newer and pulls torch, transformers at 5 or newer, flash-linear-attention, numpy, huggingface_hub and jinja2. The optional extras are where the variants live: serve for fastapi, uvicorn and httpx, train for datasets, accelerate, scikit-learn and pillow, games for the RL environments, gguf for llama-cpp-python, and metal for mlx on Apple Silicon.

The serve extra is what the HTTP server is, and the changelog shows it being hardened. In 1.8.1, decider.serve bounds the shared-prefix forward for Gemma-4 and for long per-question suffixes, because Decision Index retrieval requests with 32 documents of about 26k tokens ran out of memory in 1.8.0 even on a 268 GB GPU. They now read at 68.5 GB peak, and a new setting DECIDER_SHARED_SUFFIX_TOKENS at 65,536 controls the bound.

That is a real production lesson about long-context question sets, and the fix is the kind of thing you only learn by running large retrieval workloads.

The 31b path goes through vLLM instead, and decider.serve_vllm gained an optional second reading of unsure questions with the options reversed, set through decider_config.json second_reading, plus an attention backend setting through vllm_attention and DECIDER_VLLM_ATTENTION. Reading an uncertain question twice with the answers in the opposite order is a cheap consistency check, and the reverse-order trick is the standard way to detect a position bias in a classifier.

The last push was on 2026-10-12, and the package version is 1.9.0.

## Conclusion

decider suits a team building a classifier, a router or a triage tool where the answer set is known in advance and a probability is more useful than a generated string, because the output space is bounded by construction and every question comes back with a distribution. It is the wrong tool for open-ended generation, and it is the wrong choice if you need a chat model, since the 31B chat variant still returns typed answers. Before adopting it, read the Limits section in the README, since the calibration claims and the regression set results are the part you need to judge rather than the games demo, and check that the model size fits your hardware, because the table in the README spans 8.4 GB of weights for the 4B model up to 65 GB for the 35B mixture of experts.

## FAQ

### What is the decider model and how does it differ from a normal LLM?

It is a model that does not generate text. It reads a state and a set of typed questions and returns, from one forward pass, a probability distribution for every question, with no decoding, no parsing and no output outside the options you defined.

### What question types does decider support?

Choice over 2 to 255 options, Score over 2 to 10 described levels, and Noul, which is the probability of yes. Each returns a distribution over the answer set you supplied.

### Which decider model should I use on a CPU?

The decider-2b-GGUF or decider-4b-GGUF builds through llama.cpp, which are Q4_K_M files of 1.3 GB and 2.7 GB, or the 2B model in PyTorch. On 8 server CPU threads a 40 to 120 token request takes 0.12 to 0.31 s with the 2B and 0.3 to 0.7 s with the 4B.

### Is decider distilled from TypeSafe AI's Jev model?

No. The README states it is an independent reproduction of the System One model class, not affiliated with or endorsed by TypeSafe AI, trained on public datasets plus data labelled by a local Qwen3.5-27B teacher, and that nothing was distilled from Jev.

### How do I install decider?

Run pip install decider-ai. The package name is decider-ai while the import name is decider, and the README flags that difference explicitly.

## Sources

- [Issues](https://github.com/Mapika/decider/issues)
- [License: Apache-2.0](https://github.com/Mapika/decider/blob/main/LICENSE)
- [Mapika/decider on GitHub](https://github.com/Mapika/decider)
- [README](https://github.com/Mapika/decider/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/mapika-decider