# Rizzo Flow asks a local model for typed decisions instead of generated text

> Rizzo Flow is a local-first reimplementation of the idea behind TypeSafe's Jev, returning typed decisions with probabilities from an HTTP API instead of text to parse. It runs a 4B model through llama.cpp, generates no tokens, and the README is unusually blunt that its probabilities are uncalibrated unless you calibrate them.

**Rizzo-AI-Academy/rizzo-flow** — The open, local take on Jev: typed decisions from an LLM, without generating a single token

- Repository: https://github.com/Rizzo-AI-Academy/rizzo-flow
- Website: https://rizzo-ai-academy.github.io/rizzo-flow/
- Stars: 931 · Forks: 65
- Language: Python
- License: Apache-2.0
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/rizzo-ai-academy-rizzo-flow

## The pitch: unstructured state in, typed decisions out

Rizzo Flow is described as the open, local take on Jev, with typed decisions from an LLM and without generating a single token. The summary line is: unstructured state in, typed, probabilistic decisions out, on your own machine.

The idea it is based on comes from a TypeSafe blog post introducing System One models and Jev. The model is a function call with judgment, which takes unstructured state and returns typed decisions with probabilities, a yes/no, a choice among options, a score on a rubric, a number, instead of text you then have to parse.

That last clause is the whole argument. The usual pattern for asking a language model a structured question is to prompt it, receive prose or JSON-shaped text, and then parse it, with a retry loop and a validation step for when the output is malformed. Here the output type is fixed by the request, so the parsing problem is designed away.

The project is explicit about what it is not. Jev is a closed hosted service. Rizzo Flow gives the same programming model on your own hardware, with open weights, and with the same HTTP interface, so code written against the TypeSafe API can point at localhost by changing one URL. It runs on llama.cpp, so the hardware can be an Apple, NVIDIA, AMD or Intel GPU, or no GPU at all.

It also states plainly that it is an independent project, not affiliated with TypeSafe, and that it does not reproduce Jev's proprietary architecture or its RLCD training. What it reproduces is the interface pattern, with an open model, Spark-X2.5, carrying a LoRA fine-tune of the project's own on public data.

## The disclaimers are the most informative part of the README

A project that leads with its own limits is telling you something about the rest of the documentation, and this one does.

The README says the project was made in the spirit of SemIf, which inspired it. It says probabilities are uncalibrated unless you calibrate them on your own data. And it says plainly that they make no claim of matching Jev or SemIf in quality, with the line that every number below comes with its caveats.

That last sentence is the framing for the entire results section, and it is a good one. Most projects publish a benchmark table and let the reader do the discounting.

The benchmark itself is on the typed-decisions dataset, with the same GPU and Q8_0 quantisation across the rows.

Three things are worth reading off that table. The fine-tuned model beats the original Spark-X2.5-4B on every metric, so the LoRA fine-tune is doing real work. The probabilities improved more than the accuracy did: KL from gold fell from 2.899 to 0.452 and ECE from 0.349 to 0.112, while accuracy moved from 0.574 to 0.648.

The third row is TypeSafe Jev 1.13.0 taken from its own dataset card, with accuracy 0.727, KL from gold 1.442 and Brier 0.148, and no ECE figure given. That missing cell matters, because the comparison is not like for like across every column.

And against Jev, the honest reading is in the README's own summary line: better answers by 7.4 points, and much better-shaped probabilities, still below Jev in accuracy. Note also that Jev's row comes from its own dataset card, with no ECE figure given, so the comparison is not like for like across every column.

The default model has been the fine-tuned one since 25 September 2026.

## Four commands, and the only slow step is a download

The quickstart is four commands, and the README says the same four work on macOS, Windows and Linux, that nothing is compiled and no GPU toolkit is installed.

```bash
git clone https://github.com/Rizzo-AI-Academy/rizzo-flow && cd rizzo-flow
uv sync --locked          # seconds: four small Python packages
uv run rizzo download     # llama.cpp for this machine + Rizzo Flow 4B Q8_0 (~4.4 GB)
uv run rizzo serve        # → http://127.0.0.1:8017/playground
```

You need Python 3.11 or newer, git and uv.

The download is the slow part, and the README breaks down exactly what it fetches. The model is 4.4 GB. The llama.cpp runtime is about 570 MB with CUDA, 30 MB with Vulkan, and 11 MB on a Mac. An interrupted download resumes where it stopped.

If you are in a hurry, the download command takes a size argument.

```bash
uv run rizzo download --size 1.7b
```

That fetches a 1.8 GB model that is fine for a first look and much less accurate afterwards, which is a sensible way to try the thing before committing to the full download.

Then you open the playground at 127.0.0.1:8017/playground, pick an example from the Examples menu or write your own state and questions, and press Run.

There is also a path that needs no server for one-off runs.

```bash
uv run rizzo decide examples/ticket.json
```

The three example files in the repository are ticket.json, numeric.json and house.json, which suggests the three question shapes: a ticket triage example, a numeric example and one with multiple choices.

If you prefer an activated environment, the README says to source .venv/bin/activate, with the PowerShell equivalent .venv\Scripts\activate, and drop the uv run prefix.

## The API call and the schema files that define the contract

The endpoint is /v1/systemone, and the request carries a state, a model name and a map of questions.

```bash
curl http://127.0.0.1:8017/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state": "Help! My payouts have been failing for 3 days.", "model": "rizzo-latest",
       "questions": {"is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"}}}'
```

A question has a type and instructions. The type in the example is noul, which is French for no, and it is spelled that way in the request. That naming convention, one question type per answer kind, is what makes the answer typed rather than free text.

The README also gives the Windows PowerShell form of the same call, which builds the body as a hashtable and converts it with ConvertTo-Json before posting to /v1/systemone.

```powershell
$body = @{ state = "Help! My payouts have been failing for 3 days."; model = "rizzo-latest"
           questions = @{ is_urgent = @{ type = "noul"; instructions = "Does this convey urgency?" } }
         } | ConvertTo-Json -Depth 5
Invoke-RestMethod http://127.0.0.1:8017/v1/systemone -Method Post -ContentType "application/json" -Body $body |
  ConvertTo-Json -Depth 6
```

Note that there is a second endpoint as well. The Snake demo caption says every move is one POST /v1/decisions, which is a different path from /v1/systemone. The repository listing includes request.schema.json and response.schema.json at the top level, and those two files are the actual contract, more so than any example.

The playground screenshot description confirms the response shape: on the right, the answers appear as probability bars, with timings and zero generated tokens. Each question comes back with a probability distribution over its answer space, and no text is generated at any point in the pipeline.

## llama.cpp by default, MLX as an optional second runtime

The dependency list is unusually short for a project of this kind.

```toml
dependencies = ["pydantic>=2.11,<3", "fastapi>=0.115,<1", "uvicorn>=0.34,<1", "jinja2>=3.1,<4"]
```

Pydantic for the typed request and response, FastAPI and uvicorn for the server, and jinja2 for the chat template of the GGUF file. The comment above the list explains the absence of a native inference library: the default runtime is llama.cpp, with prebuilt binaries fetched by rizzo download and driven through ctypes, so it adds no Python dependency besides jinja2.

That ctypes design is why the install is four small Python packages and nothing is compiled. The llama.cpp binary is fetched as a prebuilt artefact matched to your machine.

The optional extras are the alternative runtime. The mlx extra pins mlx 0.32.2, mlx-lm 0.31.3 and a git dependency on Spark-MLX-LLM at a specific commit. The cuda and cpu extras use the same three packages with the compute package swapped, and the pyproject comment says the rule is to pick at most one extra since all three are MLX and only the compute package differs.

The cuda extra installs mlx-cuda-13 on non-Mac platforms and the cpu extra installs mlx-cpu on non-Mac platforms. The tool.uv section declares a conflict between the cuda and cpu extras, because those two ship the same library files and cannot share an environment.

There is also an override-dependencies entry restricting nvidia-nccl-cu13 to Linux. The comment explains that only the Linux wheel of mlx-cuda-13 needs NCCL, that NCCL has no Windows build, and that uv reads one wheel's metadata for every platform, so it is restricted by hand.

That last detail is a good example of the project being honest about its constraints: the packaging works around a real ecosystem limitation rather than pretending it does not exist.

## Latency, memory and the Snake demonstration

The badges give the hardware numbers, and the Snake demo gives them a use.

The latency badge reads about 50 ms per decision at Q8_0 on an RTX 5060 Ti, and the GPU memory badge reads about 5.6 GiB at Q8_0.

The Snake demo is the more interesting artefact, and the README is careful about how it is presented. It is a game of Snake played by Rizzo Flow, where on every move the bars show the probability of each direction. The caption says it was recorded at real speed, not sped up: 140 moves in 25.6 seconds, about 5.5 per second, about 150 ms per decision round trip, and 0 generated tokens.

Two caveats come with it. The recording used Spark-X2.5-4B at 8 bit on an RTX 5060 Ti, recorded with the earlier MLX runtime, and the current llama.cpp runtime is about 1.8 times faster per decision. And the caption says it is one game, not a benchmark.

That framing is deliberate and worth noting. The 50 ms badge figure and the 150 ms per decision in the demo are not the same measurement, and the demo is the slower path. So the honest range for this kind of hardware is somewhere between a fast local decision and a few hundred milliseconds round trip.

The native context of the model is listed as 1M tokens, which matters for the state size question. Passing a large structured state into a single decision call is the sort of thing a long context window makes practical.

The models available are Spark-X2.5 at 4B and 1.7B.

## Where Rizzo Flow is the wrong tool

Three limits are worth stating plainly, and the README states the first two itself.

The first is accuracy. The fine-tuned 4B model reaches 0.648 on the typed-decisions benchmark against Jev's 0.727. If your decision is a yes/no that gates a customer support escalation, a 0.648 model is a starting point for calibration work, not a finished classifier. The README is direct about this and about the fact that no claim is made of matching Jev or SemIf in quality.

The second is calibration. The probabilities are uncalibrated unless you calibrate them on your own data. A model that says 0.9 might be right 60% of the time. If your system thresholds on probability, an uncalibrated model gives you thresholds that mean nothing. The ECE figures in the table show how far from calibrated the base model was, 0.349, and how much the fine-tune improved it, 0.112, which is better but not nothing.

The third is scope. The output is a decision with a probability, and the state you pass in is unstructured text that the model interprets. There is no retrieval, no tool use, no function calling beyond the typed question, and no multi-step reasoning pipeline in the described design. If what you need is an agent that reads documents and takes actions, this is the wrong layer.

There is also a maturity point. The version in pyproject.toml is 0.1.0, there are no GitHub releases, and the repository carries a .claude/ directory, a CLAUDE.md file and a skills/ directory, which suggests an active development workflow around AI-assisted contribution rather than a stable published API.

The alternative is the hosted service the project is modelled on, which you already know about: Jev is closed and hosted, so using it means sending your state off your machine and accepting a different accuracy level. For most teams that trade is not worth making, which is the case for this project existing.

## Conclusion

Adopt Rizzo Flow if you want structured decisions from a language model with probabilities attached, if the data must stay on your own machine, and if you are prepared to calibrate those probabilities on your own data before trusting them. Do not adopt it if you need frontier accuracy, since the README's own table puts the fine-tuned 4B model below TypeSafe Jev on accuracy, and do not treat the returned probabilities as calibrated until you have done that work yourself. Before you build on it, run the typed-decisions benchmark on your own hardware and read the request.schema.json and response.schema.json files, because the shape of the response is the actual contract.

## FAQ

### What is Rizzo Flow?

It is an open-source, local-first implementation of the idea behind TypeSafe's Jev, where a function call with judgment takes unstructured state and returns typed decisions with probabilities instead of text you have to parse. It runs a Spark-X2.5 model locally through llama.cpp and generates no tokens.

### How do I run Rizzo Flow locally?

Clone the repository, run uv sync --locked, then uv run rizzo download to fetch llama.cpp and the model, and uv run rizzo serve to start the server. You need Python 3.11 or newer, git and uv, and the same four commands work on macOS, Windows and Linux.

### Are the probabilities from Rizzo Flow calibrated?

No, not by default. The README states that probabilities are uncalibrated unless you calibrate them on your own data, and it makes no claim of matching Jev or SemIf in quality.

### How accurate is the Rizzo Flow 4B model?

On the typed-decisions benchmark at Q8_0, the fine-tuned Rizzo Flow 4B model scores 0.648 accuracy, against 0.574 for the original Spark-X2.5-4B and 0.727 for TypeSafe Jev 1.13.0 according to its dataset card.

### Does Rizzo Flow need a GPU?

No. It runs on llama.cpp, so the hardware can be an Apple, NVIDIA, AMD or Intel GPU, or no GPU at all. The badges quote about 50 ms per decision at Q8_0 on an RTX 5060 Ti using about 5.6 GiB of GPU memory.

### What licence is Rizzo Flow released under?

Apache-2.0, stated in the licence badge in the README and in the pyproject.toml metadata, with a LICENSE file and a NOTICE file at the root of the repository.

## Sources

- [Issues](https://github.com/Rizzo-AI-Academy/rizzo-flow/issues)
- [License: Apache-2.0](https://github.com/Rizzo-AI-Academy/rizzo-flow/blob/main/LICENSE)
- [Project website](https://rizzo-ai-academy.github.io/rizzo-flow/)
- [README](https://github.com/Rizzo-AI-Academy/rizzo-flow/blob/main/README.md)
- [Rizzo-AI-Academy/rizzo-flow on GitHub](https://github.com/Rizzo-AI-Academy/rizzo-flow)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/rizzo-ai-academy-rizzo-flow