OpenFugu: rebuilding Sakana's Fugu orchestrator as a runnable Python pipeline

The problem OpenFugu addresses: a product sold as one model is actually a router

Sakana AI's Fugu is marketed as a single model. The README's framing is that it is really a policy over models: a small coordinator that decides, per query, which frontier LLM in a pool should handle the work, then returns that worker's answer as if it came from one system. The product and its trained weights are closed. OpenFugu exists to rebuild that mechanism from the two associated papers and released artifacts, check the rebuild against real weights, train a coordinator of its own, and expose the result behind one OpenAI-compatible endpoint. The audience is narrow and technical: someone who wants to know how per-query routing over a heterogeneous model pool is actually implemented, and who is willing to run training scripts and fetch artifacts to find out. It is not a library you import to get better answers from an existing chat app.

The TRINITY mechanism: one hidden state, a bias-free linear head, one dispatched worker

The README describes the core loop in a single paragraph, and the detail is the interesting part. A roughly 0.6B backbone (Qwen3-0.6B) never answers the user. It runs the prompt and produces one hidden state at the penultimate token. A bias-free linear head scores each worker in the pool from that vector. The top-scoring worker is dispatched, and its reply is returned verbatim. The total trainable surface is about 19.5K numbers: the head plus singular-value-fine-tuning offsets on nine matrices, optimized gradient-free with sep-CMA-ES. Worker weights are never touched, which is the whole design: this is macro-level composition over other people's models rather than fine-tuning. Fugu-Ultra replaces the per-turn picker with a 7B Conductor that emits an entire workflow DAG instead of a single slot choice. The docs directory carries the math (docs/HOW_FUGU_IS_IMPLEMENTED.md) and an investigation log (docs/ARCHITECTURE.md) that the README says grades every claim as EXEC, CODE or DATA evidence. That grading scheme is the most useful thing in the repo for a reader trying to separate what was verified from what was inferred.

Getting it running: artifact fetch, environment variables, and the self-test

The quickstart assumes Python and a requirements.txt install covering torch, transformers, trl and litellm. Artifacts are not redistributed; scripts/fetch_artifacts.py pulls Qwen3-0.6B, model_iter_60.npy and the fixture from their licensed sources. Three environment variables point the code at them: FUGU_MODEL for the Qwen3-0.6B path, FUGU_VECTOR for model_iter_60.npy, and FUGU_FIXTURE for qwen_router_prompt_eval_cases.json. The first command worth running is python openfugu/mini.py --self-test, which the README reports as 95% agent and 100% role on a 37-case fixture using real weights. A mock routing demo follows with python openfugu/mini.py --demo, and a live pool needs FUGU_API_KEY, FUGU_BASE_URL and a --slot-models CSV of provider-prefixed model names passed to the same script with --live. Serving is python openfugu/serve.py --slot-models "<csv>" --port 8088, after which a plain curl POST to localhost:8088/v1/chat/completions returns one answer. The pool stays hidden behind that endpoint, which is the point of the exercise. For a fully local run, serve.py also accepts --model, --vector, --head and --local-models, and pipeline/e2e_train_serve.py chains training, serving and verification in one command so the head being served is the head just trained.

Training scripts and what the reward curves actually show

Four training paths are documented, and they differ sharply in how much they prove. train/train_trinity.py self-trains the TRINITY coordinator from scratch with sep-CMA-ES and no Sakana weights; the README says it goes from chance to optimal routing in about five generations, but on a mock, so it runs anywhere and demonstrates the optimizer rather than a production router. train/train_conductor.py runs GRPO on nvidia/ToolScale with a reported reward move from 1.21 to 1.64 over 100 steps, on 8x A800-class hardware using HF generation rather than vLLM. The recursion scripts are where the README is most candid: train_recursion.py shows a mock gain of about 9% over one-shot, but eval_recursion_real.py on honest held-out data reports round-0 versus round-1 as a tie. The adaptive-pool scripts show 0.625 to 1.000 on a real per-step k-of-n subset task at n=8, with the README attaching an explicit overfit caveat. That mix of mock wins and real ties is the honest shape of a reverse-engineering project, and the repo does not hide it.

The +107% routing number and the caveat attached to it

The central claim of the Fugu idea is that orchestration beats the best single model, and eval/eval_orchestration.py tests it per question. The README reports a trained router at +107% over the best single worker. The same table entry immediately qualifies the result: this is query-level routing, not per-step coordination, and it points to a results caveat. That distinction matters more than the percentage. Choosing one model for an entire question is a much easier problem than switching workers mid-trajectory, and a reader who quotes the number without the qualifier will be overstating what was shown. The repo deserves credit for putting the caveat in the same row as the number, but anyone evaluating OpenFugu for a real workload should read the results directory before treating +107% as a property of the architecture rather than of the specific eval setup.

Where OpenFugu is the wrong tool

Two constraints stand out. First, the core router depends on a specific hidden state from a specific backbone at a specific token position, so swapping in a different base model is not a configuration change; it invalidates the trained head and the vector. Second, the training paths that are not mocks need real resources: the Conductor run is documented for 8x A800-class GPUs, and the local serving path wants the Qwen3-0.6B directory plus a vector plus a trained head plus at least two local model directories. If you just want better answers from one provider, a router that dispatches to a pool you have to supply and pay for adds a moving part without removing any. And if you need per-step coordination rather than per-question routing, the README's own caveat says that is not what the headline eval measured.

The alternative approach: a gateway router versus a learned policy

The obvious comparison is LiteLLM, which OpenFugu already uses internally for its worker pool. LiteLLM is a gateway: it normalises provider APIs, handles fallbacks, retries and cost accounting, and its routing is configured by rules you write (model lists, fallback chains, budgets). OpenFugu is the opposite kind of object. It learns a policy from data, stores it as roughly 19.5K numbers, and makes the routing decision from the prompt's hidden state rather than from a static rule. You can run LiteLLM alone and get a working multi-provider endpoint today; you run OpenFugu when the question you care about is whether a learned coordinator picks better workers than a hand-written rule, and you are prepared to train and evaluate it to find out. The two are complementary rather than competing, which is why the OpenFugu serving path sits on top of a litellm pool.

Licence, maintenance and what to check before adopting

All OpenFugu code is Apache-2.0. Third-party material is fetched rather than redistributed, and the NOTICE file is the place to read the terms. The trained Conductor checkpoint is a fine-tune of Llama-3.2-3B-Instruct and is published on HuggingFace (huggingface.co/di-zhang-fdu/openfugu-conductor-3b) rather than in the repository, and the README states the Llama 3.2 Community License applies to it. That split matters if you plan to ship the checkpoint: the Apache-2.0 grant covers the code, not the weights. This is a description of what the repository says, not legal advice; check the Llama licence text and NOTICE yourself. On maintenance, the repository is not archived and the last push is dated 2026-06-22, but no releases were retrieved, so there is no versioned artifact to pin. Expect to track the main branch and to re-run scripts/fetch_artifacts.py when artifact URLs or upstream model names change, since the slot model names in the examples are provider-prefixed strings that providers can retire. The first things to verify on your own machine are the mini.py self-test result and the pipeline/e2e_train_serve.py run, because those are the two commands where the reconstruction is checked against real weights end to end.

Editorial conclusion

OpenFugu is for engineers who want to inspect or reproduce the mechanism behind a commercial model orchestrator, not for teams looking for a drop-in production router. The mini.py self-test and the pipeline/e2e_train_serve.py command are the two things worth running first, because they are the points where the reconstruction meets real weights. Verify the results/ caveats and the NOTICE terms before quoting the +107% routing number or redistributing the Conductor checkpoint, which is published on HuggingFace under the Llama 3.2 Community License rather than in this repo.

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. trotsky1997/OpenFugu on GitHub