# Olympus routes one prompt to twenty vision specialists, and pins two different torch versions

> Olympus is the code for a CVPR 2025 highlight paper that parses one natural-language instruction into routing tokens, dispatches them to specialist vision models, and chains the outputs into finished files. The install instructions are where to look first: the pinned requirements fix one CUDA-flavoured tensor version while the specialist step tells you to install a newer one into the same environment.

**yuanze-lin/Olympus** — [CVPR 2025 Highlight] Official code for "Olympus: A Universal Task Router for Computer Vision Tasks"

- Repository: https://github.com/yuanze-lin/Olympus
- Website: https://yuanze-lin.me/Olympus_page/
- Stars: 427 · Forks: 73
- Language: Python
- License: not declared
- Published: 2026-09-14 · Updated: 2026-09-14 · Language: en
- Canonical page: https://hysenlabs.com/projects/yuanze-lin-olympus

## A router, and then twenty specialists in the same environment

The product claim is compact: one natural-language instruction goes in across twenty vision tasks, each is dispatched to a specialist model, and the outputs are chained. The result is finished files rather than a notebook you had to wire together yourself, in three asset formats, images, videos, and 3D models.

What makes that claim more than a wrapper is that the system resolves dependencies between tasks rather than executing them in the order you typed them. The README says Olympus parses the instruction into routing tokens, dispatches each to its specialist, and resolves the dependencies between them, so no step has to be wired up by hand. The quick start prompt chains four steps, generating an image, editing that image, turning the edited image into a high-resolution 3D model, and then producing a video of a different scene.

The install split follows the same two-layer design. A conda environment on Python 3.10 with the pinned requirements is enough to run the router on its own, meaning it parses and plans. Executing the routed tasks, actually generating the images, videos, and 3D models, needs the specialist stack installed into that same environment, with no per-tool environments to manage:

```bash
pip install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements_tools.txt
bash scripts/install_specialists.sh
```

One environment for everything is convenient and is also the reason the next section matters.

## The pinned torch and the instructed torch disagree

This is the first thing to check before installing anything, and it is a genuine contradiction inside the project's own instructions.

The router's requirements file pins the tensor library at version 2.2.0 built for one CUDA release, with matching pins for its vision and audio companions, and it adds that CUDA release's wheel index as an extra index. It also pins a block of individual CUDA runtime packages, and those are built for an earlier CUDA major version than the tensor wheel itself, which is an odd combination worth noticing before you debug it.

Then the specialist section says: to execute the routed tasks, install a newer tensor version and its vision companion from a different CUDA wheel index into the same environment.

Those cannot both be true of the final environment. Following the documentation in the order it appears means the router's pinned tensor build is replaced by the newer one, along with its CUDA index and its pinned CUDA libraries. Nothing in the visible text says to install the specialist step first, or to create a separate environment for it, and the text explicitly rules out per-tool environments.

The router is a transformer library with a pinned release, so a major version jump in the tensor stack is exactly the kind of change that breaks a pinned research environment. Whether it works is not something the repository answers. What it does is put the conflict in plain sight.

## Around eighty exact pins, and no upper bounds to argue about

The router requirements file is a wall of exact versions, and the pattern is uniform.

The modelling stack is pinned across the board: an acceleration library, a parameter-efficient tuning library, a distributed training library, a Triton compiler version, a tokenizers version paired with a transformers version, a safetensors version, and a sequence length of pinned releases that place the file in a specific era of the ecosystem. Around them sit the usual analysis stack, exact-pinned as well: arrays, dataframes, a scientific library, a machine learning library, imaging, plotting, and networking clients.

Two entries are worth singling out because they affect what you can run rather than what you can reproduce. A weights and biases version means experiment tracking is wired in rather than optional, and a Sentry client means error reporting leaves your machine by default unless you configure it otherwise. Neither is a problem for reproducing a paper and both are worth knowing before running anything on a network you care about.

The upside of exact pins is the point: two people running this file get the same environment. The downside is that no security or compatibility fix arrives without editing the file, and a transformers release from that era will not load newer model families. For a research artefact whose purpose is to be run, that is the right trade, and for anything else it is a constraint.

## Gated weights, and three layers of fallback for 3D

The 3D tasks are the most demanding part of the pipeline, and the documentation handles the failure modes carefully.

Two backends serve them: one model for image-conditioned 3D generation and another for text-conditioned generation, both producing textured meshes with physically based materials. They compile several CUDA extensions, so they are installed separately from everything else with a dedicated script rather than through the requirements file.

That script also builds a third model as an ungated fallback for the image-conditioned token, and the design principle is stated plainly: each step is independent, so a failure in one does not block the others, and any 3D token whose backend is missing falls back automatically rather than erroring. For a system that chains tasks, an automatic fallback is the difference between a degraded video and a crashed run.

The gated repositories are named, with their roles: one for image conditioning and one for background removal. Accepting both licences while signed in is all that is needed. If a request is rejected, the instructions are to obtain the weights locally and point two environment variables at the folders, with the note that the hosted call is attempted first and the local paths apply only if it fails.

And there is a third option: skip the gated model entirely and let that token use the ungated fallback.

## Twenty-one JSON files, then a merge step

The released data is organised by task rather than as one blob, which is the right shape for a router.

There are twenty JSON files under a folder named for the individual tasks, one per task, plus a chain-of-action file. The routing token definitions in the paper are what map a token to a task, so the mapping between instruction and file lives in the paper rather than in the data. Each of those twenty-one files carries both training and test data, and two more files hold the collected instruction set and the benchmark set.

Three download scripts do the fetching, and each writes somewhere specific: the model script saves into a checkpoints directory, the fine-tuning script saves the instruction file into a training data directory and puts every other JSON into a newly created directory, and a third script saves the base model used for fine-tuning into the checkpoints directory alongside the router.

The fine-tuning instruction file is a combination: a public mixed instruction set of roughly six hundred and sixty-five thousand samples plus the project's own collected instructions covering the twenty tasks. If you would rather merge it yourself, the README points at the public file, a merge script, and a second script that selects which tasks to include. That is a small courtesy, and it means the merge is reproducible rather than a one-off.

## The plan prints before the work starts

Inference is one script with three arguments, and the prompt is a whole paragraph rather than a keyword.

You pass a natural-language instruction, a path to the router checkpoint, and an output directory. Inside, the sequence is parse, dispatch, resolve dependencies, and execute, with the router reading the instruction into routing tokens and handing each to its specialist.

One behaviour is stated and then cut off mid-sentence: the plan is printed before anything else happens. That is worth having in a system that executes models you did not choose, because it gives you the opportunity to read what it intends to do before it does it. The rest of that sentence is not visible in what is readable here, so whether the plan is merely printed or also interruptible is not something the visible text settles.

The repository layout supports the reading that there are two entry points rather than one. A prediction script and a shell wrapper sit at the top level next to the tool runner, and there are separate directories for the training code, the router's own tool integrations, evaluation, and experiment tracking. Which of them a given task needs is not documented in the visible portion, so treating the tool runner as the only documented path is the safe reading.

## A highlight paper, released as scripts and weights

The publication context explains the shape of the repository.

This is the official code for a paper accepted as a highlight at a computer vision conference in 2025, available as a preprint, with a project page and both a model and a dataset hosted on a model hub under the first author's account. Six authors are listed, spanning the first author, two researchers in the same group, and three senior researchers at two institutions.

The news list is a checklist of released artefacts, and all five items are ticked: the prompt-to-assets behaviour, the code for integrating task-specific models, the training and inference code, the datasets, and the router model itself. That is a more complete release than most papers get, and it is the main reason this repository is worth reading even if you never run it.

Two facts about the repository's own history. There are no tagged releases, so nothing is versioned in the way you would expect for something with a model checkpoint attached, and the last push to the main branch is dated 2026-08-05. The repository metadata carries no licence value, so anyone planning to reuse the code or the weights should establish the terms directly rather than assuming one.

## Conclusion

Olympus suits a researcher evaluating whether a task router is worth building, since the model, the collected data, and the training and inference code are all released. It does not suit anyone who needs it to work today without reading the install order carefully, because the router's pinned tensor version and the specialist stack's requested one conflict. Before installing, decide whether you only need the router, check the two CUDA-related pins against your driver, and accept the two gated model licences before anything else.

## FAQ

### What does Olympus actually do?

It routes a single natural-language instruction across 20 computer vision tasks, dispatches each to a specialist model, and chains their outputs, so one prompt produces finished image, video, and 3D model files. It parses the instruction into routing tokens and resolves dependencies between tasks rather than running them in the order typed.

### What does running Olympus require?

A conda environment on Python 3.10 with the pinned requirements file is enough to run the router. Executing the routed tasks additionally needs the specialist stack installed into that same environment, plus a separate install script for the 3D backends, which compile CUDA extensions.

### Why might an Olympus install break?

The router requirements file pins one tensor library version built for a specific CUDA release, while the specialist instructions tell you to install a newer tensor version from a different CUDA wheel index into the same environment. Following both in order replaces the pinned version, and the text explicitly rules out per-tool environments.

### What happens if a gated Olympus model is unavailable?

The 3D backends include an ungated fallback model, each install step is independent so one failure does not block the others, and a 3D routing token whose backend is missing falls back automatically rather than erroring. Local weight folders can also be supplied through two environment variables when a hosted request is rejected.

### What data does Olympus release for training?

Twenty per-task JSON files plus a chain-of-action file, each containing training and test data, along with collected instruction and benchmark files. The fine-tuning instruction file combines a public mixed instruction set of about 665k samples with the project's own collected instructions, merged by script.

## Sources

- [Issues](https://github.com/yuanze-lin/Olympus/issues)
- [Project website](https://yuanze-lin.me/Olympus_page/)
- [README](https://github.com/yuanze-lin/Olympus/blob/main/README.md)
- [yuanze-lin/Olympus on GitHub](https://github.com/yuanze-lin/Olympus)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/yuanze-lin-olympus