Cosmos-Transfer2.5: a multi-control world model for synthetic video

Generating a video from several control inputs at once

Cosmos-Transfer2.5 is built on top of Cosmos-Predict2.5 and produces high quality world simulations conditioned on multiple spatial control inputs. The model is described as a multi-controlnet, which is the architectural decision that matters: instead of taking one conditioning signal, it accepts structured input across several video modalities at once, with RGB, depth, segmentation and more named in the description, and the release history adding edge and blur checkpoints.

The purpose is data augmentation for physical AI. The project states that physical AI trains upon data generated in two important data augmentation workflows, and this repository is the model behind one of them. The audience is therefore research and engineering teams working on autonomous vehicles, robotics and video analytics who need additional training data that respects the geometry of the real world, and who can express the geometry as control video rather than as text.

That framing explains a design choice that would otherwise look strange. A general purpose text to video model is optimised for prompt adherence. This one is optimised for conditional fidelity, so a depth map of a scene can be used to hold the structure of that scene in the output while other signals drive appearance.

The licensing is Apache-2.0 and the code is published with a paper, a Hugging Face model card for the 2B checkpoint, a product website, and a Cosmos Cookbook collection of recipes and post-training scripts.

Configuration runs through JSON controlnet_specs

Users configure generation with JSON based controlnet_specs, and inference runs with a few commands. That split is the practical shape of the tool: the model and its weights come from a checkpoint, the experiment is described in a JSON file, and the run is launched from the command line.

The Python package is published as cosmos-transfer2 and the project requires Python 3.10 or newer, with the packaging metadata classifying it as beta, GPU only for NVIDIA CUDA, and aimed at a science and research audience.

[project]
name = "cosmos-transfer2"
requires-python = ">=3.10"

The dependency list is short in appearance and heavy in what it pulls. Alongside the usual data handling libraries it depends on cosmos-oss, which is pinned exactly, on segmentation through sam2, and on a depth model that is installed straight from a Git repository rather than from a released package.

dependencies = [
  "cosmos-oss==0.1.0",
  "cattrs>=25.2.0",
  "easydict>=1.9",
]

A dependency that comes from a Git URL is worth noting for anyone building a reproducible environment, because the version of that component is whatever the default branch of that repository happens to be at install time. The repository does ship a `uv.lock` file, which is the mechanism that keeps the rest of the environment pinned.

Automatic control map generation and the control modalities

The model supports automatic control map generation, which removes the most tedious part of the workflow. Instead of producing a depth map or a segmentation mask yourself, you can let the tooling derive the control input and then condition on it. Two later additions made that path more usable: on the fly computation support for depth and segmentation, and the autogeneration of spatiotemporal masking for control inputs when a prompt is given.

The control modalities themselves have grown over time. The December 12, 2025 release refreshed the checkpoints for the 2B model across blur, depth, segmentation and edge, which tells you those four are the ones with published weights. Later, on February 23, 2026, a distilled edge model and its inference guide were released for low latency, edge deployment inference, with more distilled controlnets described as coming.

A distilled checkpoint is the interesting one for production use. A full controlnet produces better results and costs more, while a distilled variant is intended to run close to the edge, which is the configuration a vehicle or a robot would need. The repository also notes fixes to an issue with autoregressive multiview generation when the number of conditional frames is zero, which is exactly the sort of corner that a research preview runs into and a production system cannot afford.

The result is a menu rather than a single path: pick a control modality, decide whether to compute the control map automatically or supply one, and decide whether you need the distilled variant.

Four example scripts cover four inference shapes

The `examples/` directory is the clearest evidence of what the model is for. It contains `inference.py` for the basic single video path, `multiview.py` for multi camera output, `plenoptic.py` for light field style rendering, and `robot_multiview_agibot_control.py` for robot control with the AgiBot dataset.

The multiview path has its own development history. A new multiview cross-attention module was added in November 2025 together with autoregressive sliding window generation for longer videos, and the Auto and Multiview code was refactored the following week. The longer video mode matters because autoregressive generation is how a model extends a clip past the length it was trained on, and a sliding window is the compromise that keeps memory bounded while doing it.

Image conditioning arrived separately. The December 19, 2025 release added Image2Image and ImagePrompt capabilities with their own inference guide, so a run can be conditioned on a still image or on a prompt as well as on control video.

There is also a path outside video. In November 2025, as part of the Cosmos family, the project released a recipe, a reference diffusion model and a tokenizer for synthetic LiDAR point cloud generation from an RGB image. That work lives in a separate repository, which shows how the Cosmos family is split into specialised components rather than one large codebase.

Environment setup runs through just and uv

The repository ships a `justfile` rather than asking developers to memorise long command lines, and the Docker image mirrors the same tooling. The `just` recipes cover installation, running commands inside the package environment, linting, type checking and a test of the install command itself.

Installation resolves the CUDA variant from the machine architecture, choosing cu130 on aarch64 and cu128 elsewhere, and then syncing the environment with the matching extra.

  uv sync --extra={{cuda_name}} {{args}}

Running a command inside that environment goes through a second recipe, which syncs first and then executes without resyncing.

  uv run --no-sync {{args}}

The optional dependency groups in the packaging metadata match those names, with a gradio extra and separate cu128 and cu130 extras, so the CUDA version is selected by extra rather than by an environment variable. The container image takes the same path: it is built on a CUDA 12.8.1 cuDNN devel Ubuntu 24.04 base, installs uv and the just binary, and adds ffmpeg, git, git-lfs and tree.

ARG BASE_IMAGE=nvidia/cuda:12.8.1-cudnn-devel-ubuntu24.04
FROM ${BASE_IMAGE}
ENV UV_LINK_MODE=copy

One number in the `justfile` is worth reading carefully, because it encodes a hardware expectation. The default maximum GPU count is 4 on aarch64 and 8 on other architectures, which tells you the multi GPU path is designed for a machine with several accelerators and that a smaller host has to be configured deliberately.

Type checking, linting and an install test

The engineering scaffolding in this repository is more thorough than a typical research drop. There is a `pyrefly.toml` and a `pyrefly-src.toml` pair, and the `justfile` runs pyrefly twice, once with the default configuration and once with the src configuration, with an option to whitelist errors. There is a `.pre-commit-config.yaml` and a base variant of it, a `.ruff.toml`, a `.pytest.ini`, a `conftest.py` and a `.coveragerc`.

The recipe named `test-install` is a good example of the level of care. It removes the recorded CUDA name, runs a plain sync, and then expects the import of the package to fail, with the comment explaining that the CUDA extra is not installed in that state. It then syncs with the extra and expects the import to succeed. In other words, the test asserts the behaviour of the install path rather than only the behaviour of the code.

The `pre-commit` recipe has a fallback in it, running pre-commit a second time if the first invocation fails, which is a pragmatic way to deal with hook environments that need a second pass after the first modifies files.

For a reader, this is the part of the repository most worth borrowing. The model code is what it is, but the recipe file, the split type checking configuration and the install test are patterns that transfer to any Python project with heavy optional dependencies.

The repository is on limited maintenance and Cosmos 3 takes over

The state of this project is stated at the top of the README, and it is unambiguous. The repository is no longer under active development and will receive only limited maintenance updates. Future model releases, features, documentation and community support will be focused on Cosmos 3, and users are encouraged to migrate. The Cosmos GitHub repositories have moved to the main NVIDIA organisation, and Cosmos 3 is described as unifying capabilities that previously required several specialised models, so that a single model can reason, predict future world states, transfer across domains and modalities, and generate actions and policies for embodied agents.

The commit record is consistent with that. The last push was on 2026-06-30, and the most recent releases are v1.5.4 on 2026-05-13, 1.5.3 on 2026-05-04 and 1.5.2 on 2026-04-29. Releases stopped in May while the code repository kept receiving commits until the end of June, which is what a wind down period looks like.

The practical consequence for an adopter is that the model weights and the code are frozen at a known state, and that the ecosystem around them, the cookbook recipes and the documentation, is no longer where the new work happens. Anyone starting a new project in late 2026 has a successor to evaluate, and anyone with an existing pipeline on Transfer2.5 has a working system that will keep running.

Where Cosmos 3 differs from a multi-controlnet on a fixed base

The alternative named by the project is Cosmos 3, and the difference in approach is architectural rather than incremental. Cosmos-Transfer2.5 is a transfer model: a base model that predicts video, extended with controlnet branches that condition it on structured signals, so each new control modality means new branches, new weights and a checkpoint to publish and maintain. That design is what makes the blur, depth, segmentation and edge checkpoints possible, and also what makes each of them a separate artefact to release and to keep compatible with the base model.

Cosmos 3 is described as a single architecture that handles language, images, video, audio and actions, so conditioning becomes one capability of one model rather than an extension per modality. The claim in the README is that it delivers stronger transfer capabilities across diverse domains, sensors, environments and embodiments, and unifies reasoning, prediction, transfer and action generation.

The other option is to build the control yourself on a different base model, with your own controlnet branches and your own training data. That is the slowest path and the one with the most control, and it is what a team with proprietary data and a different hardware target may need anyway. The choice between the two published options is a bet on the platform: Transfer2.5 gives you a documented, frozen recipe with published checkpoints, while Cosmos 3 gives you one architecture that will keep changing under you.

Editorial conclusion

Cosmos-Transfer2.5 still fits a team that needs controllable synthetic video for physical AI data augmentation and is content to pin a known checkpoint, such as the 2B model on Hugging Face. It is the wrong base for new work, because the README states the repository is no longer under active development, that future model releases and features will be focused on Cosmos 3, and that users are encouraged to migrate. Before adopting it, check that your hardware path matches the CUDA extra you select, that the control modality you need has a published checkpoint, and whether the Cosmos 3 repository already covers your case better than this one does.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. nvidia-cosmos/cosmos-transfer2.5 on GitHub
3. Project website
4. README
5. Releases