FlashDreams is NVIDIA's runtime for interactive autoregressive video and world models

What it is: the runtime that grew out of a demo

FlashDreams is a high-performance inference and serving library for interactive autoregressive video and world models.

The second sentence of the README is the most informative one about where the project came from. It began as the optimized runtime behind the NVIDIA OmniDreams closed-loop demo for GTC 2026 and has grown into a general platform for real-time world-model applications across gaming, autonomous vehicles, robotics, simulated or virtual environments, and more.

That origin explains both the strengths and the limits. A runtime built to make a closed-loop autonomous driving demo interactive at frame rate has been shaped by one demanding workload: a model that consumes observations and produces the next frame quickly enough to be steered. Generalising that into a platform is what the current state represents, and the direction of travel is visible in the API split, which is aimed at two different audiences.

The primary language is Python, and the homepage is the documentation site at nvidia.github.io/flashdreams. The topics are efficiency, interactive, video-models and world-models.

This is not a model release. It is the serving and inference infrastructure around a model, and reading the repository as a collection of models would be a mistake. The citation block in the README cites the project itself alongside the OmniDreams paper, with the latter described as NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation, which tells you the flagship application the runtime was built for.

The repository has no GitHub releases, so the version is whatever the default branch carries, and the last push was on 2026-10-20 with the repository not archived.

Two APIs for two different jobs

The Developer guides section splits the project by who you are, and that split is the clearest way to understand what the library offers.

For demo developers, the guidance is to make demos using world models with the Demo API. For model developers, it is to implement world models with the Inferencing API.

The difference in the names is doing real work. The Demo API is the application-facing surface: you have a model running and you want to drive it, and the Demo API is what you call to make that happen. The Inferencing API is the model-facing surface: you are writing the model itself and need the runtime to schedule and serve it.

Both are documented separately on the documentation site, under documentation/demo_api/ and documentation/inferencing_api/, and the README says the complete setup is in the quick-start guide, with an install section and a section on running your first model.

That two-audience split is a reasonable way to structure a serving library, and it is also the thing to check first when evaluating it. If you are integrating an existing world model into an application, the Demo API is what you want and the Inferencing API is irrelevant to you. If you are implementing a new model and need the runtime's scheduling and streaming, the reverse.

The demos are worth looking at before the API reference, because they show the intended usage patterns. There is an interactive driving demo with a clip of it in the documentation assets, and a game-like crazy-robotaxi demo with its own documentation page. Both are links into the quick-start and demos sections of the site, and both are the kind of thing that tells you what shapes of application the runtime was designed around: an interactive simulation loop where the model's output feeds back into the world.

The workspace layout tells you it is a platform, not a library

The top level entries are numerous and they describe the architecture better than the README does.

There is flashdreams/ for the core package, and then integrations_v2/ which contains what the package metadata calls standalone external integration repos, with the comment naming causal_forcing and self_forcing plugins and omnidreams and lingbot integrations. There is apps/ for applications, docker/ for container builds, configs/ for configuration, docs/ for the Sphinx-style documentation, tests/, tools/ and scripts/.

And there is skills/, which is an unusual directory to find in a serving library and suggests the project ships agent-facing instructions or task definitions rather than only human documentation.

The workspace definition in pyproject.toml makes the structure explicit:

[tool.uv.workspace]
members = [
    "flashdreams",
    # Each populated subdir under ``integrations_v2/`` is a standalone
    # external integration repo (e.g. ``causal_forcing`` / ``self_forcing``
    # plugins, ``omnidreams`` / ``lingbot`` integrations).
    "integrations_v2/*",
    "apps/*",
    # Nested sub-packages that the integration globs do not reach.
    "integrations_v2/omnidreams/impl/ludus-renderer",
]
exclude = [
    "apps/__init__.py",
    "apps/README.md",
    "integrations_v2/README.md",

The wildcards are the interesting part. integrations_v2/* means every populated subdirectory is treated as its own repository, so a plugin author gets an isolated package without the core project having to know about it. The exclusion list then handles the cases the glob gets wrong, with the explicit addition of integrations_v2/omnidreams/impl/ludus-renderer as a nested sub-package the integration globs do not reach.

There is an ARCHITECTURE.md and a DEV.md at the root, which is a good sign for a project at this scale, and an AGENTS.md alongside a PR_DESCRIPTION.md, which suggests the project expects contributions to come partly from automated tooling.

The naming of the two plugin families is worth noting for anyone trying to work out the model landscape. Causal forcing and self forcing are the two approaches to conditioning a video model on the previous state, and omnidreams is the NVIDIA closed-loop simulation stack. The library is designed to sit across several model families rather than one.

The dependency overrides are where the real engineering shows

The pyproject.toml of a machine learning library is normally uninteresting: a list of dependencies. Here it is the most revealing file in the repository, because the override-dependencies block is a list of deliberate pins with reasons attached.

The first one is a cuBLAS version conflict. The comment says nvidia-cublas is overridden to >=13.4 because transformer-engine-cu13 2.14.0 was compiled against cuBLAS 13.4.x while torch 2.11 pins nvidia-cublas 13.1.0.3, and that cuBLAS 13.4 is backwards compatible with 13.1. That is a native ABI problem between two GPU libraries, resolved by forcing the newer one.

The second is a numpy cap. The comment describes a temporary cap for mediapy 1.2.6, because numpy 2.5 makes npt.NDArray a PEP 695 TypeAliasType that mediapy subclasses during import, with an issue link. The pin is numpy>=1.24,<2.5. A dependency floor and ceiling chosen to work around a subclassing conflict in a video writing library is the kind of detail that only appears in a project with real users.

Then there are the security floors, and they are explicitly labelled as such. urllib3>=2.8.0,<3 for HTTP proxy TLS, chunked decoding and items-view leaks. multidict>=6.9.1,<7 for transitive async and TLS dependencies. anyio>=4.14.2 and cryptography>=50.0.0 as further security floors.

The protobuf entry is the most interesting because of the reasoning. The comment says to keep protobuf off the 6.33.x line flagged by BDSA for the ParseDict recursion-depth follow-up, that 7.x is outside the published affected range, and that the latest compatible patch should be used to avoid scanner version-map ambiguity. The pin is protobuf>=7.36.2,<8.

Then setuptools>=83.0.0,<84 with a comment referencing CVE-2026-59890 and noting that PyTorch's own cap was unnecessary.

What this block shows is a project that tracks advisories actively and pins accordingly, with a comment for each pin explaining the reason. That is a very different picture from the version numbers alone.

Licensing, compliance and how the third-party code is handled

The licence section is short and, for a project with this much vendored native code, unusually careful.

FlashDreams is released under the Apache License 2.0, and third-party components and their licences are listed in THIRD-PARTY-NOTICES and NOTICE. The repository is REUSE-compliant, with REUSE.toml and a LICENSES/ directory.

REUSE is the SPDX based licensing standard, and the SPDX header in the README makes it concrete: the file carries a copyright line for NVIDIA CORPORATION & AFFILIATES with SPDX-License-Identifier: Apache-2.0. That combination, a per-file SPDX identifier plus a LICENSES/ tree, means the licence of every file in the repository is machine readable, which is what an organisation that has to answer a licence audit for its dependencies wants to see.

Two specific cases are called out. The independently packaged Ludus renderer, at integrations_v2/omnidreams/impl/ludus-renderer, retains its own Apache licence file. The upstream cudaraster terms, in the ludus renderer's _cpp/cudaraster directory, and per-file exceptions are described in the third-party notices.

The renderer is worth pausing on, because it explains the shape of the whole project. A renderer that draws a world model output into a frame, with its own CUDA rasterisation code, is the layer between the model's tensors and the screen. It is independently packaged because it has its own dependency footprint and its own licence, and it sits inside the omnidreams integration rather than in the core package.

The repository listing classifies the licence as a custom one that GitHub cannot classify, which is a metadata limitation rather than a disagreement: the README badge, the LICENSE file and the SPDX headers all say Apache 2.0. The practical difference from a standard Apache 2.0 project is the REUSE apparatus and the third-party notices tree, and for a company shipping this internally the notices file is the part that matters.

This is a licensing note rather than legal advice, and the point to check is whether THIRD-PARTY-NOTICES covers the components you actually deploy.

Where FlashDreams is the wrong tool

The first limit is that this is a runtime, not a model, and not a training framework. There is nothing here to train a world model, and the repository layout, with the flashdreams/ package, the integrations_v2/ plugins and the apps/, has no training loop in it. If you need to fit a model, you are looking at a different repository.

The second limit is the hardware and software floor. The dependency overrides tell you this is a CUDA world: transformer-engine, cuBLAS with a forced version, cudaraster inside the renderer. A runtime that overrides cuBLAS to resolve an ABI conflict with a specific torch version is not going to run on a machine without an NVIDIA GPU, and the pinned combination of torch, transformer-engine and cuBLAS is a moving target that you inherit rather than choose.

The third is that the platform is young. It grew out of a demo for GTC 2026, the repository publishes no GitHub releases, and the version you get is the default branch. The upstream Omniverse Discord channel and the #flashdreams channel are where the project talks to its users, and a project with an active Discord and no release tags is one where you are expected to work against the tip.

The fourth is a subtle one about the integration model. The workspace treats each populated directory under integrations_v2/ as a standalone external integration repo, which is good for isolation and bad for you, because your plugin has to track the core's API and the core moves. The README's own description of the two plugin families, causal forcing and self forcing, suggests the model conditioning interface is still in flux, and the OmniDreams integration with its independently packaged renderer is the deepest part of the tree.

The alternative to weigh is building directly on the model framework. If your world model is a single model you control and your latency budget allows a straightforward serving loop, writing the inference server yourself on top of the framework gives you a much smaller dependency surface and no version conflicts with someone else's pinned cuBLAS. The difference in approach is that FlashDreams is a general platform with plugin isolation, multiple model families and a serving API, while a direct implementation is one model, one process and your own scheduling.

If you are building the interactive driving or robotaxi application the runtime was written for, that trade goes the other way and you would be rebuilding scheduling, streaming and integration plumbing that already exists.

Maintenance, contribution and how the project asks you to work with it

The maintenance picture is current. The last push was on 2026-10-20 and the repository is not archived. The repository has no GitHub releases, which for a platform that is distributed as source plus containers is a reasonable choice, and the docker/ directory plus the docker-compose configuration in the repository suggests that is the intended distribution path.

The contribution guidance is explicit about what is wanted. The CONTRIBUTING.md file covers how to contribute, and the README says new integrations, bug reports, feature requests, performance tuning, and documentation edits are all welcome. That list is revealing about where the project sees its own gaps. Performance tuning being listed alongside bug reports tells you the runtime is expected to be tuned per deployment rather than being one size fits all, and new integrations being first on the list tells you the model ecosystem is the part that is still growing.

Defects and feature requests go to GitHub Issues, and there is a separate coordinated disclosure process in SECURITY.md for potential security vulnerabilities, which is standard for a project with a Discord presence and a security-conscious dependency posture.

There is a CODE_OF_CONDUCT.md, and the community is on the NVIDIA Omniverse Discord with a dedicated #flashdreams channel linked in the README.

The citation request is worth a word. The README asks you to cite the project if it is useful in your research or product, and provides a BibTeX entry for FlashDreams itself alongside one for the OmniDreams paper with the arXiv identifier 2606.03159. For a platform rather than a model, asking for citation is a sign that the authors expect it to be used as a reference point in work rather than borrowed from quietly.

One more practical note for anyone evaluating it. The README's own pointer for setup is the quick-start guide, and the README does not itself give installation commands. The repository is where the code lives and the documentation site is where the instructions are, which is the arrangement you would expect for a project whose installation depends on a specific CUDA and GPU stack rather than a pip install.

Editorial conclusion

Adopt FlashDreams if you are building an application that runs a world model interactively and need the serving layer rather than just the model, since that is what this library provides. Do not adopt it if you are looking for training code or for a self-contained model implementation, because the repository is the runtime around a model rather than the model itself. Verify first which of the two APIs your work needs, whether your GPU and CUDA stack meet the runtime's requirements, and how the integration plugins under integrations_v2/ are versioned against your own model code.

Frequently asked questions

Official sources

1. Issues
2. NVIDIA/flashdreams on GitHub
3. Project website
4. README