dexmal/opendw and DW05: a world model for robot action, video and value

What DW05 is meant to decide, and for whom

DW05 targets a specific question: given a robot, a language instruction and a stream of camera frames, what happens next, and what action should be taken. The README describes a multimodal world model for embodied decision-making that unifies future video prediction, action generation and state-value estimation in one framework. Those three outputs are usually separate systems. A video model imagines the future, a policy emits actions, and a value function scores states. DW05 puts them behind one backbone and three expert heads.

The intended reader is a research or robotics engineer who already has episode data in a RobotWin-style format and wants to train or evaluate a world-action model on it. This is not an end-user application. The pyproject.toml classifies the package as Development Status 4 - Beta, Intended Audience Science/Research, and the dependency list includes deepspeed, hydra-core, wandb and decord, which is the shape of a training stack rather than a library you import into a product. If you want a pretrained policy you can call from a service, the released artifacts are a base checkpoint and a Robotwin 2.0 fine-tune, and you still supply the data pipeline.

The MoT backbone and its three expert heads

The README states that DW05 follows an MoT framework where a Wan backbone is followed by three expert heads for video, action and value modeling. Inputs are language, image or video, robot type, state and action. The Value Expert is explicitly deferred: the README notes it will be updated in a later release. That means the architecture diagram shows three heads, but the released weights and code cover video and action modeling. If your evaluation depends on value estimates, you are reading a design document, not a shipped capability.

Data flows from episode annotations. Each episode is a JSONL file where every line describes one frame, with frame_idx, image paths and robot state updated per frame. A sample declares its type as action, wm, or both. Action samples need a robot prompt at robot.prompt; video samples need a world-model caption at worldmodel.caption. Camera observations are referenced by images_1, images_2 and images_3, or by equivalent fields you configure through data_config.images_keys. Text embeddings are read from a cache directory, with a configured fallback for missing embeddings, and normalization statistics live in norm_stats.json. The dataset can read local paths or remote paths through megfile, with local mirrors configured by path-prefix overrides. That mirror mechanism matters: if your media lives in object storage, you can point DW_LOCAL_MIRROR_PREFIXES at a local copy instead of downloading on every run.

Installing dexbotic-dw05 and running a first smoke task

The package name is dexbotic-dw05 and the README recommends Python 3.10+ with a CUDA-enabled PyTorch build. Install the repository in editable mode from its checkout:

cd /path/to/dexbotic-dw05
pip install -e .

Optional attention packages are installed separately, and the README warns they must be compatible with your CUDA and PyTorch stack:

pip install -e '.[attention]'

Weights come from Hugging Face. The README gives two checkpoints: Dexmal/DW05-Base, pretrained on a multi-source dataset, and Dexmal/DW05-Robotwin, fine-tuned on Robotwin 2.0 for evaluation.

huggingface-cli download Dexmal/DW05-Base --local-dir ./checkpoints/DW05-Base
huggingface-cli download Dexmal/DW05-Robotwin --local-dir ./checkpoints/DW05-Robotwin

Three runtime environment variables are listed as common settings. DW05_MODEL_BASE_PATH points at the local model root, DIFFSYNTH_MODEL_BASE_PATH mirrors it, and TOKENIZERS_PARALLELISM is set to false to avoid tokenizer fork warnings.

export DW05_MODEL_BASE_PATH=/path/to/local/model/root
export DIFFSYNTH_MODEL_BASE_PATH=$DW05_MODEL_BASE_PATH
export TOKENIZERS_PARALLELISM=false

Dataset paths are also environment variables. The built-in recipe is robotwin_baseline, registered in dexbotic/data/dataset/dw05/data_source.py.

export DW05_ROBOTWIN_ANNOTATIONS=/path/to/dw05_demo/annotations
export DW05_ROBOTWIN_TEXT_EMBED_DIR=/path/to/dw05_demo/text_embeddings
export DW05_ROBOTWIN_NORM_STATS_PATH=/path/to/dw05_demo/norm_stats.json
export DW05_ROBOTWIN_DATA_PATH_PREFIX=/path/to/media/root
export DW05_ROBOTWIN_INDEX_PATH_PREFIX=/path/to/dw05_demo/annotations

With those set, the README's first real run is a smoke task through the example entry point. You can either use the built-in recipe or bypass it with explicit overrides, which is the safer path when your directory names differ from the demo layout.

python playground/example_dw_exp.py --task smoke \
  data_config.annotations=/path/to/annotations \
  data_config.data_path_prefix=/path/to/media/root \
  data_config.index_path_prefix=/path/to/index/root \
  data_config.text_embedding_cache_dir=/path/to/text_embeddings \
  data_config.norm_stats_path=/path/to/norm_stats.json

If your action data has no normalization statistics yet, the same script computes them. The README shows the output path as ./runs/dw05/norm_stats, with batch_size and max_batches controls.

python playground/example_dw_exp.py --compute-norm-stats \
  data_config.annotations=/path/to/annotations \
  data_config.data_path_prefix=/path/to/media/root \
  data_config.text_embedding_cache_dir=/path/to/text_embeddings \
  norm_stats_config.norm_save_path=./runs/dw05/norm_stats \
  norm_stats_config.batch_size=128 \
  norm_stats_config.max_batches=500

What you should see is a completed smoke run and, for the second command, a norm_stats file under ./runs/dw05. The README does not document expected loss values or runtime, so treat a clean exit as the signal.

Where DW05 breaks: data layout, missing value head and media access

The sharpest limitation is stated in the README itself: the Value Expert will be updated in a later release. Any workflow that assumes state-value estimation from the released code is premature. The architecture diagram and the description promise it; the shipped artifacts do not include it.

The second constraint is the data contract. A full training sample needs camera observations, a type list, a robot prompt for action samples, a world-model caption for video samples, robot state, optional proprio, action trajectories and cached text embeddings, plus optional normalization statistics. The README is explicit that type, robot.prompt and robot.state are sufficient for action-only norm-stat computation, but that full training samples should include camera observations and cached text embeddings for the selected prompts and captions. If your dataset stores frames as images rather than videos, each video item must be replaced with an image item such as {"type":"image","url":"images/front/frame_000000.jpg"}, which means a conversion pass before training starts.

Media access is the third failure mode. The dataset reads local or remote paths through megfile, and mirrors are configured with DW_LOCAL_MIRROR_PREFIXES using the platform path separator. If your annotations point at remote URLs that are not reachable and no mirror root is set, the loader has nothing to fall back on. The README does not document a retry policy or a timeout for remote reads, so a slow object store can surface as a stalled run rather than a clear error.

Finally, the dependency set is heavy and version-pinned in places: numpy is pinned to 1.26.4, boto3 to 1.35.18, botocore to 1.35.66, and torch must be at least 2.6.0. Installing into an existing environment with a different numpy or boto3 pin is likely to conflict, and the optional attention extras are described as installable only when compatible with your CUDA and PyTorch stack.

How DW05 differs from a plain video-prediction or policy model

The nearest alternative approach is to keep the three capabilities separate: a video diffusion model for future frames, a policy trained on action trajectories, and a learned value function. That is the conventional stack, and it has an advantage DW05 gives up. Each component can be trained, versioned and replaced independently, and a broken value head does not block action generation. DW05 instead shares a Wan backbone across the heads and trains them jointly, which is the reason the README can describe one framework producing video, action and value. The cost is coupling: the deferred Value Expert holds back part of the promised output, and the training recipe is tied to one annotation format.

A second comparison is with general-purpose video generation models. Those take text and produce video, but they have no notion of robot type, robot state or action trajectories, and no action head. DW05's sample schema carries robot.prompt, robot.state and an action trajectory alongside the caption, and the type field distinguishes action samples from world-model samples. That schema is the actual difference: DW05 is built to be conditioned on embodiment, not just on text. Whether that conditioning pays off is a question the released weights are meant to answer, and the repository gives you the training code to test it on your own episodes.

Licence, maintenance and the cost of upgrading

DW05 is released under Apache-2.0, declared in the LICENSE file and in pyproject.toml as License :: OSI Approved :: Apache Software License. The README carries the same badge. Apache-2.0 is permissive and includes an explicit patent grant, which matters for a model you may fine-tune and redistribute. The licence covers the repository; it does not automatically settle the terms of the Hugging Face checkpoints, so check the model cards for Dexmal/DW05-Base and Dexmal/DW05-Robotwin before you ship a derived model. That is a factual boundary, not legal advice.

On maintenance, the repository is not archived, and the last push was on 2026-07-09. The README lists a single update entry: the initial DW05 public release with weights and inference code in 2026-07. There are no retrieved releases, so versioning currently runs off the 0.1.0 value in pyproject.toml and the git history. Upgrading therefore means pulling the repository and re-running pip install -e ., then re-checking the pinned dependencies against your environment. Because the README states the Value Expert is coming in a later release, an upgrade is also the moment to re-read the architecture notes: a new expert head can change checkpoint expectations, and the repository does not document a migration path between checkpoint versions.

Editorial conclusion

DW05 is worth adopting if you already have RobotWin-style episode annotations, a CUDA-enabled PyTorch 2.6+ environment and a reason to train or evaluate a joint video-and-action model; the base and Robotwin checkpoints on Hugging Face let you start from released weights rather than from scratch. It is the wrong tool if you need a value model today, because the README states the Value Expert will be updated in a later release, or if you want a single pip package that runs without a dataset layout. Before committing, verify that your annotations carry the images_1, images_2 and images_3 keys or that you can remap them through data_config.images_keys, and confirm that your CUDA stack can build the optional flash_attn and xformers extras.

Frequently asked questions

Official sources

1. dexmal/opendw on GitHub
2. Issues
3. License: Apache-2.0
4. README