# Wall-X: training and inference code for the WALL embodied models

> Wall-X is the open source training and inference stack for the WALL series of embodied foundation models from X-Square Robot, with LeRobot data preparation, flow matching and FAST action branches, a WebSocket serving runtime and CUDA operators compiled during install.

**X-Square-Robot/wall-x** — Building General-Purpose Robots Based on Embodied Foundation Model

- Repository: https://github.com/X-Square-Robot/wall-x
- Website: https://x2robot.com/en/research
- Stars: 1,313 · Forks: 111
- Language: Python
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/x-square-robot-wall-x

## A repository for the WALL series of embodied models

The stated aim is to build embodied foundation models that capture continuous, high-fidelity physical interaction, with a direct feedback loop between model decisions and the body's lived experience. The repository is the training and inference code for the WALL open-source embodied foundation models, and it states plainly what is inside: LeRobot data preparation, model configuration, flow matching and FAST action branches, public serving and evaluation utilities, and exported CUDA operator sources that compile during package installation. The model weights themselves are not in the repository. Four checkpoints are published on Hugging Face under the x-square-robot organisation, WALL-OSS-0.5, WALL-OSS-FLOW, WALL-OSS-FLOW-0.1 and WALL-OSS-FAST, so the code and the weights are handled separately, which is the usual arrangement for a release of this size. The current workspace guide targets WALL-OSS-0.5; it directs users of WALL-OSS-FLOW or WALL-OSS-FAST to check out the earlier commit specified in that guide.

## The install compiles CUDA, so pip alone will not do it

This is the part that separates Wall-X from a normal Python model repository. The exported CUDA operator sources live under `wall_x/model/core/ops/csrc/`, and `setup.py` builds them with PyTorch `CUDAExtension` when Wall-X is installed, compiling a binding file plus the other `.cu` sources into an extension module with optimised compiler flags for both host and device code. The build fails with an explicit error if the sources are missing from the export. Two knobs matter. `ninja` is included in the requirements so the compile can run in parallel, and `MAX_JOBS` controls that parallelism. `--no-build-isolation` is required, because with isolation the build would use a different torch than the one installed in your active environment, and the extension has to be compiled against that exact torch.

## The environment recipe is fixed and its order matters

The setup sequence starts from a conda environment on Python 3.10.

```bash
conda create --name wallx python=3.10
conda activate wallx
```

The requirements are then installed, and flash-attn is built for the GPU you actually have, which is why the arch list is computed from the device rather than hard coded.

```bash
pip install -r requirements.txt
export FLASH_ATTEN_CUDA_ARCHS=$(python -c 'import torch; print(f"{torch.cuda.get_device_capability()[0]}{torch.cuda.get_device_capability()[1]}")')
MAX_JOBS=4 pip install flash-attn==2.8.3 --no-build-isolation
```

DMuon, which the default training configs use, is installed straight from the maintainers' GitHub repository, and LeRobot is cloned and installed in editable mode without its dependencies, so it does not override the versions already pinned by `requirements.txt`. The repository itself is installed last, editable and without build isolation.

```bash
MAX_JOBS=8 pip install --no-build-isolation -e .
```

## Training runs from a YAML config through an FSDP trainer

Training is configuration driven, and the details live in the workspace README rather than in the top level file. That page covers the training script configuration, the GPU setup, the model and data paths, the robot degrees of freedom configuration and the training hyperparameters. The documented procedure is to download the pretrained checkpoint, copy the example template `workspace/example/lerobot/qwen2_5_lerobot_template.yml`, replace the placeholder paths and launch the trainer with the config path. Replace the example path below with the path to your own configuration.

```bash
python -m wall_x.trainer.fsdp_trainer.train_fsdp --config /path/to/config.yml
```

The module path tells you the shape of the thing, an FSDP trainer inside the `wall_x.trainer` package, which means sharding across devices is the default assumption rather than an option you turn on. For Wall-OSS-0.5 fine-tuning, normalization, LIBERO evaluation and open-loop WebSocket evaluation, the same workspace README holds the instructions, so the actual procedure for the released checkpoint is split across two documents.

## Inference comes with a minimal script and a WebSocket server

The smallest end to end example is a script called `fake_inference.py`, which loads the Wall-OSS model with `Qwen2_5_VLMoEForAction.from_pretrained()`, prepares proprioceptive inputs, attention masks and dataset specs, runs inference in validate mode at bfloat16, and then validates the output shape and checks numerical stability. That is a good first thing to run, because it exercises the model and the data path without a robot attached. For real use there is a WebSocket server wrapper. Replace the example paths with your own checkpoint and configuration; bind to localhost or otherwise secure access if the server should not be reachable on the network.

```bash
bash scripts/run_serving.sh \
  --checkpoint-path /path/to/checkpoint \
  --train-config-path /path/to/config.yml \
  --port 32195
```

The wrapper has no built-in checkpoint path, so it fails loudly rather than guessing, and by default it returns raw model action chunks, which suits open-loop evaluation; clients expecting robot-serialized actions pass `--serialize-actions`. The helper scripts are public and are referenced from the repository root.

## Three evaluation paths, one of them optional

Closed-loop simulator evaluation has convenience launchers under `scripts/`, with LIBERO setup described as optional and documented with its own helper scripts and a separate `requirements-libero.txt`, so you can skip the simulator entirely if you are not using it. Replace the example paths with your own files before running the commands.

```bash
bash scripts/run_libero.sh /path/to/checkpoint
```

Open-loop evaluation compares what a running server predicts against LeRobot ground truth, and produces a plot.

```bash
python scripts/draw_openloop_plot.py \
  --uri ws://127.0.0.1:32195 \
  --dataset-root /path/to/lerobot_dataset \
  --train-config /path/to/config.yml \
  --episode-indices 0,1,2
```

The episode indices are a comma separated list, so you can look at a couple of episodes before committing to a full sweep. The scripts directory also carries utilities for computing normalization statistics, merging sharded weights and merging the tokenizer, which are the practical chores of working with a distributed checkpoint.

## A release history told through papers and version numbers

The news section of the README reads as a release log. In September 2025 the project introduced WALL-OSS, an end to end embodied foundation model built on large-scale multimodal pretraining. In May 2026 came WALL-OSS-0.5, described as a deployment-ready vision-language-action model with gradient-bridged pretraining for real robot manipulation, and WALL-WM, a world action model that couples future-video imagination with action prediction at semantic event boundaries. In June 2026, Wall-X 1.1.0 updated the open-source training and inference stack for Wall-OSS-0.5, adding the public serving and evaluation runtime, DMuon training support and install-time CUDA operator builds. The package version in `setup.py` is 1.1.0, the licence is Apache-2.0, and the last push was on 2026-09-18, so the stack is still moving even though there are no tagged GitHub releases.

## Conclusion

Wall-X is the right codebase for a team that wants to fine-tune or serve an open vision-language-action model on real robot data, because it ships the data preparation, the training entry point, the serving wrapper and the evaluation scripts in one repository. It is the wrong choice for a machine without an NVIDIA GPU, since the install compiles CUDA extension sources and pins torch, transformers and flash-attn versions. Before you start, read the configuration notes in workspace/README.md and copy the example YAML, because the training script takes a config path and the placeholder paths inside it have to be replaced with your own checkpoint and dataset locations.

## FAQ

### What does the Wall-X repository contain?

Training and inference code for the WALL open-source embodied models, including LeRobot data preparation, model configuration, flow matching and FAST action branches, serving and evaluation utilities, and CUDA operator sources compiled at install time.

### Why does installing Wall-X need --no-build-isolation?

The install compiles CUDA extension sources against the torch already present in your environment, and build isolation would compile against a different torch. The CUDA sources must also be present in the export, since setup.py raises an error when the binding file is missing.

### How do I start a Wall-X WebSocket server?

Run the serving wrapper with a checkpoint path, a training config path and a port. The wrapper has no built-in checkpoint path, returns raw model action chunks by default, and needs the `--serialize-actions` flag for clients that expect robot-serialized actions.

### Where are the Wall-X training configuration details?

In the workspace README, which covers the training script configuration, GPU setup, model and data paths, robot degrees of freedom configuration and hyperparameters, alongside an example YAML template to copy and edit.

## Sources

- [Issues](https://github.com/X-Square-Robot/wall-x/issues)
- [License: Apache-2.0](https://github.com/X-Square-Robot/wall-x/blob/main/LICENSE)
- [Project website](https://x2robot.com/en/research)
- [README](https://github.com/X-Square-Robot/wall-x/blob/main/README.md)
- [X-Square-Robot/wall-x on GitHub](https://github.com/X-Square-Robot/wall-x)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/x-square-robot-wall-x