LightZero: one harness for MCTS plus reinforcement learning

A benchmark spanning Atari, board games and classic control

LightZero sets out to standardize a family of search based reinforcement learning algorithms that otherwise live in separate codebases, and it describes itself as a unified benchmark for Monte Carlo tree search in general sequential decision scenarios. Scope is legible in the repository topics rather than in prose: alpha-beta-pruning, alphazero, efficientzero, gumbel-muzero, sampled-muzero, stochastic-muzero, self-play, muzero, gym, pytorch, reinforcement-learning, alongside atari, gomoku, tictactoe, board-game and continuous-control. That combination fixes the audience. A researcher comparing two search algorithms wants one harness, one set of environments and numbers they can place side by side. A team that wants a packaged agent to drop into a service wants something else, and this repository does not set out to be that. Four papers attached to the project follow the same line: LightZero at the NeurIPS 2023 Datasets and Benchmarks track, UniZero, ReZero and ScaleZero.

The search loop is Cython, and that shows up in packaging

The efficiency claim is narrow and points at one place: mixed heterogeneous computing for the most time consuming part of the MCTS algorithms. Packaging backs that claim more concretely than prose does. setup.py imports cythonize from Cython.Build, collects the numpy include directory, reads the Python include path through get_python_inc, and on darwin appends a Homebrew framework include directory when that path exists on disk. pyproject.toml lists Cython and numpy among its build requirements for the same reason. So the inner tree search loop ships as compiled extension code rather than Python bytecode, and every algorithm built on that loop inherits the same compilation cost. One consequence sits ahead of the features: there is no pure Python install path, and a machine whose toolchain is incomplete fails during installation instead of during training.

Building the container and running the cartpole MuZero config

The Dockerfile is the shortest documented route to a working environment. Its header says to move the file into a new empty directory before building, so the daemon does not receive the rest of the tree, and it installs the interpreter and compiler toolchain before cloning the repository in editable mode:

FROM ubuntu:20.04
WORKDIR /opendilab
RUN apt-get update && \
    apt-get install -y python3.8 python3-pip gcc g++ swig git && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Three commands then produce an interactive container. The tag name is fixed in the Dockerfile comments, and the last one is the first thing worth running, because it exercises the compiled search code and the config loader together:

docker build -t ubuntu-py38-lz:latest -f ./Dockerfile .
docker run -dit --rm ubuntu-py38-lz:latest /bin/bash
python ./LightZero/zoo/classic_control/cartpole/config/cartpole_muzero_config.py

Read the path closely. `zoo/` holds the per environment configuration, `classic_control/cartpole/config/` holds this task's YAML files, and one file names the algorithm in its filename, so `cartpole_muzero_config.py` is the MuZero configuration for CartPole. A note in the Dockerfile header adds that the working directory inside the image is `/opendilab`, so no directory change is needed before running the script.

Four architectures build, and no wheel is tested

pyproject.toml configures cibuildwheel across three platforms, and the architecture lists are worth copying into your own planning. Windows targets `x86` and `AMD64`, macOS targets `x86_64` and `arm64`, Linux targets `x86_64` and `aarch64`. A global block skips `pp*` so no PyPy wheel is built, and the Linux block also skips `pp* *musllinux*` because the dependencies do not build against musl. More telling is what sits directly underneath. In all three platform blocks, `before-test` and `test-command` are commented out, with the pytest invocation and the file copy steps left in place but disabled. Wheels are therefore produced for those four architectures without any unit test running inside the build. A broken Cython extension on an untested combination surfaces at import time, long after the install has reported success.

A flat requirement list reaches far past the search code

requirements.txt is a single flat list, and setup.py loads every `requirements-*.txt` file it finds in the project root into package extras, so the extras exist but the runtime dependencies do not use them. That list is wider than a tree search library would need: `DI-engine>=0.5.3`, `gymnasium[atari]>=1.0.0`, `ale-py>=0.10.1`, `numpy>=1.24.1,<2`, `transformers`, `pympler`, `minigrid`, `moviepy`, `pytest`, `line_profiler`, `xxhash`, `einops`, `openai`, `nltk`, `pyecharts`, `numba`, and `simple_parsing` for the Jericho environment. Installing LightZero therefore pulls an LLM client and a video library into an environment whose hot path is Cython tree search. One real contradiction sits in this file: pyproject.toml requires `numpy>=1.18.0` at build time while the runtime list pins `numpy>=1.24.1,<2`, and the extension code compiles against whichever numpy headers are present when the wheel is built.

The v0.2.0 header date and the tag behind it disagree

The README banner reads `Updated on 2026.03.11 LightZero-v0.2.0`, but the v0.2.0 tag on the repository dates from 2025-04-09, and the earlier visible tags are v0.1.0 from 2024-07-12 and v0.0.5 from 2024-04-16. Read together, the banner marks a documentation refresh rather than a new build, so anyone pinning v0.2.0 expecting March 2026 code gets the April 2025 tag. News entries tell a similar story from the other direction: it claims v0.0.1 was officially released in 2023.04, yet no tag for that version appears in the visible tag history. The project is not archived and the last push was on 2026-08-28, so changes continue to land on main. A `CHANGELOG.md` sits in the repository root beside `docs/`, `assets/` and a `Makefile`, which is where a maintainer would record which of those movements belongs in which build.

Where the benchmark promise exceeds the documented surface

Stated purpose is a performance comparison of all implemented algorithms under one framework, and the Benchmark and Awesome-MCTS sections are linked from the table of contents. Visible README text does not reproduce those numbers, so the comparison cannot be checked without building the tasks in `zoo/` and producing the numbers yourself. The README also does not document rollback, does not pin exact dependency versions beyond the numpy ceiling, and does not state which Python versions are supported beyond the 3.8 that the Dockerfile installs. Combined with the disabled wheel tests, that makes the project a poor fit for anyone who needs a guaranteed reproducible environment. It is also the wrong tool when a single tuned policy matters more than the comparison, since the emphasis is on breadth of algorithms rather than on a finished agent.

DI-engine and hand-rolled MuZero as the two alternatives

One alternative is already in the dependency tree. `DI-engine` is the base reinforcement learning library LightZero is built on, and a project doing model free work has no reason to take on the search stack, the Cython build and the Atari and board game environments along with it. Another route is the one most researchers start from: implement EfficientZero or Gumbel MuZero from the papers and control every line of the search yourself. What LightZero buys against that route is a shared harness, per environment configuration under `zoo/` with the algorithm named in the filename, and documentation diagrams for each integrated algorithm. What it costs is the flat requirement list and the fact that a source install compiles extensions on your machine. Benchmark breadth is the reason to choose it; the build is the reason to time the choice.

Editorial conclusion

LightZero earns a place in a research group that needs to run two search based reinforcement learning algorithms under one harness, and it is the wrong choice for a team that wants a tuned agent to ship this quarter. Verify two things before committing: build the Cython extensions on your own architecture, since every test command in pyproject.toml is commented out, and read the benchmark table in `zoo/` yourself instead of taking the framing on trust. If the build fails on musl or PyPy, stop there and keep the rest of your stack on `DI-engine`.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. opendilab/LightZero on GitHub
3. Project website
4. README
5. Releases