Library / SDK
thu-ml/tianshou avatar
thu-ml/tianshou

Tianshou 2 reorganises the class hierarchy around Algorithm and Policy

An elegant PyTorch deep reinforcement learning library.

11,054 stars1,349 forksPythonMIT

At a glance

What is it?
Tianshou is a deep reinforcement learning library built on pure PyTorch and Gymnasium, and its second major version is not backwards compatible: learning algorithms and policies became separate abstractions and the class tree was rebuilt to separate on-policy, off-policy and offline methods at the type level. The breadth is real, the packaging carries two carved-out PyTorch versions and a hand-maintained list of Gymnasium optional environments, and the last push landed on 2026-04-03.
Who is it for?
Tianshou 2 suits researchers who want to write their own algorithm against a separated Algorithm and Policy interface, and who can accept a Python 3.11 floor and a library that is still labelled Beta by its own classifiers.
Can I use it commercially?
Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Activity is slowing. The repository last received commits 7 months ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • thu-ml/tianshou11,054 ★ · +64
Star history
Repository2026-09-222026-11-15
thu-ml/tianshou11k11.1k

64 Stars in the last 54 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

Version 2 is not backwards compatible, and the class tree was rebuilt

The change note at the top of the README is blunt about scope: version 2 is a complete overhaul of the software design of the procedural API, and because of the extent of the changes it is not backwards compatible with previous versions. Three things changed structurally. Learning algorithms and policies are now separated into distinct abstractions called `Algorithm` and `Policy`. Parameter interfaces were redefined and documented extensively, with some parameters renamed so the names read consistently. And the class hierarchy was fully revised to separate on-policy, off-policy and offline algorithms at the type level, with inheritance relationships intended to be meaningful. The last point is the one that bites hardest. The migration information is in CHANGELOG.md. Meanwhile pyproject.toml still classifies the package as `Development Status :: 4 - Beta`, so a version 2 library is being labelled pre-stable by its own metadata.

Python 3.11 is the floor and two PyTorch versions are carved out

The dependency block is more opinionated than the README prose suggests. Python is pinned as `^3.11`, so anything older is out, and the only interpreter classifier is `Programming Language :: Python :: 3.11`. Torch is declared as `^2.0.0, !=2.0.1, !=2.1.0`, with an inline comment pointing at a PyTorch issue numbered 100974 for the 2.0.1 exclusion. Two consequences follow. First, a resolver cannot install this library against torch 2.0.1 or 2.1.0 at all, so if your CUDA setup or another dependency has settled on 2.1.0, installing Tianshou means changing your torch version, not just adding a package. Second, the supported surface is a range with holes, which is why the statement that Tianshou is based on pure PyTorch needs reading alongside that exclusion list rather than as a blanket claim. NumPy is `>=1.24.4`, Gymnasium is `>=0.28.0`, pandas is capped below 3.0, and numba is required at `>=0.60.0`. The virtualenv pin is the odd one out, since it excludes 20.4.5 and 20.4.6 and drops to `<20.16.4` under a `sys_platform == 'win32'` marker, with a comment calling it special sauce for a flaky Poetry bug on Windows.

Gymnasium optional environment groups are re-declared by hand

There is a comment in pyproject.toml that explains an unusual dependency pattern. Atari, box2d, classic-control and mujoco environments are all optional dependencies of Gymnasium, and Tianshou cannot expose extras that rely on several optionals of another package because of a Poetry issue, cited as issue 7911. So the project maintains its own list of those dependencies instead of forwarding Gymnasium's extras. The effect on you is that installing a Tianshou extra does not give you whatever Gymnasium would have resolved. It gives you Tianshou's hand-picked versions. That is a defensible engineering choice. The rest of the required set is small and mostly instrumentation: deepdiff, overrides, h5py, matplotlib, packaging, tensorboard, tqdm and sensai-utils, plus pettingzoo at `^1.22`, which is the multi-agent surface, and it is there to support a feature the README labels experimental.

The examples directory is the reference, and the wheel does not ship it

At the top level, the examples directory is organised by environment family: atari/, box2d/, discrete/, inverse/, modelbased/, mujoco/, offline/ and vizdoom/, alongside an __init__.py. Read as a map of the library, the directory names line up with the claims: offline/ is where the offline algorithms are wired, modelbased/ corresponds to the experimental model-based support. Now the packaging catch. pyproject.toml sets `exclude = ["test/*", "examples/*", "docs/*"]`, so test/, examples/ and docs/ are all kept out of the built distribution. If you install Tianshou from PyPI you get the library and none of those directories. To read the example for offline RL or model-based wiring, you have to clone the repository, which puts you on whatever is on the default branch rather than the version you installed. The distribution is built with poetry-core as the backend, and poetry.lock is committed, so a lock-based install of the source tree is the reproducible path, while the examples you need for reference only ever exist on the branch.

The speed claims rest on vectorized sampling and numba just-in-time compilation

Three separate mechanisms carry the performance side. Vectorized environments are supported for all algorithms, in both synchronous and asynchronous modes, and there is a second path for super-fast vectorized environments based on EnvPool, also for all algorithms. Separately, n-step return estimation and prioritized experience replay are supported for all Q-learning based algorithms, and GAE, nstep and PER are described as highly optimized thanks to numba's just-in-time compilation. That last clause has a practical edge. Just-in-time compilation means the speed exists only after the first call has compiled, so the first batch of a run pays a compilation cost that later batches do not, and a short debugging run can look slow for reasons that have nothing to do with your algorithm. It also explains why numba is a required dependency at `>=0.60.0` rather than an extra,. The benchmark claim itself is scoped and worth quoting precisely: state-of-the-art results in MuJoCo benchmarks for REINFORCE, A2C, TRPO, PPO, DDPG, TD3 and SAC. That is seven algorithms on one suite, not a statement about Atari, the offline algorithms, or the multi-agent path.

The Dockerfile forces linux/amd64 and installs into the system interpreter

The container definition is worth reading before you build on it, because two of its choices are deliberate and one is inherited from Codespaces:

dockerfile
# Use the official Python image for the base image.
FROM --platform=linux/amd64 python:3.11-slim

# Set environment variables to make Python print directly to the terminal and avoid .pyc files.
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1

# Install system dependencies required for the project.
RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    build-essential \
    git \
    wget \
    unzip \
    libvips-dev \
    gnupg2 \
    && rm -rf /var/lib/apt/lists/*


# Install pipx.
RUN python3 -m pip install --no-cache-dir pipx \
    && pipx ensurepath

# Add poetry to the path
ENV PATH="${PATH}:/root/.local/bin"

# Install the latest version of Poetry using pipx.
RUN pipx install poetry

# Set the working directory. IMPORTANT: can't be changed as needs to be in sync to the dir where the project is cloned
# to in the codespace
WORKDIR /workspaces/tianshou

# Copy the pyproject.toml and poetry.lock files (if available) into the image.
COPY pyproject.toml poetry.lock* README.md /workspaces/tianshou/

RUN poetry config virtualenvs.create false
RUN poetry install --no-root --with dev

# The entrypoint will perform an editable install, it is expected that the code is mounted in the container then
# If you don't want to mount the code, you should override the entrypoint
ENTRYPOINT ["/bin/bash", "-c", "poetry install --with dev && poetry run jupyter trust notebooks/*.ipynb docs/02_notebooks/*.ipynb && $0 $@"]

`--platform=linux/amd64` is declared explicitly, so on an ARM machine or an ARM server the image runs under emulation rather than natively, which costs speed you will attribute to the library. `poetry config virtualenvs.create false` followed by `poetry install --no-root --with dev` puts every dependency, dev group included, into the image's own interpreter, so the shipped image carries test and tooling packages. And the working directory is pinned to `/workspaces/tianshou` with a comment saying it cannot be changed because it has to stay in sync with the Codespaces clone path. The entrypoint trusts notebooks on every start, which rewrites their signatures each time.

Six months of quiet after v2.0.1, so pin exactly

The release history is short and recent. v2.0.0b2, tagged as a beta release, went out on 2025-10-22. v2.0.0 followed on 2025-12-01, and v2.0.1 on 2026-04-02, which matches the version in pyproject.toml. The last push to the default branch, master, is dated 2026-04-03, one day after that release. Between them the picture is a library that shipped a major rewrite in December, a patch in April, and has had no commits since. That is not a criticism so much as a planning input. Nobody is shipping compatibility fixes for new Gymnasium or torch releases in the meantime, and the dependency ranges in pyproject.toml are wide (Gymnasium is `>=0.28.0`, torch is `^2.0.0`), so the combination you can actually install may drift under you without the library changing at all. Pin tianshou to 2.0.1 and pin your Gymnasium and torch alongside it rather than letting a resolver move. Read CHANGELOG.md before any upgrade, and note that the repository also ships benchmark/, test/, docs/, a .devcontainer/ and a .readthedocs.yaml for the documentation build.

Editorial conclusion

Tianshou 2 suits researchers who want to write their own algorithm against a separated Algorithm and Policy interface, and who can accept a Python 3.11 floor and a library that is still labelled Beta by its own classifiers. Check three things first: whether your torch version survives the `!=2.0.1, !=2.1.0` exclusion, whether you need the experimental multi-agent or model-based paths knowing they are experimental, and whether you need the offline algorithms wired up, since examples/offline/ and examples/inverse/ exist only in the repository and not in the wheel. Pin 2.0.1 with a matching Gymnasium, read CHANGELOG.md before migrating from v1, and do not plan on upstream fixes for dependency drift: the last push was 2026-04-03.

Frequently asked questions

Is Tianshou 2 backwards compatible with Tianshou 1?

No. The project states version 2 is not backwards compatible with previous versions because of the extent of the changes, including the new Algorithm and Policy abstractions and a fully revised class hierarchy. Migration information is in the CHANGELOG.md file.

What Python version does Tianshou 2 require?

Python 3.11 or higher, declared as `python = "^3.11"` in pyproject.toml, with 3.11 as the only listed interpreter classifier. The container image is built from python:3.11-slim.

Which reinforcement learning algorithms does Tianshou support?

The list runs past thirty entries, including DQN and its variants Double, Dueling, Branching, Categorical C51, Rainbow, QRDQN, IQN and FQF, the policy gradient family PG, NPG, A2C, TRPO and PPO, the off-policy trio DDPG, TD3 and SAC plus REDQ, the offline methods BCQ, CQL, TD3+BC and their discrete variants, and the add-ons PER, GAE, PSRL, ICM, HER and GAIL.

Does Tianshou support multi-agent and model-based reinforcement learning?

Both are described as experimental support rather than finished features. The pettingzoo dependency at `^1.22` is the multi-agent surface, and model-based algorithms have their own example directory in examples/modelbased/.

How do I install Tianshou?

The package is published on PyPI as tianshou and the project homepage is tianshou.org, with the documentation at tianshou.readthedocs.io and the source on GitHub. Builds go through the poetry-core backend with a committed poetry.lock, and test/, examples/ and docs/ are excluded from the distribution, so a PyPI install does not include the examples.

Official sources

  1. License: MIT
  2. Project website
  3. README
  4. Releases
  5. thu-ml/tianshou on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/thu-ml-tianshou.svg)](https://hysenlabs.com/projects/thu-ml-tianshou)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/thu-ml-tianshou.svg)](https://hysenlabs.com/projects/thu-ml-tianshou)

Was this useful?