skrl: a modular reinforcement learning library for PyTorch, JAX and NVIDIA Warp

What skrl is for, and who ends up using it

skrl addresses a narrow but common problem: reinforcement learning implementations are usually written once per paper, per framework and per environment interface, so comparing two algorithms or moving a policy from a Gymnasium task to a robotics simulator means rewriting code. skrl's README states the library is "designed with a focus on modularity, readability, simplicity, and transparency of algorithm implementation." That sentence is the whole pitch. The intended reader is someone who wants to see the update rule, not a wrapper around it.

The audience is therefore narrower than a general deep learning framework's. If you train policies on Gymnasium or Gym tasks, on PettingZoo multi-agent environments or on ManiSkill, skrl fits. If you train in NVIDIA Isaac Lab or MuJoCo Playground, skrl is built to load and configure those environments rather than treat them as an afterthought. The robotics topics on the repository (isaacsim, isaaclab, brax, robotics) match that. Someone who only needs a single PPO baseline on CartPole will find the modularity unnecessary overhead.

Three backends and the agent-scope training model

The mechanism that distinguishes skrl is the parallel implementation of the same algorithms in PyTorch, JAX and NVIDIA Warp. The three are separate optional dependency groups in pyproject.toml, so a JAX install does not drag in torch. Algorithm code is written per backend rather than translated at runtime, which is why the tests are split into pytest-torch, pytest-jax and pytest-warp workflows in the README badges. The cost of that design is duplication: a bug fix in an algorithm has to be applied in each backend that implements it.

The second mechanism is training by scopes. The README describes it as "enabling agents' simultaneous training by scopes (subsets of environments among all available environments), which may or may not share resources, in the same run." In practice this means one run can hold several agents, each assigned a subset of the available environments, instead of launching one process per agent. For Isaac Lab users running many parallel instances, that is the feature that changes how a training script is structured. The documentation is the place to check the exact trainer API, since the README only names the capability.

Installing skrl and running a first example

skrl is on PyPI as skrl and requires Python 3.10 or newer. The base package pulls in gymnasium, packaging, tensorboard and tqdm. The learning backends are extras, so pick the one you will actually use; the all extra installs torch, jax, jaxlib, flax, optax, warp-lang and warp-nn together.

pip install skrl
pip install "skrl[torch]"

The first command installs the core library. The second adds the PyTorch backend, which is the one most examples assume. Swap torch for jax or warp if that is your backend.

The repository ships a runnable example tree rather than a single demo: examples/gymnasium/, examples/gym/, examples/isaaclab/, examples/mani_skill/, examples/playground/, examples/shimmy/, examples/real_world/ and examples/utils/, with examples/run.bash as a launcher. The intended first use is to run one script from the directory matching your environment interface, then read the agent configuration inside it. That configuration is where the library's modularity becomes visible: the trainer, the agent and the environment are assembled separately, so changing the algorithm does not mean editing the environment code. The README does not name individual example scripts, so list the directory first and pick the file whose name matches the algorithm you want. Expect TensorBoard output, since tensorboard is a base dependency.

Where skrl stops being the right tool

The README is explicit that this is a training library. It says nothing about exporting a policy to TorchScript, ONNX or a C++ runtime, and it presents no serving path. If your goal is to take a trained policy into a latency-sensitive production loop, skrl gets you to the checkpoint and no further; the deployment story is yours to build. That is a real boundary, not a documentation gap you can wait out.

The backend split is the second limitation. Choosing JAX means the PyTorch examples and any community code written against them do not apply directly, even though the algorithm names are the same. Choosing Warp means depending on warp-lang and warp-nn, which are newer entries than torch and jax in this project's optional dependency list. And because the README points readers at the develop branch for "the latest updates to be released," anyone tracking that branch is running code that has not been tagged. The stable tags are the safer default for reproducibility.

skrl against Stable-Baselines3

Stable-Baselines3 is the obvious comparison point for a Python RL library, and the difference is structural rather than a matter of features. SB3 is PyTorch-only and wraps algorithms behind a consistent, opinionated interface; you get a small number of well-tested implementations and a stable API, and you accept that reading the update rule means going into the library's internals.

skrl inverts that. It exposes the algorithm implementation as the thing you read, and it multiplies the surface by implementing it in three backends. The practical consequence: if you want one dependable PPO on Gymnasium and nothing else, SB3 asks less of you. If you want to compare an implementation across PyTorch and JAX, or you are already inside Isaac Lab and need scope-based multi-agent training, SB3 has no equivalent mechanism and skrl does. The choice follows from whether backend portability or API stability matters more to your project.

Maintenance, licensing and what an upgrade costs

The repository is not archived and the last push was on 2026-09-10, so it is being worked on. The release cadence is visible in the tags: 1.4.3 in March 2025, then 2.0.0 in April 2026 and 2.1.0 in May 2026. The gap between 1.4.3 and 2.0.0 is roughly a year, and the jump to a major version signals breaking changes across that boundary. Anyone on the 1.x line should read CHANGELOG.md before moving, because a major bump in an RL library usually means agent or trainer APIs moved.

Upgrade cost is dominated by the backend extras. Because torch, jax and warp are separate optional groups, an upgrade can require bumping several independently versioned packages at once, and the minimums in pyproject.toml (torch>=1.11, jax>=0.4.31, flax>=0.9.0, warp-lang>=1.12) tell you what the project tests against at minimum, not what it tests against at maximum. The licence is MIT, stated both in the README badge and in pyproject.toml as "MIT License", which permits commercial use and modification; that is a fact about the licence text, not advice about your situation.

Editorial conclusion

Adopt skrl if you are training RL agents in PyTorch or JAX and want algorithm code you can read and modify, or if you run Isaac Lab and need several agents training by scopes in one run. Do not adopt it as a deployment runtime; the repository is a training library, and the README does not present an inference or serving path. Before committing, install the backend extra you actually need (skrl[torch], skrl[jax] or skrl[warp]), check that the example directory matching your environment interface exists, and confirm which release you are pulling, since the README points at the develop branch for unreleased updates while pyproject.toml pins version 2.1.0.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. Toni-SM/skrl on GitHub