RLinf runs embodied and agentic RL on a Ray scheduler with a hand written dependency list
RLinf: Reinforcement Learning Infrastructure for Embodied and Agentic AI.
At a glance
- What is it?
- RLinf is an Apache 2.0 licensed Python package that presents itself as reinforcement learning infrastructure for embodied and agentic AI, wrapping Ray and torch behind a Hydra driven config surface. The interesting part is not the model list but the packaging: a Pre-Alpha classifier, test and lint tooling installed as runtime dependencies, a dependency floor held down by a Hydra workaround, and a version field of 0.4.0 that matches the v0.4 release tagged on 2026-10-20.
- Who is it for?
- Take RLinf if you are training or fine tuning vision language action policies and need LIBERO, real robot rollout wrappers and a Ray based scheduler in one tree, and if you can accept owning the environment yourself. Do not take it if you need a package with a settled dependency contract, because the manifest marks itself Pre-Alpha, carries test tooling as runtime dependencies, and holds Hydra below a development build to stay compatible with another package.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- RLinf/RLinf5,654 ★ · +1,361
| Repository | 2026-07-28 | 2026-11-15 |
|---|---|---|
| RLinf/RLinf | 4.3k | 5.7k |
1,361 Stars in the last 110 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
Two readings of inf, one package for robots and language models
The name carries two meanings at once. The README says the inf in RLinf stands for Infrastructure, which frames the project as a backbone other training code is written against, and also stands for Infinite, tied to open ended learning and continued generalization. Underneath the wordplay the stated scope is narrow: a flexible and scalable open source RL infrastructure designed for Embodied and Agentic AI. The packaging, however, is wider than that sentence. The project name in pyproject.toml is rlinf, the author is listed as RLinf Contributors, the license is Apache 2.0, and the keywords are reinforcement learning, embodied intelligence, and large language models, which puts a mixture of experts language model trainer and a robot policy trainer in a single distribution rather than two. Documentation is served from rlinf.readthedocs.io with a Simplified Chinese mirror at the zh cn path, and the repository keeps README.md beside README.zh-CN.md, so two languages are maintained rather than machine translated on read. The Python floor is >=3.10, which closes the door on 3.9 and older.
The manifest, not the README, is where setup starts
The visible part of the README routes readers to the documentation site, and the repository itself is where the declared floor lives. A clone brings pyproject.toml, a requirements/ directory, a docker/ directory, and the docs tree:
git clone https://github.com/RLinf/RLinf
cd RLinfThe manifest states what the package is built from and who it is for:
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "rlinf"
version = "0.4.0"
requires-python = ">=3.10"One classifier decides how much to trust all of it. Development Status :: 2 - Pre-Alpha is the project's own statement of maturity, printed next to Intended Audience :: Developers and Programming Language :: Python :: 3. The single environment classifier is Environment :: GPU :: NVIDIA CUDA :: 12 :: 12.8, which pins the metadata to one CUDA version on one vendor, while the same README announces support for Biren GPUs, Moore Threads through MUSA, Huawei Ascend through CANN, and AMD through ROCm. So the packaged metadata describes a narrower world than the README does, and a reader on a non NVIDIA accelerator has to check the hardware support matrix the README links to for model and environment combinations instead of trusting the classifier.
pytest, ruff, and pip ship as runtime dependencies
Open the dependencies array in pyproject.toml and the line between the library and its own tooling disappears. The core system group is short and is described in a comment as the dependencies of the core scheduler:
dependencies = [
# Core System
# These are the dependencies of the core scheduler
"ray[default]>=2.47.0",
"torch>=2.5.0",
# dev8 rejects our version_base="1.1"; prerelease bound keeps agentic's dev1 pin valid
"hydra-core<1.4.0.dev8",
"numpy",
]The utilities group underneath is where the surprise sits, because accelerate, debugpy, einops, nvitop, pybind11, setuptools, ninja, pytest, gsutil, ruff, pre-commit, uv, and pip are all installed alongside the training stack. Linting, formatting, test discovery, and the installer itself land in the same environment as Ray and torch. The practical cost shows up when you build a launcher image or a governed cluster environment from this manifest: formatters and test runners arrive and update through the dependency resolver whether or not the training job asked for them. Data processing packages such as datasets, torchdata, and scipy sit in their own group, so a second pass over the array is needed before you can tell what a minimal RL run actually pulls in.
The hydra ceiling is a workaround, not a preference
One bound carries its own explanation, written as a comment above the package it constrains: dev8 rejects our version_base 1.1, and the prerelease bound keeps the agentic package's dev1 pin valid. That single line tells you the ceiling exists to stay compatible with another project's development release rather than with a stable Hydra API, and it explains the unusual shape, a floor with no version number and a ceiling sitting on a development build. In a shared environment this is the line to read first, because one Hydra version has to satisfy both RLinf and whatever else pins it, and raising either pin is a joint decision. The logging group has the same character:
# Codecs for `env.obs_compression` (Env -> Rollout observation transfer)
"lz4",
"zstandard",
# Logging
"wandb<0.25.1",
"swanlab>=0.6.11",
"tensorboard",
"sortedcontainers"wandb is held below 0.25.1 while swanlab must be at least 0.6.11, so experiment tracking is pinned from both ends, and lz4 with zstandard exist for one named setting, env.obs_compression, which is the observation transfer path from an environment into a rollout.
Dependencies twice over, documentation in two languages
The root holds pyproject.toml and a separate requirements/ directory. Documentation is duplicated the same way. README.md and README.zh-CN.md sit side by side, the documentation site publishes an English tree at rlinf.readthedocs.io and a Chinese tree under the zh cn path. Four text surfaces therefore track one release. The same layer also carries files aimed at tools rather than at people, with AGENTS.md and CLAUDE.md at the root and .agents/ and .claude/ directories beside them, plus a .pre-commit-config.yaml for hooks and a DCO next to CODE_OF_CONDUCT.md and CONTRIBUTING.md. For a reader the effect is that pyproject.toml alone may not tell the whole dependency story, and a contributor has to find out by reading which of these files is the one the maintainers actually edit.
Seven example families and a robotics interface split three ways
examples/ is divided into seven families, and the split is the quickest map of what the project actually runs: agent/, diffusion/, embodiment/, offline_rl/, reasoning/, reward/, and sft/. It is wider than the name suggests: the README's news list covers reinforcement learning for SD3 and Wan2.2 video generation models through Diffusion-NFT, and GRPO training for Moonlight-16B-A3B, described as DeepSeek-V3 MLA plus MoE. The embodied side covers LIBERO and real hardware: real-robot post-training algorithms (RLT, STEAM, OPD, RTC), Evo-1, RL for pi0-FAST, MolmoAct2 evaluation, and native PyTorch OpenPI pi0 and pi0.5. The September 2026 refactor then split real world robotics into a unified interface where hardware, tasks, and rollout wrappers are managed separately, with separate documentation pages for the interface, the architecture, real world tasks and environments, adding a robot, adding a task, and teleoperation. The limit of the example tree is that a directory name tells you the family and not the simulator, robot, or checkpoint a given script expects, so you still have to open the matching doc page.
main says 0.4.0 while the newest tag is v0.3
The version field in pyproject.toml reads 0.4.0, and the three most recent tags are v0.2 from 2026-03-26, v0.3 from 2026-07-15, and v0.4 from 2026-10-20, with the default branch last pushed on 2026-10-30. So the manifest version matches the newest release, while tracking main means running commits pushed after that tag. The README describes v0.4 as taking RL out of simulation and into agile control of the physical world, with a modular real-robot system (rlinf.robotics), real-robot post-training algorithms, full-modality models and eight accelerator backends. Reach also runs outward from this repository: Isaac Lab v3.0.0 documents RLinf as its reinforcement learning training infrastructure, the project was welcomed into the PyTorch ecosystem in August 2026, and papers were accepted at OSDI 2026 for RLinf and for DynaRL, which reallocates compute, memory, and communication across components to raise end to end throughput, at RSS 2026 for RLinf-VLA and RLinf-USER, and at NSDI 2027 for FUSCO, which fuses data transformation with communication to speed up MoE all to all shuffling. Check which of those layers you are actually installing.
Editorial conclusion
Take RLinf if you are training or fine tuning vision language action policies and need LIBERO, real robot rollout wrappers and a Ray based scheduler in one tree, and if you can accept owning the environment yourself. Do not take it if you need a package with a settled dependency contract, because the manifest marks itself Pre-Alpha, carries test tooling as runtime dependencies, and holds Hydra below a development build to stay compatible with another package. Before you commit, check three things: which commit or tag you are installing, whether your accelerator appears in the hardware support matrix, and whether the model and environment pair you want is one of the documented examples.
Frequently asked questions
What is RL Infra?
In this project's terms it is what RLinf sets out to be: the README calls it a flexible and scalable open source RL infrastructure designed for Embodied and Agentic AI, and the name stands for both Infrastructure and Infinite. The manifest backs that with ray[default]>=2.47.0 and torch>=2.5.0 as the dependencies of the core scheduler, plus hydra-core for configuration and torchdata, datasets, and scipy for data processing.
What is rlinf
rlinf is the Apache 2.0 licensed Python package behind the RLinf project, built for reinforcement learning in embodied and agentic AI work. It ships seven example families under examples/, a core scheduler on Ray and torch, a docs site at rlinf.readthedocs.io in English and Simplified Chinese, and a classifier that still declares the project Pre-Alpha.
What are the four elements of reinforcement learning?
The repository never spells out a four part definition. What it does describe is the DynaRL work, which names compute, memory, and communication as the resources it reallocates across heterogeneous RL components to improve end to end training throughput.
Is RL a dead end?
Nothing in the project takes a position on that. The record it keeps shows continued investment instead: tagged releases v0.2 on 2026-03-26, v0.3 on 2026-07-15, and v0.4 on 2026-10-20, a default branch last pushed on 2026-10-30, a 0.4.0 version in the manifest, and papers accepted at OSDI 2026, RSS 2026, and NSDI 2027.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/rlinf-rlinf)Star history chart for your README
[](https://hysenlabs.com/projects/rlinf-rlinf)Was this useful?