SkyRL: a full-stack RL library for training LLMs, split into four packages

Four subprojects in one repository, and why that is the whole design

The README describes SkyRL as a full-stack RL library and then immediately splits it into four directories, each with its own purpose. skyrl-train is the training framework. skyrl-tx is a backend implementing the Tinker API, with a unified engine for training and inference. skyrl-agent is the layer for long-horizon, multi-turn agent training. skyrl-gym is a collection of tool-use tasks (math, coding, search, SQL) written against the Gymnasium API. A fifth directory, skyrl, is described as the unified library for RL on your own hardware, and the README says it combines skyrl-train and skyrl-tx.

The target reader is not someone who wants to call a hosted RL endpoint. It is someone who wants to own the training loop. The README's getting-started section routes you by intent: model training goes to skyrl, environment building goes to skyrl-gym, agentic pipelines goes to skyrl-agent. That routing is the actual product decision. Each subproject can be read and modified on its own, which matters when the thing you need to change is the reward computation or the rollout scheduler, not the model.

The cost of that split is real. There is no single "install SkyRL and run one command" path in the README. You choose a subproject first and then follow the docs for it.

What the training stack actually pins, and what that tells you

The root pyproject.toml declares the package name skyrl, requires Python >= 3.12, and lists runtime dependencies including datasets>=4.0.0, transformers>=5.6.1,<=5.16.1, tokenizers>=0.23.1, peft==0.18.1, cloudpathlib>=0.23.0, zstandard>=0.23.0 and xxhash>=3.0.0. Two of those are exact pins, not ranges: peft and transformers' upper bound. A comment in the file explains the tokenizers floor tracks the transformers pin, because transformers 5.16.1 requires tokenizers>=0.23.1.

Exact pins on peft and an upper bound on transformers mean SkyRL is coupled to a specific generation of the Hugging Face stack. That is a normal choice for a training framework, where a silent behaviour change in a LoRA implementation can invalidate a run, but it also means you cannot freely upgrade transformers in the same environment. If your project already depends on a newer transformers, SkyRL's environment and yours will conflict.

The optional extras are where the hardware story lives. gpu pulls jax[cuda13] and tpu pulls jax[tpu], both marked linux-only. ray pins ray[default]==2.57.0. The aws extra adds cloudpathlib[s3] and s5cmd, with gcp and azure doing the same for their object stores. Three separate extras named jax, fsdp and megatron carry the dependencies the engine needs for --backend="jax", --backend="fsdp" and --backend="megatron" respectively, per the comment in the file. Backend selection is therefore an install-time decision as much as a runtime flag.

Installing SkyRL and running a first training job

The README does not give a pip command. It says to look at the Development Guide in the docs, and to check out the skyrl directory for model training with the quickstart docs. So the honest installation path starts from a clone and the extras declared in pyproject.toml.

The extras are named gpu, tpu, ray, aws, gcp, azure, jax, fsdp, megatron, tinker and skyrl-train in pyproject.toml. The project metadata lists them as optional-dependencies, and each one carries a different set of packages, so which extras you need depends on your hardware and on which subproject you are using. A CUDA machine that wants the Ray execution layer needs both the gpu and ray extras; a Tinker API deployment needs the tinker extra, which pulls tinker>=0.3.0,<=0.24.1 along with fastapi[standard], sqlmodel, sqlalchemy[asyncio], aiosqlite, asyncpg and psycopg2-binary, the server side of that API.

For a first real run, the repository ships example directories rather than a single script: examples/train/, examples/train_scripts/, examples/train_integrations/, examples/serve/ and examples/tinker/. The README points at the quickstart docs for the actual invocation, and that is where the run command lives. What can be confirmed from the repository layout is that the examples are grouped by intent, so examples/train/ is where a plain training run starts and examples/tinker/ is where a Tinker API run starts. Read the quickstart page for the exact arguments before you launch anything; the README does not reproduce them.

The Tinker API backend is the most concrete thing here

Of the four subprojects, skyrl-tx has the clearest external contract. The README states that SkyRL implements the Tinker API, and that this lets you run any training script written in the Tinker API on your local GPUs. The February 2026 announcement in the news list says the same thing: SkyRL now implements the Tinker API, run any training script written in the Tinker API on your local GPUs.

That is a different proposition from "we have a training framework". It means the API surface is defined elsewhere, by Tinker, and SkyRL provides an implementation of the backend. The tinker extra's dependency list supports that reading: FastAPI, SQLModel, SQLAlchemy with asyncio, aiosqlite and asyncpg point at an HTTP service with a database behind it, not a library you import into a script. The README describes skyrl-tx as a cross-platform library implementing a backend for the Tinker API, with a unified engine for training and inference.

The practical consequence: if you have Tinker API scripts already, porting them to your own hardware is a configuration exercise rather than a rewrite. If you do not, the Tinker API is an abstraction you would be adopting along with SkyRL, and the README does not argue for why you would.

Where SkyRL is the wrong tool

SkyRL is the wrong choice if you want a small, single-machine fine-tuning loop. The dependency set assumes a serious environment: Ray pinned to 2.57.0, JAX with CUDA or TPU support, flash-attn in the skyrl-train extra, and a transformers version held inside a narrow band. Installing that on a laptop to try an idea is not the intended use, and the README never suggests it is.

It is also the wrong tool if you need a documented rollback path. The README does not document rollback, checkpoint compatibility between versions, or what happens to a run in progress when you upgrade. There are three releases listed, skyrl-v0.1.0 in March 2026, skyrl-v0.2.0 in April 2026 and skyrl-v0.3.0 in July 2026, and the README does not state whether a checkpoint written under 0.1.0 loads under 0.3.0. For a training framework that is a meaningful gap, because the artifact you care about is a checkpoint and the cost of losing one is measured in GPU hours.

A third boundary: the README gives no supported-model list of its own. It links to a Supported Models page in the docs. If your model is not on that page, the README gives you nothing to go on, and the repository's own documentation is the only source.

SkyRL compared with VERL, and the Tinker API question

The search data pairs SkyRL with verl often enough that the comparison is worth stating plainly, but the README does not describe verl's internals, so the honest difference is structural rather than a feature table. What the SkyRL README establishes is a four-way split: training, an agent layer, an environment gym, and a Tinker API backend. The agent layer is the part that is hardest to substitute, because it is aimed specifically at long-horizon, multi-turn tool use in real environments, and the README ties it to a paper (arXiv 2511.16108) and to a reproduction commit for the SkyRL-v0 results.

The other axis is the Tinker API. SkyRL implements it, which means training scripts written against that API can run on your own GPUs. If you are already invested in Tinker-style scripts, that is a migration path rather than a rewrite. If you are not, it is an extra abstraction layer between you and the trainer.

The environment library is a third differentiator. skyrl-gym implements math, coding, search and SQL tasks against the Gymnasium API, so a reward function written as a Gymnasium environment plugs into the same training stack. Whether that matters depends entirely on whether your task looks like one of those four.

Licence, maintenance and the upgrade bill

SkyRL is Apache-2.0. That permits commercial use and modification, and it includes an explicit patent grant, which matters if you are building on the training stack inside a company. It does not tell you anything about the licences of the model weights you train or the datasets you pull in through datasets>=4.0.0, and those are separate questions the repository does not answer. Nothing here is legal advice.

The repository is not archived, and the last push was on 2026-09-10. The release cadence visible in the README's news list is roughly quarterly: 0.1.0 in March 2026, 0.2.0 in April 2026, 0.3.0 in July 2026. Between releases the main branch moves, and the README itself demonstrates the cost of that: to reproduce SkyRL-v0 results exactly, it instructs you to check out commit a0d50c482436af7fac8caffa4533616a78431d66 rather than use a release.

That instruction is the upgrade story in miniature. Exact results are tied to a commit, not a version. Budget for pinning your own environment and for re-validating a training run when you move forward, because the README does not promise checkpoint or config compatibility across versions.

Editorial conclusion

Adopt SkyRL if you already have GPUs and want to modify the RL training loop rather than call a hosted service, or if you want to run Tinker API training scripts on your own hardware. Do not adopt it if you need a stable single-package API: the repository is four subprojects with different entry points, and the README points you at the docs for installation rather than giving a pip command. Verify first that the pinned transformers range (>=5.6.1,<=5.16.1) and peft==0.18.1 match the rest of your stack, and check which backend extra (jax, fsdp or megatron) your hardware needs before you install anything.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. NovaSky-AI/SkyRL on GitHub
3. Project website
4. README
5. Releases