PTAN is the PyTorch port of AgentNet, a reinforcement learning toolkit

What AgentNet was, and why a port exists

PTAN stands for PyTorch AgentNet, and it is a reimplementation of the AgentNet library for PyTorch. That original AgentNet was a Yandex Data School project, and the point of the port is not that AgentNet was missing but that the ecosystem it was built for was not PyTorch. The stated purpose here is narrow and clear: it is a reinforcement learning toolkit for PyTorch.

The library was used in the book Deep Reinforcement Learning Hands-On, second edition, and the sample sources for that book live in a separate repository published by Packt. That connection explains the shape of the project better than anything else. PTAN is the library half of a teaching package. The book's code is written to be read, the library is written to be small enough to read, and the whole thing is arranged so that a reader can trace from the chapter's idea to the line of code that implements it.

The practical audience is therefore a learner or an instructor, and secondarily a researcher who wants to see how the moving parts of a reinforcement learning loop fit together without reading a framework's internals. It is not a batteries-included library of state of the art agents, it is the plumbing underneath them. The repository layout matches that reading: a ptan/ package directory, a docs/ directory holding an introduction notebook, a samples/ directory with a dqn_vec example, a tests/ directory, and a setup.py with a Makefile for running the tests and opening the coverage report.

Experience sources, fire schedules and adapters

The library's contribution is the decomposition of a training loop into pieces that can be swapped. The documentation points at a specific piece of that: ExperienceSource compared with ExperienceSourceFirstLast, with an issue link explaining the difference. That single comparison tells you the abstraction. An experience source decides what the agent observes and how it acts, and the difference between the two variants is about what portion of the source's output counts as an observation for the purposes of a learning step, which is the kind of detail that is easy to get wrong and hard to reason about.

Fire schedules sit alongside. A fire schedule decides when the agent is updated, and separating that decision from the source means you can change how often learning happens without touching how experience is gathered. Agent adapters handle the conversion between the agent's action representation and the environment's, which is the layer that lets one agent run against many environments.

Taken together, this is the classic structure of a reinforcement learning system, but it is worth saying why it is done this way rather than in one class. A monolithic training loop is easy to write and hard to extend. Once you want to compare two exploration strategies, or change the update frequency, or run the same agent on two environments, the pieces have to come apart anyway. PTAN makes them separate from the start, which is why it reads as a teaching artefact rather than a product.

Branches, because the dependency set moved

The project description is unusually candid about its own release management. The repository is maintained to keep dependency versions up to date, and the project says this requires effort and time to test all the examples on new versions, so asks you to be patient. The logic it describes is that there are several branches corresponding to the major PyTorch versions the code was tested against, and that due to incompatibilities in PyTorch and other components, the code printed in the book might differ from the code in the repository.

The branches available are listed. The master branch contains the code with the latest PyTorch that was tested, which at the time of writing is PyTorch 1.7. The torch-1.3-book-ed2 branch holds the code printed in the second edition of the book with minor bug fixes, and it uses PyTorch 1.3, which is available only on conda repositories. The torch-1.7 branch is PyTorch 1.7 and has been merged with master.

That branch policy is a rational response to the problem, and it also dates the project description. Every branch uses Python 3.7, and more recent versions were not tested. The last push to the repository was on 2026-09-16, so the code is being touched, but the listed release history shows the last published release as v0.8.1 on 2024-10-29, before that v0.6 in December 2019 which was used in the second edition of the book.

Now the interesting part. The requirements file in the repository does not match the requirements list in the project description. The description lists PyTorch at version 1.1.0, OpenAI Gym, PyTorch Ignite, OpenCV and TensorBoardX. The requirements.txt file lists gymnasium with the atari and classic-control extras at 0.29.0 or newer, moviepy pinned at 1.0.3, numpy below 2, opencv-python at 4.10.0.84, torch at 2.5.0, torchvision at 0.20.0, pytorch-ignite at 0.5.1 and stable-baselines3 at 2.3.2. So the dependency reality has moved to a modern PyTorch and from Gym to Gymnasium, while the prose in the description is older. Anyone installing today should follow the requirements file rather than the prose, and the prose is exactly the kind of thing that sends a reader down a wrong path.

Installing it and the sample environment

There are three documented install routes. From sources, the classic setuptools invocation:

python setup.py install

From PyPI, the ordinary pip command:

pip install ptan

And from GitHub, the repository itself, by pointing pip at the git URL for Shmuma/ptan.

For Anaconda Python users there is a separate note, because some of the samples need modules that pip alone will not resolve cleanly:

conda install pytorch torchvision -c pytorch
pip install tensorboard-pytorch
pip install gym
pip install gym[atari]
pip install opencv-python

The requirements file is what the installed package actually pulls in, and it is worth reading it before you start, because it fixes a fairly specific combination:

gymnasium[atari]>=0.29.0
gymnasium[classic-control]>=0.29.0
moviepy==1.0.3
numpy<2
opencv-python==4.10.0.84
torch==2.5.0
torchvision==0.20.0
pytorch-ignite==0.5.1
stable-baselines3==2.3.2

The numpy below 2 constraint and the exact opencv pin mean you want a dedicated environment rather than a shared one. The presence of stable-baselines3 in the requirements is itself informative, because it suggests the samples compare against or reuse baseline implementations rather than implementing agents from nothing. The tests are run with pytest, and the Makefile has a pytest target plus a cov target that opens the coverage report in a browser.

Where PTAN stops being the right choice

The honest limitation is the dependency contradiction, and it is not a small one. The prose says PyTorch 1.1.0 is required, the branches say PyTorch 1.3 and 1.7, and the requirements file pins torch 2.5.0 with torchvision 0.20.0. A reader who follows the documentation ends up with an environment that does not match the code, or a code path that predates the environment they had to build. Nothing in the repository reconciles these, so the first hour of adoption is spent working out which set of instructions is current.

The second limitation is scope, and it is a design decision rather than a defect. PTAN supplies the loop, not the algorithms. If you want a working agent with a modern algorithm and a maintained implementation of the state of the art tricks, stable-baselines3 in the requirements list is doing that job for you, and PTAN is the layer that lets you mix your own pieces with it. That is a legitimate niche, but it is a niche, and if your goal is to get a benchmark result then the toolkit is an extra layer between you and the result.

Third, the release history is thin. v0.6 in 2019 and v0.8.1 in 2024, with a push on 2026-09-16 and no release to go with it, means the pip package on PyPI is the artefact you get, and whether it matches the current requirements file is not something the description tells you. A project whose value depends on tracking dependency versions has to publish those versions, and this one is a small step behind its own requirements file.

On licensing there is a discrepancy worth checking yourself rather than assuming. The project is MIT licensed with a LICENSE file at the root of the repository, while setup.py declares the license field as GPL-v3. The setup.py also carries the author as Max Lapan with a contact address, and sets the version to 0.8.1, matching the last published release. If the licence question matters for your use, read the LICENSE file in the repository, because the packaging metadata and the repository metadata disagree.

How this compares with writing the loop yourself

The realistic alternative is not another library, it is twenty lines of your own. A reinforcement learning training loop is short: collect transitions until you have a batch, compute a loss, step the optimizer, occasionally sync a target network, log a scalar. Writing that yourself gives you full control, zero dependencies and no version archaeology, and for a single environment and a single algorithm it is often less work than fighting a dependency set.

The difference in approach is what PTAN actually offers. Where a hand-written loop tends to entangle the collection of experience with the choice of when to update, PTAN separates them into an experience source and a fire schedule, so an experiment can change the update cadence without touching data collection. It also gives you the AgentNet vocabulary and an adapter layer for actions, which matters when you move the same agent between environments. And it gives you something you cannot easily write in an afternoon: a set of examples and tests that demonstrate the intended usage, which for a learner is worth more than the code itself.

A third path is a modern framework with the algorithms already implemented, such as the stable-baselines3 package that appears in PTAN's own requirements. That path gives you working agents and a maintained ecosystem, and it gives up the readable, small, explicit structure that makes PTAN useful for understanding. The choice between them is a choice about what you are optimising for. If you are optimising for understanding the loop, PTAN is closer to the answer. If you are optimising for a result on a benchmark in a month, the framework is closer to the answer, and the compatibility cost of PyTorch 2.5 with a library last released in 2024 is real.

Editorial conclusion

Adopt PTAN if you are learning reinforcement learning or want a small, readable implementation of the training loop that separates experience generation, firing schedules and agent updates, and you are content to work inside its pinned dependency set. Do not adopt it as the base of a new production reinforcement learning stack, since the toolkit itself has not been released since v0.8.1 in 2024 while its pinned dependencies move. Before you install, read requirements.txt rather than the requirements list in the project description, because the pinned torch 2.5.0 and gymnasium 0.29 lines differ from what the older text says, and confirm the branch you clone matches the PyTorch version your environment already has.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. Shmuma/ptan on GitHub