The book's reinforcement learning code arrives as chapter notebooks

Code for a book, arranged the way the book is

This repository exists to accompany a specific book. The description says it plainly, code from the Deep Reinforcement Learning in Action book from Manning, Inc, and the README repeats the scope as code snippets from that title, linking to the publisher's page. What it is not is a library. There is no package to install and no public API; it is a reading companion, and its usefulness depends entirely on how closely it tracks the book. The organisation follows the book's structure, with chapter directories from Chapter 1 through Chapter 10, plus an Appendix/ directory and an Environments/ directory for the simulation setups the chapters rely on. The primary language is recorded as Jupyter Notebook, which tells you the intended form of every listing: an executable document with prose, code and output interleaved. For a reader who wants to see a worked example with the plots it produces rather than a pasted listing, that format is the whole point.

Notebooks are launched from their own chapter directory

The operational instruction is short and easy to miss. The README says to use a fresh Python 3.13 environment and to launch each notebook from its chapter directory. That last detail is not cosmetic, because notebooks resolve paths relative to the working directory, and a notebook started from the repository root will look for its assets and outputs in the wrong place. The chapter directories hold the maintained notebooks, and the README points to a separate page for book compatibility and corrections, where API updates and corrections to printed listings are recorded, so a reader who finds a listing that no longer matches the printed book has a documented place to look first. The Errata/ directory and the Errata.md file at the root serve the other half of that workflow, carrying links to the maintained chapters and to an immutable pre-refresh source, and the contribute section asks you to file an issue for examples that fail and to edit Errata.md with a pull request for book errors. This is a repository that expects reader feedback as part of its maintenance loop.

A pinned dependency set rather than a loose one

The install step is a single command against a requirements file:

python -m pip install -r requirements.txt

The file pins exact versions for everything that matters, which is the right choice for a repository whose purpose is reproducible listings. The list begins with the simulation stacks and the deep learning core:

ale-py==0.12.0
gymnasium==1.3.0
gym_super_mario_bros==9.1.0
numpy==2.5.1
torch==2.13.0
torchvision==0.28.0

The rest of the file covers the notebook tooling itself, with jupyterlab==4.6.2 and ipython==9.15.0, the plotting and data stack with matplotlib==3.11.1, pandas==3.0.3 and scikit-image==0.26.0, and a long tail of environment packages including magent2, nes-py, minigrid, pyglet and einops, plus requests, urllib3 and tqdm. The README states that in order to run many of the projects you will need at least NumPy and PyTorch, which is the minimum if you only want to read a specific chapter. The size of the file is the cost of the approach, since a reader who only cares about one algorithm still downloads a full deep learning and multi environment toolchain.

Fresh notebooks sit next to the immutable pre-refresh source

The repository holds two generations of the code, and the layout is deliberate. The chapter directories contain the maintained notebooks. Alongside them sits old_but_more_detailed/, a directory whose name says what it is, the earlier and more verbose versions of the listings. The Errata/ directory holds links to both the maintained chapters and the immutable pre-refresh source, so a reader can always get back to the code as it was when the book went to press. That pairing is the most interesting design decision in the repository. Refreshing a book companion is a genuinely hard problem, because the printed listing is a fixed artifact while the libraries underneath keep moving, and a silent update would leave readers unable to reconcile the page with the screen. Here the update happens in the open, with BOOK_COMPATIBILITY.md documenting the API updates and corrections, the corrected notebooks in the chapter directories, and the original source preserved for anyone who needs to see what changed.

Fast checks versus full training runs

The README separates two kinds of verification, and the difference matters for anyone planning to use this in a CI pipeline of their own. The fast checks are one command:

python -m unittest discover -s tests -v

The automated testing document, TESTING.md, covers more: PR regression and smoke checks, weekly and manual full-training runs, the local commands, and the merge-protection setup. VALIDATION.md holds the full training outcomes and the tested limits, which is where you would look to see how far a given listing is expected to get. The distinction is a direct consequence of what reinforcement learning code does. A unit test can check that an update rule computes the right thing or that a network has the expected shape, but it cannot check that a policy learned to walk, because the result depends on random seeds and on many hours of training. The README states plainly that model architectures and the methods described in the text are preserved, while stochastic learning results vary between runs. Any expectation you take from a notebook output has to carry that caveat with it.

Where the environments come from

The chapters need worlds to learn in, and the requirements file is where you can see what they are. Arcade learning environment support comes from ale-py, the modern gymnasium interface from gymnasium, and specific games from packages with memorable names, gym_super_mario_bros for the Super Mario Bros environments, magent2 for the multi agent grid world, nes-py for NES emulation, and minigrid for the miniature grid world. A reader can infer a fair amount about the book's coverage from that list alone, since it spans single agent grid worlds, multi agent settings and pixel based arcade games. The Environments/ directory in the repository holds the corresponding setups, and the Appendix/ directory carries the appendix material. The learning stack underneath is PyTorch, with torchvision for the vision side, einops for tensor rearrangement, and numpy at 2.5.1 as the base. Anyone reproducing a listing needs the environment package for that chapter installed, not just the deep learning framework.

MIT licensed, recently touched, and never released

Two facts frame the maintenance story. The last push to the repository was on 2026-09-14, and the repository is not archived, so the notebooks are being touched and the compatibility work is current. The repository publishes no GitHub releases, so there is no version to pin and no changelog to diff, and the practical versioning reference is the commit history plus the dated entries in BOOK_COMPATIBILITY.md. The licence is MIT, with the LICENSE file at the root, which means the code can be reused, modified and redistributed with only attribution required, a permissive grant that matters if you lift a listing into your own work. The community path is documented too: issues for examples that will not run, pull requests against Errata.md for book mistakes. What this repository is not is a maintained library with semantic versioning, and treating it as one, by depending on it from production code or expecting an upgrade channel, is the mistake worth avoiding.

Editorial conclusion

DeepReinforcementLearningInAction suits a reader working through the Manning book who wants the listings runnable rather than transcribed, and who will treat the notebooks as reference implementations to read alongside the text rather than as a library to depend on. It is a poor fit for a production reinforcement learning codebase, since there is no package API and no released version to pin. Verify first that your Python version matches the 3.13 environment the README asks for, install from the pinned requirements.txt rather than picking library versions yourself, and read VALIDATION.md and BOOK_COMPATIBILITY.md before trusting a listing, because the project states that stochastic learning results vary between runs.

Frequently asked questions

Official sources

1. DeepReinforcementLearning/DeepReinforcementLearningInAction on GitHub
2. Issues
3. License: MIT
4. README