The Little Book of Reinforcement Learning: a short path with runnable code
The Little Book of Reinforcement Learning
At a glance
- What is it?
- The Little Book of Reinforcement Learning is a compact introduction running from the basics to applied algorithms, distributed with its PDF in the repository, PyTorch implementations from Monte Carlo methods to PPO, and a supplement of rigorous proofs for the dynamic programming material. Version 1, June 2026, under a Creative Commons licence.
- Who is it for?
- The Little Book of Reinforcement Learning fits newcomers who want a complete short introduction with runnable PyTorch code and optional proofs, and returning practitioners who need a fast refresher on the classical arc from Monte Carlo to PPO.
- Can I use it commercially?
- Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
- Is it still maintained?
- Yes. The repository last received commits 66 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 17, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
A short introduction, on purpose
The Little Book of Reinforcement Learning is exactly what its name claims: a short introduction to reinforcement learning, from the basics to applied algorithms. In a field whose canonical free textbook runs to hundreds of dense pages, a deliberately small book with a June 2026 first version is a positioning choice, and a reasonable one: newcomers drown in comprehensive treatments long before they write their first agent.
The author is Alexandre Torres Leguet, identifiable from the repository's print-edition listing, and the repository is the book's associated GitHub page rather than an afterthought: the PDF, the code and a proofs supplement all live in the tree.
A print copy is sold through Amazon for readers who want paper, which coexists comfortably with the free electronic distribution rather than replacing it. The last push was on 2026-07-14, and the README says more material is subject to be added along the way, so the repository is best read as a living companion rather than a frozen artefact.
What the repository adds over the book alone
Three things ship alongside the text. First, the book itself as a PDF, which makes the repository a one-stop distribution point rather than a pointer to a publisher's landing page.
Second, the algos directory: PyTorch-based implementations of the algorithms covered in the book, spanning from Monte Carlo methods to PPO. That range is the pedagogically useful arc, start-to-finish value estimation through policy-gradient methods, and having the code organized by the book's progression means a reader can alternate between page and editor without translating between two taxonomies.
Third, the supplementary directory: detailed explanations and rigorous proofs for the dynamic programming algorithms the book covers only briefly. The README dates this document to 2021, which makes it older than the book itself; the author folded prior material in where the book chose brevity. That is a small but telling detail about the book's design: proofs were pushed to an optional annex rather than allowed to bloat the main text.
What the repository does not show: tests, releases, or build tooling. The code is presented as a companion to reading, not as a maintained library, and adopting it as either would be a category error.
The licence paragraph contradicts itself
The README's licence sentence deserves careful reading, because it disagrees with itself in the space of one line: the book is described as distributed under a non-commercial Creative Commons licence, and the licence named is CC BY-SA 4.0.
Those are different things. CC BY-SA 4.0 permits commercial reuse, conditioned on attribution and share-alike; it is the licence Wikipedia content carries, and it was designed to allow commercial use. A genuinely non-commercial distribution would use one of the NC variants, which CC BY-SA is not. Either the intended licence is an NC variant mislabelled as BY-SA, or the non-commercial description is loose phrasing around a share-alike licence that actually permits commercial use.
The repository tree contains no licence file at all, so the README sentence is the only licence statement that exists, and GitHub accordingly reports no recognised licence. For readers this is mostly academic: reading, learning from and running the code is safe under any plausible reading. For anyone considering reuse beyond that, course materials, translations, derivative works, the prudent move is to ask the author which half of the contradictory sentence governs.
Where it sits among RL learning resources
The obvious comparison set is small and well established. Sutton and Barto's Reinforcement Learning: An Introduction is the canonical free text: comprehensive, rigorous, and long, the very qualities a little book exists to dodge. A reader who finishes this one and wants depth knows where to go.
OpenAI's Spinning Up in Deep RL occupies the practical middle: deep-RL-first, with curated code and documents aimed at getting practitioners training policies quickly, where this book spends its early chapters on the classical foundations before reaching applied algorithms. The two pair well in either order.
The third alternative is a course, which adds structure and deadlines at the cost of pace, and the fourth is the primary literature, which is where every path eventually terminates. The Little Book's specific slot is the fast, self-contained pass: short enough to finish, with code aligned to the chapters, and a proofs annex for the mathematically inclined, at the price of breadth and of a licence statement that needs the author's clarification.
Who it is for, read plainly
The fit is the student or engineer beginning reinforcement learning who wants a complete short pass before deciding whether the field deserves a serious investment of time, with runnable PyTorch code beside the text and optional rigor behind it. It is also a reasonable refresher artifact for someone returning to the basics after years away from the classical material.
It is not a reference: a book this size cannot be one, and its own framing as a short introduction is accurate rather than modest. It is not a code library: the algos directory is teaching code by design. And it is not a finished product line: version one, single author, no releases, supplementary material predating the book, and a stated intention to add more over time.
Read at that weight, it does its job: a weekend-sized on-ramp to a field whose standard texts are semester-sized, with the code to type along and the licence question flagged for anyone whose use goes beyond learning.
Editorial conclusion
The Little Book of Reinforcement Learning fits newcomers who want a complete short introduction with runnable PyTorch code and optional proofs, and returning practitioners who need a fast refresher on the classical arc from Monte Carlo to PPO. It does not fit anyone needing a reference work, a maintained code library, or licence clarity, since the README simultaneously calls the distribution non-commercial and names CC BY-SA 4.0, which permits commercial reuse, and no licence file exists to break the tie. Verify first: that the chapter arc matches the topics you actually need, that the algos directory runs against your PyTorch version, and, if your use goes beyond learning, the author's answer on which licence term governs. The last push was on 2026-07-14.
Frequently asked questions
Is The Little Book of Reinforcement Learning free?
The PDF is distributed in the repository itself, and the README describes the book as Creative Commons licensed, naming CC BY-SA 4.0 while also calling it non-commercial, which is a contradiction worth noting. A print copy is sold on Amazon.
Which algorithms does the book and repo cover?
The book runs from the basics to applied algorithms, and the accompanying algos directory implements them in PyTorch from Monte Carlo methods through PPO. A separate supplement provides detailed explanations and rigorous proofs for the dynamic programming algorithms covered briefly in the book.
Who wrote The Little Book of Reinforcement Learning?
Alexandre Torres Leguet, per the print-edition listing the repository links to. The supplementary proofs document dates to 2021, and the book itself is version 1, dated June 2026, with more repository material promised over time.
Community notes