Open-source project
yandexdataschool/Practical_RL avatar
yandexdataschool/Practical_RL

Practical_RL: a lab first reinforcement learning course kept as notebook folders

A course in reinforcement learning in the wild

6,616 stars1,809 forksJupyter NotebookUnlicense

At a glance

What is it?
A YSDA and HSE course that teaches reinforcement learning one week at a time, from tabular CEM on Taxi-v0 through PPO to model based planning, shipped as notebooks with no packaged environment and no tagged release since spring 2020.
Who is it for?
Practical_RL suits a reader who wants a worked, lab first tour of reinforcement learning and is willing to assemble the working environment out of a discussion thread and a directory listing. It does not suit anyone who needs a packaged library, pinned dependency versions or automated checking, and the wiki FAQ pages plus the branch you pick matter as much as the notebooks do.
Can I use it commercially?
Yes. Unlicense is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 26 days ago.
What is it written in?
Mainly Jupyter Notebook, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • yandexdataschool/Practical_RL6,616 ★ · +44
Star history
Repository2026-09-112026-11-15
yandexdataschool/Practical_RL6.6k6.6k

44 Stars in the last 65 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

One folder per week, wrapped in course scaffolding

Read Practical_RL from the top down and it is plainly a course archive rather than a library. No package manifest and no dependency file appears among the top level entries. What sits there instead is one directory per week: week01_intro, week02_value_based, week03_model_free, week04_approx_rl, week05_explore, week06_policy_based, week07_seq2seq, week08_pomdp, week09_policy_II, week10_planning, plus two recap folders and a catch all called yet_another_week. Around those sit the pieces needed to run a lab on a student machine: a docker directory, a setup_colab.sh script, an xvfb entry for a virtual framebuffer, an Amazon GPU howto document and youtube_dl_lectures.sh for pulling lecture recordings down. The primary language is Jupyter Notebook, the license is Unlicense and the default branch is master. Nothing here installs as a library, so the only sensible way to use the repository is to open notebooks inside it and run them by hand.

The hosted notebook recipe misspells the repository path

The course info block names three environments. The first is a hosted notebook runner that opens files straight from a repository, and its recipe is a menu path rather than a command: set open, then github, then the owner and repository name, then a branch, and pick any notebook. Copy that path carefully, because the spelling is wrong. The repository is Practical_RL, with a t after the i, while the menu path says pracical_rl. Anyone typing the path literally ends up looking for a repository that does not exist. The branch placeholder is the second thing to notice. The default branch is master, and the two published releases are spring20, named Spring 2020 and dated 2020-08-04, and spring19, dated 2019-06-17. The releases mark individual runs of the course, and master is the default branch. The third environment named is Azure Notebooks.

code
open -> github -> yandexdataschool/pracical_rl -> {branch name}

.gitignore
Amazon GPU howto.md
LICENSE.md
README.md
docker/
setup_colab.sh
week01_intro/
week02_value_based/
week03_model_free/
week04_[recap]_deep_learning/
week04_approx_rl/
week05_explore/
week06_policy_based/
week07_[recap]_rnn/
week07_seq2seq/
week08_pomdp/
week09_policy_II/
week10_planning/
xvfb
yet_another_week/
youtube_dl_lectures.sh

Local setup is a link to a thread, not a dependency list

The local machine option is the one marked recommended, and it is the least pinned down part of the setup. Its link goes to issue number 1, a technical issues thread, rather than to an installation page. No requirements file, no environment file and no lock file appears among the top level entries, so the library versions that made these notebooks run in the semester they were written are not recorded in any form a package manager can consume. The docker directory and the xvfb entry hint that the graphical environments needed more than a bare notebook kernel, yet nothing in the course info says which image to build or which display settings to set. In practice the dependency list lives in a conversation you have to find and read. So the repository cannot hand you a reproducible environment, and it cannot tell you which release of any simulator a given lab was written against. That is the price of a course whose priority is the ideas.

Homework pointers name folders the tree does not have

The syllabus sends homework readers to files whose names do not line up with the directories on disk. Week one says the homework description is in week1/README.md, week two points at week2/README.md and week three at week3/README.md, while the folders are called week01_intro, week02_value_based and week03_model_free. Two recap directories also exist: week04_[recap]_deep_learning and week07_[recap]_rnn. Only the first appears in the syllabus, which jumps from week06_policy_based straight to week07_seq2seq, so the recurrent network recap sits on disk and outside the reading order. Two more naming details matter if you script over the tree: several folder names carry square brackets, and the advanced policy folder is spelled with two capital I characters in week09_policy_II. None of this is fatal, but it means there is no single index mapping the printed course plan onto the folders you actually hold.

The syllabus opens by warning that the schedule will slip

Before planning a study calendar, read the note at the head of the syllabus: it is approximate, the lectures may occur in a slightly different order, and some topics may end up taking two weeks. That warning is the honest shape of a course built around a live cohort taught on campus at HSE and YSDA and kept friendly to online students in English and Russian. The content itself moves fast. Week one runs from reinforcement learning problems around us through decision processes and stochastic optimization to the crossentropy method, draws a line between parameter space search and action space search, and holds a seminar that opens openai gym with tabular CEM on Taxi-v0 and deep CEM on box2d environments. Week two closes on a lecture item titled Discounted reward fails. Week three puts Q-learning, SARSA and Expected Value SARSA in the same seminar. Nothing in that sequence carries a date, a lecture number or an expected return, so a self learner gets an ordering, not a timetable.

The labs name simulators, and one of them wants a screen

Every lab is pinned to a named environment, which makes the scope easy to see and easy to overrun. Taxi-v0 and box2d carry week one. CartPole and Atari carry approximate Q-learning with experience replay in week four. Contextual bandits carry bayesian exploration in week five, toy tasks carry MCTS in week ten, simple robot control carries approximate TRPO in week nine, and week eight pairs recurrent A3C and DRQN with deep kung fu and doom. The frameworks are named too: week one works inside openai gym, the deep learning recap introduces pytorch and tensorflow with a convolutional image classification exercise, and several tensorflow assignments came from Scitator. The xvfb entry at the top level is there because some of those environments render a window, and the Amazon GPU howto document sits beside it. The syllabus lists no expected returns, and staff are credited with homework checkups.

Ten named people carry it, and the newest tag is from 2020

The work is credited to named people rather than to anonymous contributors. Pavel Shvechikov handles lectures, seminars, homework checkups and the reading group. Nikita Putintsev runs seminars and checkups and is credited with organising the course. Alexander Fritsler teaches, Oleg Vasilev does technical support, and Dmitry Nikulin is credited with tons of fixes, far and wide. Four more people cover seminars and homework checkups, and Fedor Ratnikov does admin work. Supporting text sits in wiki pages for the course FAQ, an Online Student's Survival Guide and an RL reading group, with lecture slides on a Yandex disk and an anonymous feedback form. Outside help includes pictures from the Berkeley AI course, heavy borrowing from CS294, and Alexey Umnov during spring2018. Two facts are worth holding on to. The repository is not archived and its last push is dated 2026-10-20, yet the newest release is spring20 from 2020-08-04, so there is no recent tag to pin. And the wiki pages that carry the FAQ can change without any release, so check them against the branch you are running.

Editorial conclusion

Practical_RL suits a reader who wants a worked, lab first tour of reinforcement learning and is willing to assemble the working environment out of a discussion thread and a directory listing. It does not suit anyone who needs a packaged library, pinned dependency versions or automated checking, and the wiki FAQ pages plus the branch you pick matter as much as the notebooks do. Before scheduling a semester from it, confirm the branch you use still runs against the simulator versions installed on your machine, since the newest tag is spring20 from 2020-08-04 while the last push is 2026-10-20.

Frequently asked questions

What exactly is RL according to the Practical_RL syllabus?

Week one starts from reinforcement learning problems around us, decision processes, stochastic optimization and the crossentropy method, then contrasts parameter space search with action space search. Its seminar opens openai gym and implements tabular CEM on Taxi-v0 and deep CEM on box2d environments.

Why is RL better than SFT in the Practical_RL course?

The syllabus never makes that comparison. The closest topic is week seven, reinforcement learning for sequence models, which covers problems with sequential data, recurrent networks, backprop through time, vanishing and exploding gradients, gradient clipping and a character level RNN language model seminar.

Does Practical_RL cover RL in finance?

No finance example appears anywhere in the syllabus. The listed labs run on Taxi-v0, box2d, CartPole, Atari, deep kung fu and doom, plus toy tasks for MCTS and simple robot control for approximate TRPO.

What practical examples of RL does Practical_RL use?

Each seminar names its environment: Taxi-v0 for tabular CEM, box2d for deep CEM, CartPole and Atari for approximate Q-learning with experience replay, contextual bandits for bayesian exploration, toy tasks for MCTS, and a character level RNN language model in week seven.

Official sources

  1. Issues
  2. License: Unlicense
  3. README
  4. Releases
  5. yandexdataschool/Practical_RL on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/yandexdataschool-practical-rl.svg)](https://hysenlabs.com/projects/yandexdataschool-practical-rl)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/yandexdataschool-practical-rl.svg)](https://hysenlabs.com/projects/yandexdataschool-practical-rl)

Was this useful?