Dopamine (google/dopamine): a small JAX research framework for reinforcement learning
Dopamine is a research framework for fast prototyping of reinforcement learning algorithms.
At a glance
- What is it?
- Dopamine is Google's compact RL research framework, now built around JAX agents. Here is what it solves, how the code is laid out, how to install it, and where it stops being the right tool.
- Who is it for?
- Adopt Dopamine if you are a reinforcement learning researcher who wants a small JAX codebase to fork and edit, and who is comfortable installing Atari or Mujoco environments before the framework itself. Do not adopt it if you need a maintained, versioned library with stable releases: the last push was on 2026-03-24, the newest release is v2 from 2019-09-26, and the README says pip installation is not the recommended path.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 175 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What google/dopamine is for, and who it is not for
Dopamine describes itself as "a research framework for fast prototyping of reinforcement learning algorithms." That sentence is the whole scope. The stated goal is a codebase small enough to be read end to end, so that a researcher can change an algorithm and rerun a benchmark without first untangling a framework. The README lists four design principles: easy experimentation, flexible development, compact and reliable implementations of a few battle-tested algorithms, and reproducibility, with the evaluation setup following Machado et al. (2018).
The audience is narrow on purpose. If you want to train an agent on a business problem, or you want a library with a stable public API that a vendor supports, this is the wrong project. The README says plainly that Dopamine "is designed to be modified directly" and recommends installing from source for writing your own experiments. It also states that it is not an official Google product. That combination tells you the intended user is someone who will open the source files, not someone who will call a function and trust the defaults.
The JAX agent set and the legacy TensorFlow split
Dopamine supports six agents, implemented with JAX: DQN, C51, Rainbow, IQN, SAC and PPO. The README notes that many of these agents also have a TensorFlow implementation, marked legacy, and that newly added agents are likely to be JAX-only. That sentence is the most important maintenance signal in the README. It means the TensorFlow path is frozen in place while the JAX path receives new work.
The practical consequence is that a project starting today should target the JAX agents. Flax appears in the dependency list, which matches the JAX implementation. If your existing code is written against the TensorFlow agents, you are on a branch of the repository that the README describes as legacy, and you should expect to port rather than to receive fixes.
Each agent also has baselines published on the project site, and the repository ships a set of Colaboratory notebooks under dopamine/colab. Those notebooks are the fastest way to see an agent run before you commit to a local install.
Installing Dopamine from source and running the first test
The README offers three routes: Docker containers, install from source, or pip. It calls source installation the most common way to use Dopamine, and says pip is simple but not recommended for most users because the framework is meant to be modified. Start by cloning the repository.
git clone https://github.com/google/dopamineThen install the dependencies from the requirements file that ships in the repository.
pip install -r dopamine/requirements.txtThe README recommends a virtual environment for this step, and notes that you must install the environment you intend to use before Dopamine itself. For Atari, the README says the ROMs should now come packaged with ale_py, and that you may need to run extra steps to install baselines. For Mujoco, it points to the mujoco-py installation instructions and a separate licence, then `pip install mujoco-py`.
If you prefer pip over source, the package name is dopamine-rl, not dopamine.
pip install dopamine-rlTo check the install, run the Atari initialisation test from the repository root. The README gives these two commands, and the export matters because the test imports from the working tree.
export PYTHONPATH=$PYTHONPATH:$PWD
python -m tests.dopamine.atari_init_testA clean run of that test is the signal the README offers that the installation succeeded. It is an initialisation test, not a training run, so passing it does not prove your chosen agent trains.
Where Dopamine stops being the right tool
The dependency list is the clearest limitation. setup.py pins `gym <= 0.25.2` and `gymnasium >= 1.0.0` at the same time, alongside `tensorflow >= 2.2.0`, `tf-keras >= 2.18.0`, `jax >= 0.1.72` and `flax >= 0.2.0`. Carrying both the old gym and gymnasium, and both TensorFlow and JAX, means a resolver has many ways to produce a working environment and many more ways to produce a broken one. The README's own instruction to install the environment first is a workaround for that, not a solution.
A second limitation is release cadence. The newest release listed is v2, dated 2019-09-26, labelled a pre-Keras release; v1 is from 2019-01-05. The repository has been pushed since then, but the versioned artefacts have not moved with it. If your workflow depends on pinned releases rather than a commit hash, Dopamine does not give you that.
A third case is scale. Nothing in the README claims distributed training, a serving path, or a production deployment story. The framework is aimed at benchmark experiments on a single machine. If you need to train across many accelerators or to serve a policy, the repository layout gives you no entry point for it.
How Dopamine compares with Stable-Baselines3 and RLlib
The honest alternative depends on what you are optimising for. Stable-Baselines3 is a library: you install it, import an algorithm, and call learn on an environment. It is built on PyTorch and its public interface is meant to stay stable across versions. Dopamine inverts that. The README states the framework is designed to be modified directly, so the unit of work is the source file, not the API call. If you want to swap a replay buffer or change an update rule, Dopamine expects you to edit it; Stable-Baselines3 expects you to subclass or configure it.
Ray RLlib sits at the other end. It targets distributed training and a large algorithm catalogue behind a configuration-driven interface. Dopamine has neither. The trade is the opposite direction: Dopamine gives you a codebase you can hold in your head, and RLlib gives you scale and breadth you cannot easily read.
The choice is therefore about whether your bottleneck is understanding the algorithm or running it at size. For a small JAX codebase to experiment in, Dopamine is the smaller surface. For a supported PyTorch library, or for multi-node training, it is not.
Maintenance, upgrade cost and the Apache-2.0 licence
The last push to the repository was on 2026-03-24, which is recent, but the release history tells a different story: v2 dates from 2019-09-26 and v1 from 2019-01-05. So the code moves and the tagged releases do not. Anyone tracking Dopamine should track commits, not versions, and should expect to read diffs rather than release notes.
The upgrade cost follows from the dependency pins. Because setup.py allows both gym and gymnasium, and both TensorFlow and JAX, an environment that works today can break on an unrelated transitive upgrade. Pinning your own environment file is the realistic mitigation.
On licensing, the repository is Apache-2.0, and setup.py carries the standard Apache header. That is a permissive licence, but it is not legal advice, and the README adds a separate request: if you use Dopamine in your work, the project asks you to cite the white paper, with a BibTeX entry for castro18dopamine given in the README. That citation request is not a licence term, but it is a stated expectation of the maintainers.
A realistic first experiment
After the initialisation test passes, the README points to three places to go next: the docs for training agents, the published baselines for each agent, and the Colaboratory notebooks in dopamine/colab. The notebooks are the lowest-friction starting point because they remove the environment installation problem from your first hour.
For a local run, the workflow the repository implies is: pick a JAX agent, pick an Atari or Mujoco environment, and run the corresponding training entry point from the working tree with PYTHONPATH set as in the test above. The README does not spell out a single canonical training command, so the notebooks and the docs are where the concrete invocation lives. That gap is worth noting: the README covers installation and testing in detail and leaves training to the linked documentation.
If you intend to modify an algorithm, the sensible first step is to read the agent implementation for the one closest to your idea, change one thing, and rerun against the published baseline for that agent. That is the loop the four design principles describe.
Editorial conclusion
Adopt Dopamine if you are a reinforcement learning researcher who wants a small JAX codebase to fork and edit, and who is comfortable installing Atari or Mujoco environments before the framework itself. Do not adopt it if you need a maintained, versioned library with stable releases: the last push was on 2026-03-24, the newest release is v2 from 2019-09-26, and the README says pip installation is not the recommended path. Before committing, verify that your environment installs (ale_py for Atari, mujoco-py for Mujoco), that the atari_init_test passes, and that the agent you want is in the JAX set rather than the legacy TensorFlow set.
Frequently asked questions
How do I install Dopamine (google/dopamine)?
Clone the repository with git clone https://github.com/google/dopamine and then run pip install -r dopamine/requirements.txt. The README recommends installing from source over pip, and says to install your Atari or Mujoco environment before Dopamine itself.
How do I use google/dopamine?
The README says Dopamine is designed to be modified directly, so the intended workflow is to install from source, edit the agent code, and run benchmark experiments. It points to the docs for training agents, the published baselines, and the Colaboratory notebooks in dopamine/colab.
Which algorithms does google/dopamine support?
Six agents implemented with JAX: DQN, C51, Rainbow, IQN, SAC and PPO. The README notes that many of these also have a TensorFlow implementation, marked legacy, and that newly added agents are likely to be JAX-only.
How do I check that a google/dopamine installation worked?
The README gives an Atari initialisation test to run from the dopamine root directory: export PYTHONPATH=$PYTHONPATH:$PWD followed by python -m tests.dopamine.atari_init_test. It is an initialisation test, so it does not confirm that a training run will work.
What licence does google/dopamine use?
The repository is Apache-2.0, and setup.py carries the standard Apache header. The README separately asks that you cite the Dopamine white paper if you use it in your work, with a BibTeX entry for castro18dopamine.
Community notes