Melting Pot: a social generalisation test bed for multi-agent reinforcement learning

A score for generalisation, not for win rate

Melting Pot assesses generalization to novel social situations involving both familiar and unfamiliar individuals, and it has been designed to test a broad range of social interactions such as cooperation, competition, deception, reciprocation, trust and stubbornness.

That framing is the whole point. In a conventional multi-agent benchmark, an agent is trained on a distribution of opponents and evaluated on the same distribution, and the result measures how well the algorithm optimises. Melting Pot inverts that. The package offers over 50 multi-agent reinforcement learning substrates, which are the multi-agent games, for training, and over 256 unique test scenarios on which to evaluate the trained agents, and the test scenarios deliberately include individuals the agent has never seen.

The performance on those held-out scenarios quantifies two things: whether the agent performs well across a range of social situations where individuals are interdependent, and whether it interacts effectively with unfamiliar individuals not seen during training. The resulting score can then be used to rank different multi-agent reinforcement learning algorithms by their ability to generalize to novel social situations.

That last sentence is the reason to care. A generalisation score on unseen partners is a much harder target than a high average reward, and it is the kind of measurement that gets reported across papers, which is why the project says it hopes to become a standard benchmark for multi-agent reinforcement learning and plans to maintain it and extend it in coming years to cover more social interactions and generalisation scenarios.

The audience is therefore academic and industrial research teams working on multi-agent learning, particularly on cooperative or mixed-motivation settings where the behaviour of the other agents matters.

Fifty substrates, 256 scenarios, and how the two relate

The structure of the package matters more than any individual scenario, so it is worth spelling out how the pieces fit.

A substrate is a social situation rendered as a multi-agent game, and it is the unit you train on. A scenario is a specific configuration of a substrate with a specific set of individuals in it, and it is the unit you evaluate on. The test scenarios are held out, meaning the partners the agent meets there are not the ones it trained with.

The repository is organised around that distinction. The meltingpot/ directory holds the package itself, with a utils/evaluation/evaluation.py module for running evaluations and a human_players/ directory containing the interactive scripts. The docs/ directory holds the documentation, including an extending.md page that explains how to add your own scenarios, which is the extension point the README points contributors at. There are notebooks/, an examples/ directory with a tutorial subdirectory, and a bin/ directory at the top level.

The evaluation library is used to evaluate SavedModels trained on Melting Pot substrates, and the README points at the evaluation results from the Melting Pot 2.0 Tech Report, viewable in an evaluation notebook that is also openable in Colab. That is a useful detail for a benchmark: the published reference numbers are in a notebook you can open and inspect rather than a table in a paper.

The interactive side is the human_players scripts. You can try out the substrates yourself, and for the clean_up substrate the command is:

python meltingpot/human_players/play_clean_up.py

The controls are given: move around with the W, A, S and D keys, turn with Q and E, fire the zapper with 1, fire the cleaning beam with 2, and switch between players with TAB. Other substrates are available in the human_players directory, and some have multiple variants selected with the --level_name flag.

Playing the games by hand is the fastest way to understand what a substrate is asking of an agent, and for a benchmark that measures social behaviour, reading the specification is not always as clear as playing it.

The Lab2D dependency, and the wheel that may not exist for your platform

Melting Pot is built on top of DeepMind Lab2D, which is distributed as pre-built wheels, and the README is explicit about the consequence: if there is no appropriate wheel for dmlab2d, you will need to build it from source, and the README points at the dmlab2d README for details.

That is the single biggest practical constraint in this project. The environments are 2D grid-like simulation games rendered by Lab2D, and the simulator is a compiled dependency rather than pure Python. For a platform with a published wheel, installation is a pip install away. For one without, you are compiling a C++ project before you can run a single episode, and the README does not enumerate which platforms have wheels.

The installation itself is short:

pip install dm-meltingpot

After that you can import meltingpot in your own code. Note the naming, because it trips people up: the distribution on PyPI is dm-meltingpot, while the import name is meltingpot.

The assets are a separate concern from the package, and the setup.py in the repository root shows how they are handled. The build downloads a tarball of assets from a Google Cloud Storage bucket at build time, with the URL built from an ASSETS_VERSION constant, and caches the tarball if it is already present, extracting into meltingpot/assets. The version numbers in that file are worth noting: the package version is 2.4.1 while the assets version is 2.3.0, so the assets are versioned separately from the code and are not always the same release.

For a benchmark that ships scenario assets, that separation is a reasonable design, since content and code change at different rates. It also means a build can fail for network reasons, and it means the assets you get are not necessarily regenerated when you upgrade the package.

Contributing through a Codespace, and the test configuration

The development section recommends a specific workflow, and the reason it recommends it is worth repeating.

The easiest way to work on the Melting Pot source code is the pre-configured development environment via a GitHub Codespace, because this provides a tested development workflow that allows for reproducible builds and minimizes dependency management. The README strongly advises preparing all pull requests through this workflow.

The manual path is spelled out for anyone who needs it, and it is a conventional Python editable install. Clone the repository:

git clone -b main https://github.com/google-deepmind/meltingpot
cd meltingpot

Create and activate a virtual environment:

python -m venv venv
source venv/bin/activate

Install with the dev extras:

pip install --editable .[dev]

And then check the installation with:

pytest --pyargs meltingpot

The pyproject.toml shows how the test suite is configured, and the detail that stands out is that it requires the pytest-xdist plugin and sets addopts to -n auto, so the suite runs in parallel across cores, with testpaths covering the meltingpot package and the examples directory.

The style tooling is also spelled out in the configuration, which is a small but useful signal about how the project expects contributions. isort uses the google profile, pyink is the formatter with a line length of 80, two space indentation and majority quotes, and there is a pyrefly section with a set of individual error codes switched off, which tells you the type checker is configured to be permissive on a specific list of checks rather than left at defaults. The repository root also carries a .pylintrc, a .python-version, a .devcontainer/ directory and a .vscode/ directory.

For a benchmark that other people will run and compare results on, this consistency matters. Reproducibility is the point of the project, and a shared formatter, a pinned parallel test setup and a pre-configured dev container are all part of that.

Version 2.4.1, the tech report, and the disclaimer

On maintenance, the repository is not archived and the last push was on 2026-10-13, on the main branch. The recent releases are v2.4.1 from 2026-09-30, v2.4.0 from 2024-12-19 and v2.3.1 from 2024-07-02, so the project is on a 2.x line with a recent patch release and a slightly irregular cadence.

The project has been busy in 2026, and the release gaps suggest the work is episodic rather than continuous. That is a fair reading of a research benchmark maintained alongside a research programme.

The published artefacts are the two papers the README asks you to cite. The first is the ICML paper on scalable evaluation of multi-agent reinforcement learning with Melting Pot, with its arXiv identifier and DOI given in the BibTeX block, and the second is the Melting Pot 2.0 article, which is the tech report linked at the top of the README. The README also links the Melting Pot Contest at NeurIPS 2023, which is a useful reference point for anyone who wants to see how a community used the benchmark rather than only how it was designed.

The licence is Apache-2.0, and there is a LICENSE file at the root of the tree along with CITATION.bib and an AUTHORS file, which together are the conventional set for a research artefact that wants to be reused and cited properly. The setup.py header carries the Apache 2.0 notice with the DeepMind Technologies Limited copyright. This review does not interpret the licence terms.

The last line of the README is the one to remember before you build anything on this: this is not an officially supported Google product. It is a research release. That does not make it fragile, and the commitment in the README to maintain it and extend it over coming years is a meaningful signal, but it does mean the support expectation is different from a commercial product's.

Editorial conclusion

Melting Pot fits a multi-agent reinforcement learning researcher who needs a shared, citable test bed for social generalisation, because the split between training substrates and held-out scenarios, with unfamiliar partners, is a property the package enforces rather than something each paper has to construct. It is the wrong choice if you want a fast reinforcement learning loop on a single agent, since the whole design is about multiple agents and social situations, and it is the wrong choice if you cannot take the Lab2D dependency, because the environment simulator and its wheel or source build is the floor under the install. Before you evaluate it, read the Extending Melting Pot documentation, and check the version numbers carefully, since the package is 2.4.1 while the bundled assets are a separate 2.3.0 download.

Frequently asked questions

Official sources

1. google-deepmind/meltingpot on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases