PettingZoo: a standard API for multi-agent reinforcement learning environments

The problem PettingZoo solves: one API for many agents

Single-agent reinforcement learning settled on a common interface years ago: reset the environment, take a step, read an observation. Multi-agent work did not settle on anything comparable, so every paper and every environment shipped its own loop, its own turn handling, and its own way of deciding which agent acts next. PettingZoo exists to close that gap. It is a Python library for research in multi-agent reinforcement learning, described in the README as akin to a multi-agent version of Gymnasium.

The audience is narrow and specific. This is a library for people who train or evaluate several policies inside one environment, whether they compete, cooperate, or do both. If your problem has one agent, PettingZoo adds ceremony you do not need. If your problem has agents that observe and act at different times, the library's main abstraction was designed for exactly that case.

Agent Environment Cycle: the mechanism behind the API

PettingZoo models environments as Agent Environment Cycle games, citing the AEC paper in the README. The stated reason is to support all types of multi-agent RL environments under one API and to minimize the potential for certain classes of common bugs.

The practical consequence is a sequential loop rather than a batched one. You iterate over agents, ask what the last agent saw, and step one action at a time. The README's usage example shows the shape: an agent_iter loop, a call to env.last() that returns observation, reward, termination, truncation and info, and a single env.step(action) where your policy would go. When an agent is done, the example passes None as the action, which is how the cycle advances past finished agents.

That design is why the library can host turn-based card games and simultaneous physics games under the same interface. It is also the main performance trade-off. A strictly sequential cycle means you cannot hand the whole batch of agent actions to the environment in one call. PettingZoo addresses that with a secondary Parallel API, documented separately, for environments where it is valid to assume agents act at the same time. Choosing between the two is a real decision, not a detail: the AEC API is the general case, and the Parallel API is the faster path for the subset of environments that qualify.

Installing PettingZoo and running your first AEC loop

The base install is a single pip command. The README notes that this does not include dependencies for all families of environments, and that some environments can be problematic to install on certain systems.

pip install pettingzoo

Environment families are installed as extras. The README gives the Atari family as the example, and an all-inclusive extra for everything.

pip install 'pettingzoo[atari]'
pip install 'pettingzoo[all]'

The pyproject.toml lists the extras by name: atari, classic, butterfly, sisl, other and testing. Installing an extra pulls family-specific packages such as multi_agent_ale_py and pygame-ce for Atari, or chess, rlcard and shimmy for Classic.

Creating an environment requires naming both the API and the environment version. The README's example passes "aec" as the first argument and "butterfly/pistonball-v6" as the second.

from pettingzoo import make
env = make("aec", "butterfly/pistonball-v6")

Interaction follows the AEC loop. After env.reset(), you iterate agents, read the last transition, and step an action. The sampled action in the README example is a placeholder for your policy.

env.reset()
for agent in env.agent_iter():
    observation, reward, termination, truncation, info = env.last()
    action = None if termination or truncation else env.action_space(agent).sample()
    env.step(action)

What you should see is the loop visiting agents in turn and ending when every agent has terminated or truncated. The README points to the environment creation tutorial and the custom environment examples for building your own.

Platform support and the Windows gap

The README is direct on this point: PettingZoo is supported and maintained for Linux and macOS, and pull requests related to Windows are accepted but Windows is not officially supported. For a research group standardizing on one environment stack, that single sentence can decide the question. A Windows workstation is not a supported target, and the README does not promise that the environment families will install or run there.

The Python version range is equally explicit in pyproject.toml: requires-python is >= 3.10, < 3.15. The classifiers name 3.10 through 3.14. Anything outside that window is unsupported by declaration, not by accident.

There is a second, subtler constraint in the extras. Some dependencies are pinned conditionally by Python version. In the sisl extra, box2d is pinned to 2.3.10 for Python below 3.14, while box2d-py 2.3.8 and swig 4.* apply from 3.14 onward. In the classic extra, open_spiel is only pulled in for Python 3.11 and above. These conditional markers mean the install surface changes as your interpreter changes, which is worth knowing before you freeze a container image.

Versioning, wrappers and where PettingZoo stops

PettingZoo keeps strict environment versioning for reproducibility. Every environment name ends in a suffix like _v0, and when a change might affect learning results the number is increased by one. The README frames this as preventing confusion. The flip side is that upgrading the library can rename your environment, so a pinned environment string and a pinned package version belong together in any experiment you intend to reproduce.

Wrappers are deliberately not part of the library. The README states that SuperSuit, a separate Farama project, holds the commonly used wrappers such as frame stacking and observation normalization, and that it was developed in lieu of wrappers built into PettingZoo. That is a clean separation, but it means a PettingZoo install alone does not give you the wrapper utilities a training pipeline usually wants. You add SuperSuit separately.

For training examples, the README points outward to tutorials rather than shipping a trainer: CleanRL for PPO in Pistonball, Tianshou for DQN in Tic-Tac-Toe, and AgileRL for curriculum learning and self-play in Connect Four. PettingZoo supplies environments and an interface; the learning algorithms come from those other projects.

PettingZoo compared with Gymnasium

The closest comparison is Gymnasium, and the README makes the relationship explicit: PettingZoo is akin to a multi-agent version of Gymnasium. Gymnasium is also a direct dependency, listed in pyproject.toml as gymnasium>=1.0.0, so the two are not rivals in the sense of competing installs.

The difference is in the loop. Gymnasium assumes one agent, so a step returns one observation and one reward. PettingZoo's AEC API assumes many, so you iterate agents and step one action at a time, with env.last() giving you the transition for whichever agent is currently active. If you have a single-agent problem, Gymnasium is the smaller and more direct tool, and wrapping it in PettingZoo's cycle buys nothing. If you have several agents whose turns interleave, Gymnasium has no native answer, and that is the case PettingZoo was built for. The Parallel API narrows the gap for simultaneous-action environments, but it is a secondary interface, not the default.

Licence, maintenance and upgrade cost

PettingZoo is MIT licensed, stated in both the README badge and the pyproject.toml license field, with the classifier License :: OSI Approved :: MIT License. That is a permissive licence, and it is the same licence the project applies to the package as a whole. It does not automatically cover the optional dependencies you install through the extras, which carry their own licences; if you redistribute a container built with pettingzoo[all], those terms are worth checking separately. This is not legal advice.

The repository is not archived, and the last push was on 2026-09-19. Releases are frequent enough to matter for pinning: 1.26.0 on 2026-04-26, 1.26.1 on 2026-04-27, and 1.27.0 on 2026-08-13. Combined with strict environment versioning, that cadence means an unpinned dependency can shift both the library and the environment names under you. Pinning the package version and the environment string together is the low-cost way to keep results comparable.

Maintenance is handled by a named project manager and the broader Farama team, per the README, with development coordinated on a public Discord server. The Makefile shows the test entry points, including test-all with pytest coverage and narrower targets such as test-param-combs and test-unwrapped. Those targets are a useful signal of what the maintainers consider worth checking when an environment changes.

Editorial conclusion

Adopt PettingZoo if your problem is genuinely multi-agent and you want one API across Atari, Classic, Butterfly and SISL environments without writing your own harness. Skip it if you need Windows support, since the README says Linux and macOS are the maintained platforms, or if you only need single-agent environments where Gymnasium is the direct fit. Before committing, check that your Python version falls inside the declared range of >= 3.10 and < 3.15, and confirm the environment family you plan to use installs its optional dependencies on your machine, because the README warns that some environments can be problematic to install on certain systems.

Frequently asked questions

Official sources

1. Farama-Foundation/PettingZoo on GitHub
2. License: MIT
3. Project website
4. README
5. Releases