MPE2 and the quiet rework of OpenAI's particle environments

Three defaults that differ from the original OpenAI codebase

The environments in MPE2 descend from OpenAI's MPE codebase, and the difference between the two is mostly a matter of defaults that existing training scripts depend on. According to the project description, the fork carries several minor fixes that are mostly about making the action space discrete by default, making the rewards consistent, and cleaning up the observation space of certain environments. On top of that, MPE2 adds three new environments.

That sounds like a light touch, but for a research package the action space is the part every policy touches. The README states plainly that OpenAI's MPE uses continuous action spaces by default, and MPE2 reverses that. A script written for the original code, or for any tutorial that hardcodes a continuous action distribution, will not behave the same way after the switch. The reward consistency fix matters just as much in practice, because a sign or scale change moves the learning curve without changing any code a researcher wrote.

There is also a packaging-level shift. The project lists dependencies on pettingzoo, pygame-ce and scipy, and it requires Python 3.11 or newer. The original code predates the PettingZoo API entirely, so the port is not a drop-in file copy. If you are migrating, treat it as an API migration rather than a version bump.

Where communication sits inside the action space

MPE2 is a communication-oriented package, and the interesting part is that a message is not a separate channel bolted onto the interface. It is folded into the action itself. The README describes the discrete action space as a representation of combinations of movements and communications an agent can perform: agents that can move choose between the four cardinal directions or do nothing, and agents that can communicate choose between two and ten environment-dependent communication options that broadcast a message to all agents that can hear it.

So a single action value selects both where a body goes and what it says. In Simple Crypto that broadcast is how Bob and Eve signal that they have reconstructed the message. The number of options varies per environment, which means the action space size is not something you can assume from the agent count alone.

With continuous_actions=True the same coupling applies in continuous form. An agent that moves inputs a velocity between 0.0 and 1.0 in each of the four cardinal directions, where opposing velocities such as left and right are summed together. An agent that communicates outputs a continuous value over each communication channel it has access to. The README also notes that visibility gates delivery: a message reaches only agents allowed to see it.

Observations that shrink when an agent is not visible

Most multi-agent packages hand every agent the same observation layout. MPE2 does not, and the project documentation says so explicitly. When an agent is visible to another agent, the second agent's observation contains the first agent's relative position, and in Simple World Comm and Simple Tag it also contains the first agent's velocity. If an agent cannot see or observe the communication of a second agent, that second agent is not included in the first agent's observation space, resulting in different agents having different observation space sizes in certain environments.

Visibility is also described as temporary in one case. When an agent is temporarily hidden, which the README says is only possible in Simple World Comm, that agent's position and velocity are set to zero. A zeroed vector is not the same as an absent entry, so an observer has to know which convention a given environment uses before it designs a masking or attention layer.

The general observation vector is documented as a composition of the agent's own position and velocity, other agents' relative positions and velocities, landmarks' relative positions, the types of landmarks and agents, and communications received from other agents. The exact form is left to each environment's own documentation, which means you still have to read the per-environment page before fixing an input dimension.

The map scale that sets the reward scale

The geometry of these worlds is simple and worth internalising before tuning anything. Landmarks and agents typically start out uniformly randomly placed from -1 to 1 on the map, which the README notes means they are typically around one to two units apart. That single fact drives the rest of the reward design, since rewards in many of these environments depend on distance.

Landmarks are static circular features that agents cannot control. Depending on the environment they act as destinations, where proximity changes an agent's reward, or as obstacles that block agent motion.

Color is worth a note for anyone reading a rendered episode. Since all agents are rendered as circles, color is the only way a human can tell them apart, and the color of agents is described in most of the environments. The color is not observed by the agents, so it is a debugging aid for you, not a feature of the observation vector. That distinction matters when you build a scripted policy or try to map a screenshot back onto a state.

Running the Simple Push loop for the first time

Installation is a single package install, and the project states that the unique dependencies for this set of environments can be installed with:

pip install mpe2

Because Python 3.11 or newer is required, check your interpreter before installing if you are on an older environment. The wheel pulls in PettingZoo, pygame-ce and scipy automatically.

The first real use is the Simple Push environment with random agents. The README gives this example:

from mpe2 import simple_push_v3

env = simple_push_v3.env(render_mode="human")
env.reset(seed=42)

for agent in env.agent_iter():
    observation, reward, termination, truncation, info = env.last()

    if termination or truncation:
        action = None
    else:
        # this is where you would insert your policy
        action = env.action_space(agent).sample()

    env.step(action)

env.close()

Read that loop carefully, because the shape of it is the API. `env.last()` returns the observation, reward, termination, truncation and info for the agent currently being stepped, and `env.action_space(agent)` gives a per-agent space, which is why you sample inside the agent loop rather than once before it. When an agent is done you pass `None`, and the loop advances to the next agent. The `render_mode="human"` argument opens the window described in the rendering section, and `env.close()` releases it.

Why every episode stops at 25 cycles by default

Episodes in this package are short and bounded by design. The game terminates after the number of cycles specified by the `max_cycles` environment argument is executed, and the default for all environments is 25 cycles, kept as in the original OpenAI source code.

That number has consequences for training. With 25 cycles as the horizon, a policy gets very little room to demonstrate long-horizon coordination, and the cooperative environments, where agents must work together to reach their goals and receive a mixture of rewards based on their own success and the success of other agents, will be hard to train unless you raise `max_cycles` deliberately. The same goes for the adversarial set, where Simple Adversary, Simple Crypto, Simple Push, Simple Tag and Simple World Comm are described as adversarial in the sense that rewarding a good agent punishes an adversary agent and the reverse holds, though not always in a perfectly zero-sum manner.

In the adversarial environments there are good agents rendered in green and an adversary team rendered in red. The six cooperative environments are Collect Treasure, Simple Formation, Simple Line, Simple Reference, Simple Speaker Listener and Simple Spread. That is eleven environments in total, counting the three that MPE2 adds on top of the inherited set.

Rendering, output inspection, and the limits of the docs

Rendering opens a window that automatically grows if agents wander beyond its border, and communication is rendered at the bottom of the scene. The `render()` method also returns the pixel map of the rendered area, which is the documented route to pixel observations if you need them.

The documentation is uneven, and you should plan around that. The per-environment pages are where the exact observation form, the action space details and the landmark behaviour are documented, and the README itself defers to them for each of the eleven environments. Things a new user usually wants are not stated in the README at all: it does not document how to save or reload a trained policy, it does not document a vectorised or parallel training wrapper, and it does not document how to change the environment map size or landmark count. The README also does not document rollback of a package release.

Rendering also has a dependency footprint you should know about. The package depends on pygame-ce for the window and on scipy as a numeric dependency, and the testing extra pulls in pytest, pytest-cov, pytest-xdist, pynput and pytest-markdown-docs, with a pre-commit configuration at the repository root and black as the configured code style.

Licence, packaging, and the shape of the release history

MPE2 is released under the MIT License, and the project lists the MIT Licence with Farama Foundation as the author and [email protected] as the contact address. The repository is not archived, and the last push was on 2026-10-09.

The release history is short and recent. Version 1.0.0 shipped on 2026-04-03, version 1.1.0 followed on 2026-05-02 with the note Mature Farama Release, and version 1.1.1 landed on 2026-09-11. The package classifies itself as Development Status 5 - Production/Stable and lists support for Python 3.11 through 3.14.

Version numbers on a fork like this one are not a signal of feature count. The whole project is one package with one dependency set and eleven environments, so the work of upgrading is small in volume. The cost is compatibility: the 1.0.0 boundary is where the discrete action default and the PettingZoo base settled, and if you carry code from the original MPE repository, the effort sits in the action handling rather than in learning new environment APIs.

Editorial conclusion

MPE2 suits researchers who want the classic particle worlds on a current Gymnasium-style stack and can accept rewritten action handling. It does not suit codebases that assume the original continuous action space or the old reward signs. Before adopting, run the Simple Push loop from the README, then diff the action space of every environment you use against your old scripts, since the discrete default is the change most likely to break a pipeline.

Frequently asked questions

Official sources

1. Farama-Foundation/MPE2 on GitHub
2. License: MIT
3. Project website
4. README
5. Releases