light_mappo: MAPPO with the environments taken out

The argument is a file count, and it is a good one

The README answers a question a newcomer would ask anyway, which is why there is another MAPPO repository at all. Its answer is a comparison table.

The original on-policy implementation ships with five environment families, SMAC, MPE, Hanabi, GFootball and MAMuJoCo, glued throughout, and adding your own environment means tracing imports through every runner, buffer and config block. light_mappo does the opposite. The training stack is fixed and the environment is the one variable.

The table puts numbers on it. Python files, 200 or more against about 30. Bundled environments, five against one demo plus yours. Plug-in environment effort, rewriting a scenario, a registry and a runner against filling in one file. External services, wandb, a StarCraft II binary and RoboSchool, against none required. And time to first trained policy on your environment, hours against minutes.

That last row is the one that matters practically, and the demo is the proof. The project ships a UAV round-up environment where three hunter agents pursue an evading target, and the README says the policy was trained in 150K environment steps and captured the target in 49 steps, with the training command itself taking roughly three minutes on a modern CPU.

A three minute training run on a CPU is the whole pitch. It means the loop is short enough to iterate on, and iteration is what you actually need when you are bringing a new environment into a training stack for the first time and want to know whether the algorithm works before you spend a day on configuration.

The framing in the README is also deliberate. If you want to compare against MAPPO numbers on standard benchmarks, use the original. If you want to drop MAPPO into something new, start here. That is a clear statement of what the project is for and, just as usefully, what it is not.

The central training diagram, and what centralised actually means

The how it works section is a diagram with two labelled halves, and the two halves are the entire point of MAPPO.

Decentralised execution means that at inference each actor sees only its own observation. Centralised training means the critic sees the joint state, so it can credit assign properly. The critic in the diagram is V of the concatenation of all the agents observations, and the actors are drawn as separate boxes for o_1, o_2 and o_3, annotated as sharing parameters or being separated.

The parameters shared annotation matters, because the repository ships two runners, one under runner/shared for a single policy used by all agents and one under runner/separated for one policy per agent. So the choice between parameter sharing and per-agent policies is a directory you pick rather than a flag you set, which is a sensible way to handle a decision that changes the shape of the network.

Underneath, the documentation says there is PPO machinery, specifically GAE, the clipped ratio, value clipping, an optional Huber loss and optional PopArt or ValueNorm. The features table later confirms it, listing shared and separated policies as available in both runners, and a recurrent policy using a GRU or a naive recurrent option.

The action space coverage is broad for a small implementation: Box, Discrete, MultiDiscrete and MultiBinary are all listed, and the repository has a utils/distributions directory. Handling those four space types correctly is most of the work in a from-scratch PPO implementation, and it is the work that gets skipped when you plug an existing library into a new environment.

So the architecture is standard MAPPO, and the value of the project is not the algorithm but the absence of everything around it.

Writing one file is the whole integration contract

The plugging in your environment section is the part of the README that a reader with a real environment will care about most, and it is short enough to quote the shape of.

The specification is that envs/custom_env/env_core.py is the only file you need to write, and the class is called EnvCore with four fields set in the constructor: agent_num for how many agents, obs_dim for the per-agent observation size and action_dim for the per-agent action size. The reset method returns a list of length agent_num where each element has shape (obs_dim,). The step method takes a list of length agent_num where each element has shape (action_dim,) and returns a list of obs_list, reward_list, done_list and info_list.

That is a small contract, and it is deliberately smaller than a gymnasium environment, which is why the wrappers exist. For continuous actions, env_continuous.py already wraps the core into the per-agent gym.spaces.Box lists that the trainer expects. For discrete actions, the documentation says to swap one import in train/train.py:_build_single_env.

So there are two shapes of work. A continuous-action environment needs a core plus the wrapper. A discrete-action environment needs a core and a one-line import change. Neither requires touching the training loop.

The environment wrappers directory also carries a DummyVecEnv described as shared infrastructure, which is the vectorisation layer that lets the trainer step many copies of an environment at once.

The UAV demo is described as the worked example of a slightly more complex case, with heterogeneous observation sizes, a scripted opponent agent and matplotlib based rendering. That is the case that breaks naive assumptions, and it is included as a demonstration rather than as a toy.

Training and rendering are two separate commands

The quick start is a three step sequence, and the third step is the one that makes the project feel finished rather than half done.

The dependencies go in with pip install -r requirements.txt, and the training run is a single command:

python train/train.py --env_name uav --experiment_name uav_demo \
    --num_env_steps 150000 --episode_length 100 \
    --n_rollout_threads 8 --hidden_size 64 --layer_N 1

The flags are visible in that one line and they tell you the scale of the default configuration: eight rollout threads, a hidden size of 64 and a single layer, with 150,000 environment steps and an episode length of 100. That is a small network trained for a small number of steps, and the three minute figure follows from those choices rather than from any clever optimisation.

Rendering is a different script, which loads a checkpoint and writes a video:

python scripts/render_uav.py \
    --model_dir results/uav/MyEnv/mappo/uav_demo/run1/models \
    --video_path videos/uav_demo.mp4 \
    --hidden_size 64 --layer_N 1 --num_episodes 3

Separating training from rendering is the right call for a project whose argument is iteration speed. You train, you look at the video, you change something, you train again. The model_dir path also shows the experiment naming scheme, with the environment name, the algorithm, the experiment name and a run number in the path, which means repeated runs of the same configuration accumulate rather than overwrite.

The README's instruction after the commands is simply to open videos/uav_demo.mp4 and see what is in the GIF, on your own machine. The GIF in the README shows the same run rendered by the project, and the demo description says it was captured in 49 steps with three hunters, one evading target and three obstacles.

The dependency list is itself a design statement

The requirements file is short enough to read in one go, and it explains the no wandb, no SMAC, no GFootball promise.

The core is torch at 2.0 or newer and numpy at 1.24 or newer. The action and observation space layer is gym at 0.26 or newer and gymnasium at 0.29 or newer, the latter noted as used by the UAV environment. Logging is tensorboardX at 2.6 or newer, which is the local writer rather than a hosted experiment tracker.

The rendering group is imageio at 2.30 or newer with the imageio-ffmpeg mp4 backend at 0.4 or newer, matplotlib at 3.7 or newer and Pillow at 10.0 or newer. The comments in the file say these are used by the render script and the UAV environment.

Notice what is absent. There is no wandb client, no StarCraft II, no RoboSchool, and no terminal UI framework, which is a deliberate design choice: the tool should be readable on a laptop, and every dependency is a thing a reader has to understand before they can follow the training code.

There is also no packaging to speak of. The repository listing is a Python project laid out directly, with algorithms, envs, runner, scripts, train and utils directories, a config.py, a requirements.txt, a LICENSE, a README and a README_CN.md, and there is no setup.py, no pyproject and no build directory. That is what a project that expects you to run it from a clone looks like, and it is consistent with the training commands being given as python train/train.py rather than as a console entry point.

The Chinese README and a bilibili walkthrough badge sit alongside the English one, and the project licence is MIT.

Where light_mappo is the wrong tool

Three situations rule it out.

The first is if you are reproducing published numbers. The README is direct that if you want to compare against MAPPO numbers on standard benchmarks you should use the original implementation, and it is right. A baseline that has been restructured for readability is not a baseline, because the structure of the training code affects seeding, batching, normalisation and therefore the result. The projects measure different things and only one of them is comparable to a paper.

The second is if you need scale. The demo is 150,000 steps, 64 hidden units, one layer and eight rollout threads, and the README is honest that this is a CPU run of about three minutes. Nothing in the documentation describes distributed training, a multi-GPU setup, or a large-scale configuration, and the project positions itself as the local and quick option. If your environment is expensive to step, or your problem needs millions of steps, the thing you have optimised for is the wrong thing.

The third is multi-threading within one environment. The crate is single-threaded in the sense that matters here, and the vectorisation is done through a DummyVecEnv in Python, which is the same approach the original uses but without the surrounding infrastructure. For a fast simulator that steps natively, the Python side will be the bottleneck, and the documentation does not address that.

There is one more practical limitation that the repository layout implies rather than states. The recurrent policy option exists, the features table lists GRU and naive recurrent, and the utils directory has an rnn module, but the shared and separated runners and the CustomEnv wrapper are the parts you would be trusting with your environment. The UAV demo is included as the worked example for exactly the awkward cases, so it is the first thing to read and the first thing to run.

Editorial conclusion

light_mappo fits a researcher who has an environment and wants cooperative multi-agent training running against it today, and it fits an engineer who wants to read the MAPPO implementation end to end in an afternoon. It is the wrong choice if your goal is reproducing published MAPPO numbers on SMAC or Hanabi, because the documentation itself sends you to the original implementation for that, and it is the wrong choice if you need multi-threaded training, since the project positions itself as a local, CPU friendly setup rather than a distributed one. Before adopting it, write your own EnvCore against the four-field contract and check that your observation layout survives the shared and separated runners, because the UAV demo exists precisely because heterogeneous observation sizes are the case that breaks the simplest assumptions.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. tinyzqh/light_mappo on GitHub