Unity ML-Agents: training game agents with reinforcement learning

What problem ML-Agents solves, and who it is built for

Most reinforcement learning libraries assume you can write your environment as Python code. That assumption breaks down when the environment is a game: physics, animation, navigation meshes and rendering live in the engine, not in a Python process. ML-Agents exists to close that gap. The README describes it as a toolkit that "enables games and simulations to serve as environments for training intelligent agents", with PyTorch-based implementations of the training algorithms. You attach components to GameObjects in a Unity scene, the scene becomes the environment, and a Python process drives training over a socket.

The stated audience is two groups. Game developers and hobbyists get prebuilt algorithms so they can train agents for 2D, 3D and VR/AR games without writing a learner. Researchers get a Python API for reinforcement learning, imitation learning, neuroevolution or custom methods. The README also lists non-gameplay uses: controlling NPC behaviour in multi-agent and adversarial settings, automated testing of game builds, and evaluating design decisions before release. That last use is the one teams underestimate. A trained agent that plays your level badly is a signal about the level, not only about the agent.

How the Unity side and the Python trainer talk to each other

The architecture splits into two processes. Inside Unity, the com.unity.ml-agents package provides the SDK: sensors that collect observations, an Agent class that receives actions, and a communicator that ships data out. On the other side, the mlagents Python package runs the trainer. The two are linked over a socket, and the README notes that the connection can be configured as a Unity environment controlled from Python or wrapped as a gym or PettingZoo environment.

Training is not the only path. The README describes an Inference Engine that provides native cross-platform support, so a trained model can run inside a built game without a Python process attached. That is the deployment story, and it matters: it means the artefact you ship is a model file consumed by the Unity runtime, not a service.

On the algorithm side, the README lists PPO, SAC, MA-POCA and self-play for reinforcement learning, plus BC and GAIL for imitation learning. PPO is the general-purpose choice for single and multi-agent setups; SAC is listed as an option, and the repository also ships a trainer plugin example under ml-agents-trainer-plugin/, which is the supported route if you want to add your own algorithm rather than fork the trainer.

Installing the Python package and the Unity package

The Python side installs from PyPI. Release 23 pairs the mlagents Python package at version 1.1.0 with Unity package 4.0.0, so pin the Python package rather than taking whatever is newest. The repository also contains a Dockerfile that installs both ml-agents-envs and ml-agents in editable mode from a checked-out SHA, which is how the project builds its own training image.

pip install -e /ml-agents/ml-agents-envs
pip install -e /ml-agents/ml-agents

Those two lines are the installation steps from the repository's Dockerfile, run against a checkout of the repository. On the Unity side, the README states that com.unity.ml-agents is verified for Unity 2020.1 and later, and that verified package releases are numbered 1.0.x. Add the package to your project, then attach the SDK components to the objects that should learn. The project ships 17+ example Unity environments, linked from the package documentation, and those examples are the fastest way to see a working scene before you modify your own.

The README also points at TensorBoard for watching a run, which is where you check whether the reward curve is actually moving before you spend hours on a longer configuration. The README does not document a rollback path if a training run or a package upgrade goes wrong, so keep your configuration files under version control separately from the package.

Where ML-Agents is the wrong tool

The coupling is the cost. If your environment is not a Unity scene, you are paying for an engine you do not need: the Unity Editor, the package version, and the socket link all become part of your training loop. A pure Python environment wrapped in a gym interface will train faster to set up and easier to debug.

Version alignment is the second trap. The README presents Release 23 as August 28, 2025 with Python package 1.1.0 and Unity package 4.0.0, and it points to a separate Migration page for upgrades. A mismatch between the Python package and the Unity package is not a configuration detail; it is the failure mode. The README also labels the develop branch as under active development and potentially unstable, so pinning to a release tag rather than develop is the safer default for anything you intend to reproduce.

Documentation has moved. The README carries a migration notice stating that the web docs at unity-technologies.github.io/ml-agents are deprecated and that Unity Package documentation is now primary. Tutorials and blog posts written against the old site may describe APIs that no longer match the current package, and the repository keeps a localized_docs/ directory alongside docs/, which means the two can drift.

Finally, the licence badge in the README says Apache-2.0, while the repository metadata reports NOASSERTION. The LICENSE.md file is the authority here. If your organisation has strict licence review, resolve that discrepancy before adoption rather than after.

Alternatives and the real difference in approach

The closest alternative in spirit is a Python-first RL library such as Stable-Baselines3 paired with a Gymnasium environment. The difference is where the environment lives. With Stable-Baselines3 you write the environment in Python, and the simulator is whatever you can wrap in the Gym interface. With ML-Agents the environment is a Unity scene, and the Python side is a trainer that connects to it. You get physics, rendering and multi-agent scenes for free; you give up the ability to run the whole loop in one process.

For multi-agent work specifically, the README positions MA-POCA and self-play as first-class options, and the PettingZoo wrapper means a Unity scene can be presented to code written against that API. That is a narrower claim than "ML-Agents is a general RL framework": it is a bridge between an engine and a trainer, and it is strongest exactly where that bridge is needed.

Maintenance, releases and upgrade cost

The repository is not archived. The last push to the default branch was on 2026-09-02, and the most recent tagged release in the list is release_23_tag, published on 2025-09-02. Release cadence in the listed history is roughly annual: release_21 in October 2023, release_22 in October 2024, release_23 in September 2025.

That cadence sets your upgrade budget. Because the Unity package and the Python package are versioned separately (4.0.0 and 1.1.0 for Release 23), an upgrade is two coordinated changes, not one. The README points to a Migration page for moving between releases, and the release notes on the Releases page carry the details. Budget for re-running a training configuration after the upgrade and comparing curves, because a changed default in the trainer does not announce itself in your code.

On licensing: the README shows an Apache-2.0 badge, and the repository metadata does not assert a licence. Read LICENSE.md and the Third Party Notices file at the repository root, which exists precisely because bundled components can carry their own terms. That is a factual starting point for your own review, not a legal conclusion.

Editorial conclusion

Adopt ML-Agents when the environment you care about is already a Unity scene: the SDK, the Inference Engine and the Python trainer are designed around that pairing. Do not adopt it if you only need a Python RL loop over a non-Unity simulator, because the Unity package and the Editor become mandatory parts of your workflow. Before committing, verify that your Unity version satisfies the package requirement (the README states com.unity.ml-agents is verified for Unity 2020.1 and later), and confirm the Python package version that matches your release: Release 23 pairs mlagents 1.1.0 with Unity package 4.0.0.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Releases
5. Unity-Technologies/ml-agents on GitHub