CyberBattleSim for reinforcement learning research

What CyberBattleSim's enterprise abstraction represents

CyberBattleSim addresses a specific research problem: how automated attacker and defender agents interact inside a simulated enterprise network without granting those agents direct access to a real system. It is intended for experimentation with reinforcement learning, particularly questions about lateral movement, topology, configuration, detection, and mitigation. The README also identifies credential storage and retrieval as an agent-design problem worth studying.

The enterprise abstraction consists of computer nodes, a fixed network topology, and planted vulnerabilities. An attacker tries to take ownership of network nodes by exploiting vulnerabilities that permit movement between them. A defender watches the same activity and attempts to detect the attack and limit its effects by evicting the attacker. The defender is described as stochastic because detection and mitigation outcomes depend on predefined success probabilities rather than on a packet-level security implementation.

This makes CyberBattleSim a research environment, not a network scanner, intrusion detection product, or digital twin of a particular enterprise. Its deliberate simplification prevents direct application to real systems, according to the project documentation. That boundary is central to adoption. A result can explain behavior within this model, but it cannot by itself establish how a trained agent would behave against real services, traffic patterns, identity systems, firewalls, or unmodeled vulnerabilities.

Attacker, defender, and simulation-step mechanics

The simulation advances through discrete steps. During an episode, the attacker selects actions available in the abstract environment and attempts to expand control by using planted vulnerabilities. The defender observes network activity, evaluates whether it detects an ongoing attack, and can initiate mitigation. Re-imaging an infected node models remediation. This operation spans multiple simulation steps, so remediation is temporal rather than instantaneous.

The Python environment exposes an Open AI Gym-style interface, allowing reinforcement learning algorithms to interact with observations, actions, rewards, and episode progression. The repository includes basic agents as starting points, but the README frames newer reinforcement learning methods as a subject for investigation rather than a single standardized agent implementation. One stated difficulty is the large action space inherent to computer-system environments. A related difficulty is memory: conventional reinforcement learning agents do not naturally retain or retrieve credentials unless the environment and algorithm expose the necessary state and decision mechanism.

CyberBattleSim evaluates agent performance through two measures: the number of simulation steps required to attain the goal and cumulative rewards across steps and training epochs. Those measures are useful for repeatable comparison inside the simulator. They are not measurements of compromise time, detection quality, business impact, or performance against production traffic, because the README explicitly says that actual network traffic is outside the current model.

Linux setup and the `cybersim` Jupyter kernel

The supported development path is Linux, either native or through WSL on Windows. Direct Python use on Windows is described as unsupported. After cloning the repository, the initialization script installs a uv-managed Python 3.10 environment and its dependencies, then registers a Jupyter kernel named `cybersim`.

The clone creates the local project directory:

git clone https://github.com/microsoft/CyberBattleSim.git

Run these commands from the cloned repository. The first installs uv through the repository's script, and the second initializes the virtual environment and Jupyter kernel:

bash install-uv.sh
bash init.sh

Open a new terminal after running `install-uv.sh`, as directed by the README. Once initialization completes, activate the environment:

source .venv/bin/activate

The expected local environment is `.venv`, with Python 3.10 managed through uv and a Jupyter kernel called `cybersim`. The dependency set includes `gymnasium==0.29.1`, `numpy==1.26.4`, `networkx==3.4.2`, `pandas>=2.2.2`, Jupyter-related tooling, and plotting libraries. Plotly static export can require native Linux libraries. If export reports a missing shared library such as `libgdk_pixbuf-2.0.so.0`, the README supplies this installation command:

sudo apt install libnss3-dev libgtk-3-0 libxss1 libasound2-dev libgtk2.0-0 libgconf-2-4

The README directs users to `docs/quickintro.md` for the first simulation walkthrough. It does not provide a complete first-run notebook sequence in the installation section, so an exact initial API call would require consulting that guide or an example under `notebooks/`.

Running the research environment through Docker

The repository includes a Dockerfile for a self-contained Linux environment based on Ubuntu 22.04. It copies uv from its container image, creates a Python 3.10 virtual environment under `/opt/venv`, installs both runtime and development requirements, installs CyberBattleSim in editable mode, and generates type stubs. This is relevant for researchers who prefer a reproducible container boundary or need the development dependencies and generated stubs.

The build command is documented directly in the Dockerfile:

docker build -t cyberbattle:1.1 .

After the image builds, start an interactive shell:

docker run -it --rm cyberbattle:1.1 bash

The container name used in the Dockerfile is `cyberbattle`, while the package metadata in `setup.py` names the distribution `cyberbattlesim` at version `0.1.0`. No GitHub releases are listed, so the Docker tag shown in the file should not be interpreted as a published release series. It is a local image tag in the documented build command.

The package is classified for scientific, engineering, and artificial-intelligence use, and it is marked OS Independent in `setup.py`, but the project documentation still treats Linux as the supported development environment. Container use does not remove the abstraction limits. It changes how dependencies are installed and where commands run, not what the simulator models.

Topology sensitivity and the cost of a fixed environment

The fixed topology is both a parameter and a constraint. CyberBattleSim is designed to investigate how topology and configuration affect lateral movement, so researchers can vary the environment in which those experiments occur. At the same time, the current implementation describes a high-level abstraction rather than actual network traffic. Conclusions depend on the planted vulnerabilities, permitted actions, topology, reward design, and defender assumptions.

That creates a concrete failure mode for comparative research. An algorithm that performs well in one topology may be exploiting structural regularities rather than learning a behavior that transfers to another. A large action space can also make training or exploration expensive, and memory limitations can prevent an agent from implementing credential-based strategies unless the experiment explicitly handles stored knowledge. The project documentation names both issues but does not claim that CyberBattleSim solves them.

The stochastic defender introduces another dependency on experimental controls. Detection and mitigation are governed by predefined probabilities, so results can reflect both agent quality and the chosen probability settings. Comparisons should preserve the topology, vulnerabilities, defender probabilities, reward definition, and step budget, or treat changes as separate experiments. The documented metrics support comparison, but they do not eliminate the need to report those conditions.

When CyberBattleSim is the wrong security tool

CyberBattleSim is the wrong tool when the objective is to model real packet behavior, validate an intrusion detection system, reproduce a production topology, or train an agent for direct operation against enterprise hosts. The README explicitly identifies the absence of actual network traffic modeling as a significant limitation. The project also says its abstraction is intentionally simplistic and unsuitable for direct application to real-world systems.

A conventional security sandbox or digital-twin approach takes a different route. Such an environment may represent hosts, traffic, identity, security controls, and realistic services more closely, at the cost of greater setup work and less control over experimental variables. That route is more appropriate when fidelity to real network interactions matters. CyberBattleSim is more appropriate when researchers need to change abstract security relationships quickly and keep automated agents away from operational systems.

The built-in defender is not a replacement for a production detection or response system. Its behavior is abstracted through stochastic detection and multi-step re-imaging. It should therefore be read as a controlled baseline for agent interaction experiments. The repository does not document integration with an external SOC, SIEM, firewall, identity provider, or production telemetry pipeline.

Maintenance, upgrades, and the MIT package boundary

The repository is not archived, and its last push was on 2026-10-16. That supports treating the source tree as recently changed, although there are no GitHub releases against which to assess versioned upgrade cadence. Upgrade planning should account for the absence of releases and inspect changes on `main` directly.

`setup.py` reports package version `0.1.0`, but dependency compatibility remains important because the repository pins several packages exactly and allows compatible ranges for others. `gymnasium==0.29.1`, `numpy==1.26.4`, and `networkx==3.4.2` are fixed, while `matplotlib` and `pandas` use minimum-version constraints. Running `bash init.sh` after a source update is the documented initialization path, but the README excerpt does not describe a dedicated upgrade or migration procedure.

The project uses the MIT License. That license supplies the repository's stated permission and condition framework, but adopting a modified environment still requires preserving the applicable license notice and reviewing included dependencies separately. The Microsoft name identifies the project origin; it does not imply a production support commitment or compatibility with a particular enterprise environment. Repository activity and licensing should be evaluated separately from experimental fitness.

Editorial conclusion

Researchers studying agent interaction, network topology, and lateral movement abstractions should adopt CyberBattleSim when an intentionally simplified environment fits the experiment. Security engineers should not treat it as a reproduction of their production networks. Verify first that the fixed topology, vulnerability model, action space, reward metrics, and stochastic defender assumptions match the research question, then confirm that the maintained `main` branch and current dependency set are acceptable for the notebook workflow.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. microsoft/CyberBattleSim on GitHub
4. README