HighwayEnv: Gymnasium Environments for Autonomous Driving Decision-Making

The gap HighwayEnv fills for driving RL

Training a driving policy usually means choosing between a full simulator with sensor rendering and vehicle dynamics, and a toy problem that has nothing to do with roads. HighwayEnv sits in between. It is a collection of environments for autonomous driving and tactical decision-making tasks, distributed as a Python package that plugs into Gymnasium. The decision layer is the point: when to change lanes, whether to yield at a merge, how to pick a gap at a roundabout, how to back into a parking spot. The README lists ten scenario families: highway, intersection, exit, lane-keeping, merge, parking, racetrack, roundabout, two-way, and u-turn, with variants for continuous control, connected lanes, multi-agent settings, and larger or oval layouts. The intended audience is reinforcement learning researchers and engineers who want a reproducible benchmark for those tactical choices, plus anyone comparing algorithms such as DQN, DDPG, value iteration, or MCTS, all of which the documentation lists under Agent Examples. It is not a driving simulator in the commercial sense, and the README never claims to be one.

How the environment loop and configuration work

The mechanism is the standard Gymnasium contract wrapped around a kinematic road model. You register the package's environments, call gym.make with an environment ID, and optionally pass a config dictionary that overrides defaults such as the number of lanes. The README's example passes config={"lanes_count": 3} to highway-v0. From there the loop is reset, step, and check terminated or truncated, exactly as in any Gymnasium environment. The configuration dictionary is the main control surface: the documentation page for each environment lists its options, and the README points there for the full list rather than reproducing it. Observations are configurable too, which matters because the choice changes what your algorithm has to learn from. The repository's search interest around observation types reflects that: the same scenario can be presented as kinematic features or as a rendered representation, and the documentation's environment pages are where that mapping lives. Traffic participants are scripted rather than learned, so an episode is a policy interacting with a fixed behavioural model. That is a deliberate simplification, and it is what keeps episodes fast enough for the kind of iteration the project is built around.

Installing HighwayEnv and running a first episode

Installation is a single package. The README gives pip install highway-env, and for uv users it gives two forms: uv add highway-env, which adds the dependency to a project and installs it, or uv pip install highway-env, which installs without touching project metadata. The README states that Linux and macOS are supported primarily, with Windows maintained on a best-effort basis, so a Windows failure is a known category rather than a surprise. The package requires Python 3.10 or newer according to pyproject.toml.

pip install highway-env

After installation, the README's usage example registers the environments, creates highway-v0 with three lanes and a human render window, then steps with random actions. Run it and you should see a window with the episode animation and the loop printing nothing until the episode ends and resets.

import gymnasium as gym
import highway_env

gym.register_envs(highway_env)

env = gym.make("highway-v0", config={"lanes_count": 3}, render_mode="human")
obs, info = env.reset()
for _ in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()
env.close()

Swap env.action_space.sample() for a policy and you have a training loop. The documentation's quickstart page extends this to Stable Baselines3 training and Colab notebooks, which is the practical next step if you do not already have a training harness.

Where HighwayEnv stops being the right tool

The environments model tactical decisions on a kinematic abstraction. There is no sensor simulation, no camera or lidar model, no tire or suspension dynamics, and no claim of fidelity to logged real-world trajectories. If your problem is perception, end-to-end driving from pixels to control in a photorealistic scene, or validating a controller against measured vehicle behaviour, this project does not address it and the README does not pretend otherwise. A second limitation is behavioural realism in traffic: other vehicles follow scripted rules, so a policy can exploit regularities that would not hold against human drivers, and results should be read as a benchmark of decision logic rather than a prediction of road performance. Platform support is uneven by the project's own statement, with Windows on a best-effort basis. Finally, the README does not document rollback or downgrade procedures, so if a release changes vehicle behaviour, as the v1.12 notes describe, pinning a version is something you would have to work out from your own package manager rather than from project documentation.

HighwayEnv compared with SUMO and CARLA

The closest alternatives differ in what they simulate. SUMO is a traffic simulation suite built around large road networks and macroscopic or microscopic traffic flow; it is the natural choice when the question is about network throughput, signal timing, or corridor-level effects, and it comes with its own simulation engine and tooling rather than a Gymnasium API. CARLA is a full driving simulator with rendering, sensors, and physics; it is the natural choice when perception or photorealistic evaluation is part of the task, at the cost of a much heavier runtime and setup. HighwayEnv makes the opposite trade in both directions: it gives up network-scale traffic modelling and sensor realism, and in exchange you get a small Python package, a Gymnasium interface, and scenarios that are cheap enough to run in a training loop without a separate simulator process. If your experiment is about a tactical decision under a handful of scripted neighbours, the extra machinery of the other two is overhead. If it is about traffic flow or perception, HighwayEnv is the wrong layer.

Maintenance, releases and the MIT licence

The project is not archived, and the last push to the default branch was on 2026-09-18. Releases are frequent and documented: v1.11 added Python 3.14 support and a new environment variant, v1.12 covered a vehicle behaviour fix, a generic merge environment, and Gymnasium-compliant render behaviour, and v1.12.1 was a bug-fix release covering the intersection environment and community contributions. The changelog is published at the documentation site, which is where an upgrade decision should start. The upgrade cost is mostly in behaviour changes rather than API churn: the v1.12 vehicle behaviour fix is exactly the kind of change that can shift benchmark numbers without breaking your code, so re-running a baseline after upgrading is the honest approach. The package is MIT licensed, which is permissive and imposes essentially no conditions beyond retaining the licence text; the project also asks that you cite it, with a BibTeX entry given in the README, but citation is a request rather than a licence term. None of this is legal advice; read the LICENSE file in the repository if the distinction matters to your organisation.

Editorial conclusion

Adopt HighwayEnv when your research question is about tactical decisions: lane changes, merging, roundabout entry, parking maneuvers, and how a policy behaves against scripted traffic. Do not adopt it if you need perception, sensor models, or vehicle dynamics validated against real data, because the project does not provide them. Before committing, verify three things: that your Python version meets the requires-python floor of 3.10, that the environment ID you intend to use appears in the documentation's environment list, and that the observation type (kinematics, grayscale image, or occupancy grid) matches what your algorithm expects. A useful first check is running the quickstart snippet with render_mode="human" on highway-v0 and confirming the episode terminates and resets as the README describes.

Frequently asked questions

Official sources

1. Farama-Foundation/HighwayEnv on GitHub
2. License: MIT
3. Project website
4. README
5. Releases