# HydroGym, reinforcement learning environments for CFD solvers

> HydroGym is a MIT-licensed Python platform that wraps six different computational fluid dynamics solvers in a Gymnasium-compatible interface, so an RL agent can drive flow control problems from cylinder wakes to shock-vector control. It ships 61 environments and asks you to run it from pre-built Docker images rather than from source.

**dynamicslab/hydrogym** — An RL-Gym for Challenge Problems in Data-Driven Modeling and Control of Fluid Dynamics.

- Repository: https://github.com/dynamicslab/hydrogym
- Website: https://dynamicslab.github.io/hydrogym
- Stars: 556 · Forks: 80
- Language: Python
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/dynamicslab-hydrogym

## An RL gym for solvers that were not written for RL

HydroGym is described as an RL gym for challenge problems in data-driven modeling and control of fluid dynamics, and the README summarises it as a platform for applying reinforcement learning to fluid dynamics and flow control.

The problem it addresses is specific. A CFD solver is a serious piece of numerical software with its own mesh, its own time integration and its own parallel decomposition, and it exposes actuation through parameters like a jet velocity or a surface wave amplitude. Turning that into a reinforcement learning environment means solving four problems: making the solver fast enough to run millions of steps, giving the agent an observation vector and an action space that map onto the physics, running the solver inside the training loop without stalling it, and exposing the whole thing through an API that an off-the-shelf RL library understands.

HydroGym's answer is to wrap six existing solvers rather than write a new one. The key features list names them: Finite Element through Firedrake, Lattice Boltzmann through MAIA LBM, Finite Volume through MAIA Structured FV, Spectral Element through NEK5000, and fully differentiable solvers through JAX-Fluids, with the JAX backend covering the differentiable case.

The interface is Gymnasium-compatible, which the README says works with Stable-Baselines3, RLlib and other RL libraries. That is the decision that makes this usable rather than a research artefact: the agent, the vectorised environment, the normalisation wrapper and the training loop all come from libraries that already exist, and what HydroGym supplies is the environment.

The scope is broad. The summary line says 61+ environments, 6 solver backends, 2D and 3D, ready for RL training, and the key features add that the environments range from canonical benchmarks to turbulent flows, with cases reaching a Reynolds number of 400,000. The licence is MIT, the documentation is at dynamicslab.github.io/hydrogym, and the last push to the repository was on 2026-10-21. The recorded releases are v1.0.1 from 2026-04-24, titled Full Multi-Solver RL Environments Release, and v0.1.2.1 from 2022-12-26, so the bulk of the work landed in the v1.0.1 release from April 2026, which followed a long gap with only minor releases.

## The environment table does not add up, and that is worth knowing

The available environments section says HydroGym provides 61 environments across 6 solver backends, and then gives a table with a count per backend.

| Solver Backend | Count | Description | Dimensions |
|----------------|-------|-------------|------------|
| Firedrake (FEM) | 20 | Canonical flow control benchmarks | 2D |
| MAIA LBM | 55 | Lattice Boltzmann method environments | 2D, 3D |
| MAIA Structured FV | 8 | High-Reynolds turbulent boundary layers | 3D |
| NEK5000 | 2 | Spectral element turbulent channel flow | 3D |
| JAX | 2 | Differentiable fluid dynamics | 2D, 3D |
| JAX-Fluids | 2 | Compressible shock vector control | 2D, 3D |

Those counts sum to 89, not 61, and the discrepancy is not a rounding artefact. A reader planning a study who trusts the headline number will misjudge how much there is to choose from, and a reader who trusts the table will not know which number the code actually produces.

The likely explanation is that the two figures count different things, the headline being environments actually registered with Gymnasium and the table being configurations or cases per backend, but the README does not say so. The honest reading is that neither number is safe to quote without checking the source.

What the table does establish clearly is the shape of the coverage. Firedrake supplies the bulk of the canonical 2D benchmarks and is the only backend marked as 2D only. The MAIA LBM entry is by far the largest, at 55, and is the one with both 2D and 3D cases. MAIA Structured FV and NEK5000 are the turbulent, high-Reynolds entries and are 3D only, which is where the compute cost lives. JAX and JAX-Fluids have two environments each and are the differentiable ones, meaning the observation can be backpropagated through, which is a different research model from reinforcement learning against a black-box simulator.

So the practical advice is to pick the backend first and count afterwards, using the environment categories as the guide rather than the totals.

## The problem set, from cylinder wakes to shock vectors

The environments are grouped into four categories, and the categories tell you what kind of paper each one can support.

The canonical benchmarks are the low to mid Reynolds number cases: cylinder wake at Re 100 to 3900 in 2D and 3D, rotating cylinder at the same range, pinball at Re 30 to 150, cavity flow at Re 4140 to 7500, a backward-facing step at Re 600, square cylinder, sphere and cube at Re 300 to 3700, and turbulent channel flow at Re_tau 206 in 3D.

The airfoil control group is NACA0012, steady and with a gust disturbance, at Re 100 to 50000 and angles of attack from 12 to 40 degrees.

The high Reynolds number group is where the machine requirements jump: zero-pressure-gradient turbulent boundary layer with jet or surface wave actuation at Re_Tau 180 to 2200, a DRA2303 airfoil with jet or surface wave actuation at Re 400,000 and Mach 0.2 to 0.7, and a NACA0012 with jet actuation at Re 200000. All of these are 3D.

The fully differentiable flows group covers shock-vector control in a single divergent nozzle at Mach above 1.0, turbulent channel flow at Re_tau 180, and Kolmogorov flow up to Re 1000.

Two things stand out about that list. First, the actuation is physical rather than abstract. The environments are actuated by jets and surface waves, which means the agent's action is a forcing on the flow rather than a change to a solver parameter, and the observation is presumably some functional of the flow field. Second, the Reynolds numbers are wide enough that a result does not transfer between environments, so the benchmark set is really a set of separate problems rather than one problem at several difficulties.

The observation and action encoding is the thing the README does not spell out, and it is the thing to check in the documentation before choosing an environment, since an agent's performance is bounded by how well the state is observed.

## Run it from the container, not from source

The quick start is unambiguous: the project strongly recommends using its pre-configured Docker containers for hassle-free setup.

```bash
# For NVIDIA GPUs (CUDA)
docker pull clagemann/hydrogym-nvhpc-26.1_cuda-12.9_hopper_blackwell:latest
# or
docker pull clagemann/hydrogym-nvhpc-26.1_cuda-12.9_turing_ampere:latest

# For AMD GPUs (ROCm)
docker pull clagemann/hydrogym-rocm-6.3.3:latest

# Run container
docker run -it --gpus all clagemann/hydrogym-nvhpc-26.1_cuda-12.9_turing_ampere:latest
```

The image names encode the software stack, and there is a lot in them. `nvhpc-26.1` is the NVIDIA HPC SDK version, `cuda-12.9` the CUDA version, and then the GPU architecture, with separate images for Hopper and Blackwell and for Turing and Ampere. A separate ROCm image covers AMD, at 6.3.3.

So the setup path is: find the image matching your GPU architecture and your vendor's stack, pull it, and run it with GPU access passed through. There is no pip install path described in the README for the solver backends, which makes sense for Firedrake, MAIA and NEK5000, since each of those needs a compiler toolchain and MPI configuration, and for JAX-Fluids, which needs a specific GPU stack. Bundling that per architecture is a large amount of work, and doing it once for the user is the sensible trade.

The examples follow the same pattern, with a shell script per backend that runs inside the container.

```bash
cd examples/firedrake/getting_started

# Test environment interactively
./run_example_docker.sh

# Train with Stable-Baselines3
./run_example_docker.sh train
```

The same shape appears for MAIA, NEK5000 and JAX, and the NEK5000 example goes further, showing a single-agent directory, a pettinzoo directory for multi-agent training and a zero-shot transfer learning directory. The MAIA example notes that the workspace preparation downloads from the Hugging Face Hub and that the run uses MPMD execution with one Python and one MAIA process on the GPU, which is the multi-process, multi-device decomposition that a lattice Boltzmann solver needs.

One dependency the containers do not remove is the environment checkpoints. The README says all required environment checkpoints are available via HuggingFace and are downloaded on the fly when an environment is first created, with an internet connection required, and that if there is no internet connection at runtime, such as on compute nodes in HPC clusters, you can pre-download the environment files as outlined in the MAIA example README.

## Training with Stable-Baselines3

The training example is a standard Gymnasium workflow, and reading it tells you what the environment configuration object looks like.

```python
from hydrogym import FlowEnv
import hydrogym.firedrake as hgym
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv, VecNormalize


# Create environment
def make_env():
    env_config = {
        "flow": hgym.Cylinder,
        "flow_config": {"mesh": "medium", "Re": 100},
        "solver": hgym.SemiImplicitBDF,
        "solver_config": {"dt": 1e-2},
        "actuation_config": {"num_substeps": 2},
    }
    return FlowEnv(env_config)
```

The configuration is nested by concern, which is the design worth noting. `flow` selects the physical configuration, here the cylinder at Re 100 with a mesh named medium. `solver` selects the time integration scheme, here SemiImplicitBDF, and `solver_config` carries its settings such as the time step. `actuation_config` holds the actuation settings, here a number of substeps.

That separation is what makes the environment catalogue useful rather than a pile of hardcoded cases. You can run the same flow with a different solver, or change the mesh resolution without rewriting anything else, and the solver choice is a config key rather than a fork of the environment.

Above that, the training code is unchanged Gymnasium boilerplate. The environment is wrapped in a `DummyVecEnv` over the `make_env` function, and then in `VecNormalize` with observation and reward normalisation, before the PPO model is constructed. There is nothing HydroGym-specific in the training loop, which is the point of being Gymnasium-compatible.

For the multi-agent cases, the NEK5000 example mentions the pettinzoo wrapper, which is a multi-agent API for PettingZoo rather than Gymnasium. That is a second interface convention alongside the Gymnasium one, and it matters because the RL libraries that support the two are different.

## The practical limits of this approach

Four constraints are worth stating, and they are all consequences of what the project is rather than shortcomings in how it is built.

The first is compute. A Reynolds number of 400,000 on a DRA2303 airfoil in 3D is a large CFD calculation, and reinforcement learning needs many thousands of such calculations. The project says as much by recommending HPC-targeted images and by listing exascale systems among the deployment targets, and the MPI-parallelised solvers with distributed RL training support are the answer to that. For a team without a cluster, the 3D turbulent environments are effectively out of reach and the 2D canonical cases are the realistic entry point.

The second is the network dependency. Environment checkpoints download on first creation, and the examples say internet connection required. On an HPC cluster where compute nodes have no outbound access, that means a staging step before the job runs. It is documented, and it is the kind of thing that bites on a shared cluster.

The third is the multi-process reality of the solvers. The MAIA example runs one Python process and one MAIA process on the GPU, which means the environment is not a single-process object and the training loop has to cope with a solver running alongside it. That is where the performance engineering in the key features list, the highly optimised GPU and CPU backends, comes from, and it also means the RL library has to tolerate an environment that spends real wall-clock time outside its own process.

The fourth is the maturity signal. The project classifiers say Development Status 4 - Beta, and the requires-python range is 3.10 to below 3.15, so Python 3.14 is the newest supported. The dependency set is small and sensible for a wrapper, with gymnasium at 1.2.3 or newer, numpy, scipy, pandas, control for the control theory side, dmsuite and huggingface-hub for the checkpoint downloads.

The alternatives are two, and they differ in approach. The first is to build your own environment around one solver, which gives you full control of the observation encoding and the action space but means reimplementing the Gymnasium wrapper, the checkpoint handling and the MPI plumbing for exactly one solver. The second is to use the differentiable JAX-Fluids environments and optimise through the solver rather than sampling policies, which sidesteps the sample-efficiency problem entirely at the cost of restricting the problem to differentiable physics and dropping the non-differentiable backends.

For a group that wants a benchmark comparison across many flow problems rather than one bespoke environment, the wrapping of six solvers behind one API is the thing you are buying.

## Conclusion

HydroGym suits a research group that wants to apply reinforcement learning to flow control without writing a solver interface first, and it suits a group that already has one of the six supported solvers and wants it exposed to an RL library. It does not suit a team with no GPU or HPC budget, since the recommended path is a pre-built container per architecture. Verify first that the environment count you were promised matches what you get, because the headline figure and the backend table in the README do not add up, that your solver backend is one of the six listed rather than one you would have to add, and that the checkpoints are staged ahead of time if your training runs on compute nodes with no outbound network.

## FAQ

### What is HydroGym?

It is an MIT-licensed Python platform for applying reinforcement learning to fluid dynamics and flow control. It wraps six CFD solvers in a Gymnasium-compatible interface, with environments ranging from canonical cylinder wake benchmarks to shock-vector control and 3D turbulent flows at Reynolds numbers up to 400,000.

### How do I install and run HydroGym?

The project strongly recommends its pre-built Docker images rather than a source install. Pull the image matching your GPU architecture, such as the nvhpc image for CUDA or the rocm image for AMD, and run it with docker run -it --gpus all, then use the per-backend example scripts such as run_example_docker.sh.

### Which CFD solvers does HydroGym support?

Six: Firedrake for finite element, MAIA LBM for lattice Boltzmann, MAIA Structured FV for finite volume, NEK5000 for spectral element, JAX for differentiable fluid dynamics and JAX-Fluids for compressible shock vector control. The MAIA LBM entry is the largest, with 55 configurations in the README table.

### How do I train an RL agent on a HydroGym environment?

Build a config dictionary with the flow, solver and actuation sections, wrap it in a FlowEnv, then use it with any Gymnasium library. The README example creates a cylinder at Re 100 with a SemiImplicitBDF solver, wraps it in DummyVecEnv and VecNormalize, and trains with PPO from Stable-Baselines3.

### Do HydroGym environments need an internet connection?

Yes for the environment checkpoints, which are downloaded on the fly from HuggingFace when an environment is first created. On HPC compute nodes with no outbound access, the README points to pre-downloading the environment files as described in the MAIA example README.

## Sources

- [dynamicslab/hydrogym on GitHub](https://github.com/dynamicslab/hydrogym)
- [License: MIT](https://github.com/dynamicslab/hydrogym/blob/main/LICENSE)
- [Project website](https://dynamicslab.github.io/hydrogym)
- [README](https://github.com/dynamicslab/hydrogym/blob/main/README.md)
- [Releases](https://github.com/dynamicslab/hydrogym/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/dynamicslab-hydrogym