PDEBench: A Benchmark Suite for Scientific Machine Learning, and What It Costs to Run

What PDEBench Is Actually For

Comparing two neural operators is hard when each paper generates its own data, its own initial conditions and its own train/test split. PDEBench addresses that by fixing the problems, the data and the baseline models in one repository. The README describes it as providing a diverse and comprehensive set of benchmarks for scientific machine learning, including challenging and realistic physical problems, and states that the repository contains the code to generate datasets, upload and download them from the data repository, and train and evaluate baseline models. The target reader is a researcher or engineer who needs a number that can be compared against someone else's number. That is a narrower audience than the topic list suggests. Physics-informed neural networks, JAX and PyTorch all appear as topics, but the deliverable is a comparison surface, not a modelling toolkit. If you already have your own simulator and your own data pipeline, PDEBench mostly offers you a set of published reference problems and pretrained weights to sanity-check against.

The Two Data Generation Paths and Why They Are Separate

The repository splits data generation into two directories, and the split matters when you plan work. The older path lives in pdebench/data_gen and holds problem-specific scripts: gen_diff_react.py for 2D diffusion-reaction, gen_diff_sorp.py for 1D diffusion-sorption, gen_radial_dam_break.py for 2D shallow-water, and gen_ns_incomp.py for 2D incompressible inhomogeneous Navier-Stokes. Alongside those sit plot.py for visualising generated data, uploader.py for pushing it to the data repository, and a .env file that stores the Dataverse URL and API token. The README is emphatic that the filename must be exactly .env, meaning you remove the example suffix from the shipped filename. Simulation arguments are problem-specific and live in YAML files under configs, with the simulation scripts themselves under src, including swe for the shallow-water equation. The second path, data_gen/data_gen_NLE, covers 1D advection, Burgers, reaction-diffusion, 2D Darcy flow and compressible Navier-Stokes, with a utils.py holding boundary and initial conditions and one directory per equation. The two paths are not unified behind a single entry point, so you pick the directory that matches your equation rather than calling one command.

Installation: pip, conda, and the Version Pinning Problem

The simplest install is pip install pdebench from PyPI, or pip install . from a local clone after upgrading pip and wheel. Data generation dependencies are optional extras, selected by Python version: pip install "pdebench[datagen310]" or pip install "pdebench[datagen39]", with the local equivalents using a leading dot. The conda route is more explicit. Create an environment with python=3.9, then conda install deepxde hydra-core h5py -c conda-forge for training dependencies, then either a CUDA 11.7 PyTorch build (pytorch==1.13.1, torchvision==0.14.1, torchaudio==0.13.1, pytorch-cuda=11.7) or the cpuonly variant. Data generation adds clawpack, jax, jaxlib and python-dotenv. DeepXDE needs its backend set to PyTorch, which the README says to configure by following the DeepXDE installation documentation rather than by setting a key in this repository. The version story is the awkward part. The project officially targets Python 3.9, JAX 0.4.11, PyTorch 1.13.0 and CUDA 11.7, but a note dated 2026-03-30 says the maintainers have verified that the core codebase, forward training in pdebench/model, and data generation in data_gen_NLE work with Python 3.12, JAX 0.9.2, PyTorch 2.11.0 and CUDA 13.0. Compatibility for the remaining components is described as still under investigation. Read that as: the newer stack is partly checked, not certified.

The Hydra Data Path Bug You Will Hit First

The README devotes a separate section to a known defect rather than burying it in an issue. During forward training, model/fno/utils.py and model/unet/utils.py mis-handle the data path because Hydra modifies data_path at runtime. The fix is to switch to the commented-out lines: lines 185 to 188 in model/fno/utils.py and lines 185 to 187 in model/unet/utils.py. The note adds that using to_absolute_path in Hydra requires Hydra 0.11.0 or later. This is a small edit, but it tells you something about the repository's state. A benchmark whose baseline training path needs a manual source edit before it runs against your own data location is a benchmark maintained for reproducibility of published results, not for plugging into a production pipeline. Budget for reading the source of the model you intend to run, not just the config YAML.

Datasets, Pretrained Models and the Licence Gap

PDEBench publishes its data and weights separately from the code, on the University of Stuttgart's DaRUS repository. The dataset carries DOI 10.18419/darus-2986 and the pretrained models carry DOI 10.18419/darus-2987, both linked from the README with badge images. Because the artefacts are hosted externally, download speed and availability depend on that service, not on PyPI. The licence field in the repository metadata reads NOASSERTION, which means GitHub could not map the licence file to a recognised identifier. The README does not state a licence in the text supplied here. For a benchmark that people may redistribute or bundle into a paper's artefact evaluation, that ambiguity is a real friction point: check the licence file in the repository and the terms attached to the DaRUS records before you ship the data or the weights inside anything of your own. Nothing here is legal advice, and the two artefact records may carry different terms from the code.

Where the Benchmark Stops Being the Right Tool

The equation list is finite and visible in the directory names. If your problem is not 1D advection, Burgers, reaction-diffusion, diffusion-sorption, 2D Darcy flow, 2D diffusion-reaction, 2D shallow-water, or incompressible and compressible Navier-Stokes, PDEBench gives you no generator and no pretrained baseline for it. The README invites the community to extend the benchmark, which is an admission that coverage is a moving target rather than a guarantee. Two other limits are worth naming. First, the release history shows a single tagged release, v0.1.0, dated 2023-08-25, while the compatibility notes carry a 2026-03-30 date. The project is being touched, but it has not cut a second release, so if you depend on PyPI you are depending on the 0.1.0 artefact rather than on the current main branch. Second, the pretrained models are baselines for the published problems. Using them to argue that your architecture is better on a problem outside that set is not something this repository supports.

DeepXDE and the Difference in Approach

The nearest thing to an overlap in this stack is DeepXDE, which PDEBench lists as a training dependency installed from conda-forge and expects to run with a PyTorch backend. The two tools answer different questions. DeepXDE is a library for building and training physics-informed models, where the residual of the PDE is part of the loss and the network is pushed toward satisfying the equation itself. PDEBench is oriented the other way: it provides generated solution fields from numerical simulation, and the baseline models in pdebench/model learn a mapping from initial conditions and parameters to those fields. In the PDEBench setting the physics lives in the data, not in the loss. That distinction decides which one you reach for. If you have no simulation data and want a model constrained by the equation, DeepXDE is the closer fit. If you have or can generate solution fields and want to measure how well a surrogate reproduces them against a published baseline, PDEBench is the closer fit. Installing both is normal, since PDEBench already depends on DeepXDE for part of its training stack.

Maintenance Cost and What to Verify

The maintenance burden here is mostly environmental. You inherit a pinned stack (Python 3.9, JAX 0.4.11, PyTorch 1.13.0, CUDA 11.7) that the maintainers say they have partly verified against Python 3.12, JAX 0.9.2, PyTorch 2.11.0 and CUDA 13.0, with the remaining components still under investigation. That means any upgrade you attempt is your own experiment. The README also notes that JAX is roughly six times faster than PyTorch for simulations in the authors' tests, which is a claim about their setup and not a guarantee for yours; if simulation throughput matters, that is a reason to install the JAX path and measure it on your hardware. The Hydra to_absolute_path edit in model/fno/utils.py and model/unet/utils.py is the first thing to apply, and the .env filename rule matters if you intend to use uploader.py. For upgrades, the honest position is that there is no migration guide in the supplied material: the only signal is the dated compatibility note, and it does not cover every component.

Editorial conclusion

Adopt PDEBench if you are evaluating a neural operator or physics-informed model and need published datasets with baseline training code in one place. Do not adopt it as a general simulation framework: the generators cover a fixed list of equations, and the repository is explicit that the pinned stack is Python 3.9, JAX 0.4.11, PyTorch 1.13.0 and CUDA 11.7. Before committing, check the licence file yourself, and run the forward training path once with the Hydra to_absolute_path lines enabled in model/fno/utils.py and model/unet/utils.py, because that is the failure the maintainers flag first.

Official sources

1. Issues
2. pdebench/PDEBench on GitHub
3. Project website
4. README
5. Releases