Terrain Diffusion: Procedural Worlds from Coarse Maps

Terrain Diffusion replaces coarse noise with learned detail

Terrain Diffusion addresses a specific gap between fast procedural maps and detailed planetary data. A conventional noise generator can produce a broad terrain layout cheaply, but the repository's authors position Terrain Diffusion as a learned successor that refines an approximate layout into high-resolution heightmaps. The intended users are game developers, interactive-experience developers, and world builders who need coherent terrain at a useful spatial scale rather than an isolated small texture.

The project offers two documented model families. `xandergos/terrain-diffusion-30m` produces 30 m per pixel output and is recommended for playable worlds, with more local variation and detail. `xandergos/terrain-diffusion-90m` produces 90 m per pixel output and is intended for larger, more coherent world-building, although the README warns that its scale can be too expansive for some uses. The corresponding coarse-map spans documented in the README are 7.7 km and 23 km.

The key design choice is staged generation. First, a coarse map defines an initial world layout. It may be procedurally generated or hand-drawn, and it does not need to be precise. Second, the selected diffusion model converts that sketch into a high-resolution elevation map. Users control how strongly the sketch influences the result, which makes it possible to begin with a rough continental shape rather than authoring every ridge or valley. The output is data for another application, not a finished game scene.

InfiniteDiffusion adds random access without loading an entire world

The repository includes `InfiniteDiffusion`, described as a drop-in replacement for `MultiDiffusion` with lazy, unbounded sampling, constant-time random access, and seed consistency. It is built on the separate `infinite-tensor` framework. That combination matters for systems that cannot generate or retain one enormous finished image before serving any coordinate. Instead, the README describes an unbounded panorama that can be sampled at arbitrary locations while preserving deterministic results for the same seed.

The self-contained `annotated_infinite_panorama.py` demo makes the underlying idea easier to inspect. It generates an infinite-width panorama with Stable Diffusion v1.5 and `infinite-tensor`, without depending on the rest of the terrain-generation stack. The README says the script saves a 2048-pixel-wide crop as `output.png` beside the script. Its top-level constants include `PROMPT`, `CROP_PIXEL_WIDTH`, and `INTERMEDIATE_TIMESTEPS`, so this is a small experimentation surface rather than a complete planetary-generation interface.

For developers, the important distinction is between the core panorama algorithm and Terrain Diffusion's data products. InfiniteDiffusion addresses access and sampling behavior. The terrain models address planetary elevation and climate refinement. The README links `infinite-tensor` as the supporting library and `terrain-diffusion-mc` as a separate Fabric mod that replaces Minecraft's world generator, but that mod is not included in this repository's top-level file list.

Installation is Python-first, with a clear NVIDIA GPU preference

The documented repository installation clones the project and installs `requirements.txt`. CUDA users are advised to install a suitable NVIDIA driver and a CUDA-enabled PyTorch build. The README explicitly says that the project can run on CPU, but will be much slower, and that Mac is CPU-only. That caveat should shape early feasibility checks.

For Windows or Linux users with an NVIDIA GPU, the documented PyTorch installation is:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

The repository installation command is:

git clone https://github.com/xandergos/terrain-diffusion
cd terrain-diffusion
pip install -r requirements.txt

The dependency file includes PyTorch, Diffusers, Transformers-related ecosystem packages, raster and geospatial libraries, FastAPI, Uvicorn, `infinite-tensor`, and several training or evaluation dependencies. The listed requirements include `cartopy`, `rasterio`, `tifffile`, `earthengine-api`, `lpips`, `torchmetrics`, and `wandb`, so installation covers much more than the interactive demo. The README does not document a smaller production-only dependency profile.

First use starts with the procedural explorer or the API

The simplest documented first use is the procedural explorer. It opens a two-panel interface: the left panel displays the coarse map, and clicking a pixel requests a high-resolution shaded relief map in the right panel. The README also documents a temperature view, adjusted for elevation. This workflow is appropriate when you want to inspect model behavior before integrating generated data.

Run it with the 30 m model as follows:

python -m terrain_diffusion explore xandergos/terrain-diffusion-30m

The README says this uses a procedural coarse map, so it does not require the reader to prepare external conditioning files first. For programmatic experimentation, the general API can query elevation and climate data:

python -m terrain_diffusion api xandergos/terrain-diffusion-30m

The repository points to `API_README.md` for API details, but the README does not include that file's contents. Consequently, the README confirms the command and its broad purpose, but does not establish the endpoint names, request schema, response format, or deployment assumptions.

GeoTIFF conditioning controls the exported elevation map

For external map data, Terrain Diffusion accepts conditioning files in a directory. The documented inputs are `heightmap.tif`, `temperature.tif`, `temperature_std.tif`, `precipitation.tif`, and `precipitation_cv.tif`. They represent elevation, mean temperature, temperature variability, annual precipitation, and precipitation variability. The README states that missing files fall back to Perlin noise and that at least one file is required. That fallback makes partial workflows possible, but it also means omitted inputs become procedural estimates rather than measured or authored values.

The TIFF export path first refines the input maps and then increases resolution by 256 times on both axes. The documented command uses the 90 m model and a five-value `--snr` list:

python -m terrain_diffusion xandergos/terrain-diffusion-90m tiff-export azgaar-output/ output.tif --snr 0.2,0.2,1.0,0.2,1.0

The README notes that `0.2,0.2,1.0,0.2,1.0` worked well for an Azgaar export. That is a reported example, not a universal setting. The `--snr` values correspond to the five conditioning inputs and alter refinement strength, so the order of the input files matters.

Azgaar users can prepare a full JSON export first. The documented conversion command is:

python -m terrain_diffusion azgaar-to-tiff "MyMap.json" azgaar-output/ --scale 100

The README explains that larger `--scale` values produce more manageable outputs, while the coarse pixel sizes of 7.7 for the 30 m model and 23 for the 90 m model are described as the most realistic values.

Hardware and documentation boundaries limit some production uses

The clearest limitation is compute cost. The project is designed around diffusion models, and the README says CPU execution is much slower. Mac users are explicitly directed to a CPU-only path. A team targeting interactive runtime should therefore investigate the separate Minecraft mod and the API's runtime behavior rather than assuming that the repository's real-time streaming statement applies equally to every hardware configuration.

The documentation also leaves important integration details unspecified. The README names a streaming capability and a generalized API, but it does not define streaming latency, memory requirements, request concurrency, cache behavior, or the exact API schema. It does not document rollback behavior. The training path points to `TRAINING.md`, which the README describes as lengthy, while the repository contains a `configs/` directory and training-related packages. Those entries indicate that retraining is possible, but the README does not establish the cost of collecting data, preparing conditioning maps, or maintaining a trained checkpoint.

A second limitation is interpretive. Temperature and precipitation inputs have ordinary-looking physical units, including degrees Celsius and millimeters, but the README does not promise that generated climate variables have the statistical guarantees of a calibrated climate model. The `Temperature` view is explicitly adjusted for elevation. Users should not treat the names alone as evidence of scientific validity. The project is best evaluated first on whether its generated relief and climate patterns match the intended artistic or game requirements.

InfiniteDiffusion differs from Perlin noise and full-image diffusion

The README positions Terrain Diffusion against Perlin noise rather than presenting it as a drop-in texture library. The synthetic-map implementation is described as “a bunch of perlin noise” with transformations intended to match real-world data statistics and provide at least reasonable climate. That approach is cheap, procedural, and easy to modify in `terrain_diffusion\inference\synthetic_map.py`. Its weakness is that it does not use the learned refinement model to turn a coarse planetary sketch into the documented high-resolution terrain.

InfiniteDiffusion also differs from conventional bounded panorama generation. A fixed image pipeline has finite width and generally needs a defined output region. The documented InfiniteDiffusion design is lazy and unbounded, with constant-time random access and deterministic seed behavior. That is a different access model, not just a larger image.

The practical trade-off is operational. Perlin noise is easy to embed and modify, while Terrain Diffusion requires Python dependencies, model assets, conditioning data, and substantially more computation according to the README. Diffusion-based refinement is the right choice when coherent learned detail is the objective. A conventional procedural-noise path is simpler when speed, minimal dependencies, or direct control over the generator matters more.

Editorial conclusion

Game developers and world builders should first evaluate Terrain Diffusion when procedural elevation and climate data matter more than immediate CPU speed and the work involved in supplying conditioning maps. It is a poor fit for offline systems without an NVIDIA GPU, small experiments that need a conventional procedural-noise setup, or integrations that require exact climate semantics beyond the documented elevation, temperature, and precipitation inputs. Before adoption, verify the model scale, conditioning TIFF set, available hardware, and whether the generated elevation and climate outputs fit the target engine or GIS workflow. The repository uses an MIT license, and the last recorded push was on 2026-08-12, so its current code should be treated as an evolving research project rather than a settled production dependency.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. xandergos/terrain-diffusion on GitHub