geospatial-data-catalogs: a daily index of open geospatial data in the cloud

Five catalog APIs, one searchable file

Some of the popular cloud-computing platforms host a lot of publicly available geospatial datasets. The README names AWS, Google Earth Engine, Microsoft Planetary Computer, NASA's Common Metadata Repository and STAC Index as examples.

The problem this repository solves is that finding what is available across all of them means visiting five different catalog interfaces, each with its own search syntax and its own idea of what a dataset record looks like. The project compiles the list of all geospatial datasets on these platforms as a CSV file and as a JSON file, making it easier to find and use them programmatically. The list is updated daily.

That daily update is the feature. A catalog of open datasets is only useful if it is current, because datasets get added, renamed, deprecated and have their access rules changed. Maintaining that by hand is not viable, and a stale list is worse than no list, so the value of the project is exactly as durable as its refresh.

The primary language is Python, but that is only because of the tooling. The deliverable is data. The licence is MIT, which is the right choice for a file that other people are expected to download, parse and build on.

The audience is developers doing geospatial work who are deciding where to get their data, and analysts who want to search the catalogue of what is available before they start writing code against one particular platform.

Twelve files, split by platform and by catalog type

The repository holds the same information twice, once as tab separated values and once as JSON, and splits it across a set of files named after the platform or catalog they describe.

For AWS there are three. aws_open_datasets is the list of open datasets on AWS, aws_geo_datasets is the list of open geospatial datasets on AWS, and aws_stac_catalogs is the AWS open geospatial data that comes with a STAC endpoint.

Then there is stac_catalogs for the catalogs listed on stacindex.org, gee_catalog for the Earth Engine catalog, pc_catalog for the Planetary Computer catalog, and nasa_cmr_catalog for the NASA CMR STAC catalog.

The Copernicus services get one file each, split by service domain: copernicus_atmosphere, copernicus_climate, copernicus_emergency, copernicus_land and copernicus_marine.

The naming is worth internalising because it is the whole navigation scheme. There is no index file, no tag, no schema file; you pick the platform, you pick the format, and you read the file. The result is a repository where a user who knows what they want can get it in one request, and a user who does not know what they want has to try each file in turn.

The JSON links in the README point at a couple of sibling repositories rather than at this one, the NASA CMR STAC catalog being hosted in the NASA-CMR-STAC repository, which is a small reminder that this repository aggregates rather than owns all of these listings.

STAC endpoints are the entries that can actually be queried

It is worth separating what these files describe, because the distinction determines what you can do with a given entry.

A STAC catalog, SpatioTemporal Asset Catalog, is a standardised description of a geospatial collection that is itself queryable over HTTP. The README links the CMR-STAC documentation when introducing NASA's Common Metadata Repository, and the catalog list includes STAC endpoints gathered from stacindex.org.

So three of the twelve files are specifically about endpoints you can talk to: aws_stac_catalogs for AWS datasets that expose a STAC endpoint, stac_catalogs for the catalogs indexed at stacindex.org, and nasa_cmr_catalog for the NASA CMR STAC catalog.

The other files are lists. An entry in gee_catalog or pc_catalog tells you that a dataset exists on that platform and how to reach it, which is genuinely useful for discovery, but it is not a query interface. If your pipeline needs to search by date and cloud cover, only a STAC endpoint gives you that, and this repository tells you which ones are worth wiring up.

That makes the catalogue a routing table rather than a data source. Which is exactly what the project says it is for, making it easier to find and use datasets programmatically.

The project description lists Amazon, earth-engine, nasa-data, cloud-computing, satellite-imagery and open-access as topics, and the sibling repositories it links under Relevant Projects extend the same idea to the individual platforms, including a Maxar Open Data STAC catalog and a separate Copernicus Services Products Metadata repository.

Reading a catalog with pandas takes four lines

The README's example is deliberately minimal, and it shows the intended usage pattern: the file is a plain table, so any tool that reads delimited text can consume it.

import pandas as pd

url = 'https://github.com/opengeos/geospatial-data-catalogs/raw/master/aws_geo_datasets.tsv'
df = pd.read_csv(url, sep='\t')
df.head()

The URL points at the raw file on the master branch, the separator is a tab, and the result is a dataframe you can filter. There is no client library, no API key and no authentication, which is what makes the catalogue usable from a notebook, a script or a shell one-liner without any setup.

The raw URL is worth understanding as a design decision. Reading from the master branch means you get whatever the daily update produced at the moment you ran the script, which is convenient and also means your results are not reproducible unless you pin a commit hash. For an exploration catalogue that is the right trade. For anything that ends up in a production pipeline or a published analysis, pin the commit and record it.

The repository's only Python dependency, listed in requirements.txt, is leafmap, which is what the example notebook uses for the interactive mapping side of the workflow.

A notebook entry point and a pre-commit hook

Two files explain how the catalogue is produced and consumed.

The refresh side is auto_update.py at the root of the repository, the script that queries the platform catalogs and rewrites the TSV and JSON files. The README states that the list is updated daily, and the presence of an update script alongside a .pre-commit-config.yaml suggests the data is regenerated and committed rather than served dynamically, which is what makes the raw URLs stable enough to link to.

The consumption side is examples.ipynb, the example notebook. The README badges point at it in two ways: a Colab badge, which opens the notebook in Google Colab, and a Binder badge, which launches it in a Binder environment. Between them they cover the two common paths for someone who does not want to set anything up locally.

That is a deliberate choice about the audience. A catalogue of open datasets is most valuable to people who are exploring rather than committing, and a notebook that runs in the browser with no installation is the right first contact for that. Once you know which dataset you want, you switch to reading the file directly, as the pandas example does.

The repository layout follows the same logic. There is no package to install, no module to import and no test suite, because the artefact is the file. The root contains the twelve dataset pairs, the update script, the notebook, requirements.txt with leafmap, a pre-commit configuration, a .github directory, a licence and the README.

A catalogue of catalogues, and what the maintenance looks like

The Relevant Projects section is unusual and useful, because it shows the design of the whole effort. Alongside this repository sit aws-open-data, aws-open-data-geo and aws-open-data-stac, which are the AWS lists before they were merged here, stac-index-catalogs for the stacindex.org listing, Planetary-Computer-Catalog and Earth-Engine-Catalog for the two commercial platform catalogues, NASA-CMR-STAC for the NASA side, and maxar-open-data for the Maxar STAC catalog. The Copernicus metadata comes from a separate repository, the do-me one.

The consequence is that this repository is a union of independently maintained sources. Some of the files are generated here, some are mirrored from elsewhere, and the refresh rate of each is not necessarily the same even though the README describes the list as updated daily. If you care about the freshness of one particular platform, that is worth checking rather than assuming.

On maintenance, the facts are simple. The last push to the repository was on 2026-09-16, the project has no GitHub releases, and the MIT licence leaves the files free to use, redistribute and build on with no conditions beyond the notice.

There is one more limitation that is easy to gloss over. A catalogue tells you a dataset exists, and the README does not document what each column contains, whether the licence of each individual dataset is recorded, or how to tell whether a listed dataset is still reachable. For planning a data pipeline, the catalogue is a starting point that shortens the search, not a substitute for checking the terms of the data you actually intend to use.

Editorial conclusion

geospatial-data-catalogs suits a GIS or remote sensing developer who needs to know what open data exists before choosing a source, and it suits anyone who wants a single searchable list instead of five separate catalog pages. It does not suit you as a runtime dependency in a production pipeline, because the repository publishes data files rather than a service, and it does not replace the platform APIs, since a catalog entry tells you a dataset exists but not how to query it. Check three things before you rely on it: which of the twelve files covers the platform you care about, since the list is split by platform and by catalog type, whether the entry you need carries a STAC endpoint, and that you read the file at the moment you need it, because the list is updated daily and therefore changes under you. The last push to the repository was on 2026-09-16 and there are no GitHub releases.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. opengeos/geospatial-data-catalogs on GitHub
4. README