astroquery: one Python package per astronomical database instead of one script per archive

A sub-package per service, one idiom across them

astroquery is an astropy affiliated package containing a collection of tools to access online astronomical data, and the organising principle is that each web service has its own sub-package. The documentation gives SIMBAD as the example:

>>> from astroquery.simbad import Simbad
>>> theta1c = Simbad.query_object('tet01 Ori C')
>>> theta1c.pprint()
   main_id          ra           dec      ... coo_wavelength     coo_bibcode       matched_id
                   deg           deg      ...
------------- ------------- ------------- ... -------------- ------------------- -------------
* tet01 Ori C 83.8186095697 -5.3897005033 ...              O 2020yCat.1350....0G * tet01 Ori C

The output is an astropy table, which is the detail that makes the package more than a set of HTTP wrappers. The columns are typed, the printing is the astropy table printer, and the row you get back behaves like the rest of the astropy data structures rather than like a dictionary.

There is a standard suite of tools common to most modules, with query_object and query_region named as the examples. That shared vocabulary is what makes the package worth the dependency: a SIMBAD identifier lookup and a catalogue cone search are written the same way, and a script that queries three archives does not need three request styles.

The import convention matters in practice. Importing astroquery on its own does not get you much, because you need to import each sub-module specifically, and the documentation links to a list of available services for that reason.

Installation and the optional dependency groups

Astroquery works with Python 3.10 or later, and as an astropy affiliate it requires astropy version 5.0 or later.

The install command is a pip install with a pre-release flag, because the project ships developer versions:

$ python -m pip install -U --pre astroquery

Pulling in everything adds the all identifier:

$ python -m pip install -U --pre astroquery[all]

And there is a documented route for the bleeding edge version straight from GitHub:

$ python -m pip install git+https://github.com/astropy/astroquery.git

The optional dependency groups tell you what the extra weight buys. The test group pulls pytest with doctestplus and the astropy plugin, plus pytest-timeout, pytest-dependency on non-Windows systems, pytest-rerunfailures, fsspec with http support, and moto with s3fs. The docs group brings in sphinx, and the comment in the project metadata names the issue that motivates the sphinx-automodapi entry, which is a level of detail you rarely see in a package file.

The all group is the interesting one: mocpy, astropy-healpix, boto3, botocore and regions. Those are the dependencies of the modules that talk to the more demanding archives, and the fact that they are optional is the reason the base install stays small.

Authentication, archiving and how a query actually behaves

The runtime dependencies say what the package does at the network level: requests for the HTTP, beautifulsoup4 with html5lib for the services that return HTML rather than data, pyvo for the Virtual Observatory protocols that let an archive speak to another, keyring for credentials, and numpy and astropy for the data structures that come back.

The keyring dependency is the part worth understanding. Many of the archives need a login, and a password typed into a prompt in a notebook is both unusable in a batch job and bad practice. Putting the credential in the system keyring means the code can be committed and the secret stays out of it.

The dependency on pyvo is what makes the package more than a browser in disguise. The Virtual Observatory protocols let you query one archive for data it holds on behalf of another, so a single query can span catalogues without the caller writing per-service federation code.

The maintenance model is visible in the metadata. The version is dynamic and written by setuptools_scm into astroquery/version.py, so the version is derived from the git tag rather than typed into a file, and the package has no hard-coded version string to forget to bump. The project also carries a CITATION file and a licenses directory, and each release gets its own Zenodo DOI so a specific version can be cited.

The repository layout supports the split by service: an astroquery/ package directory holding the sub-packages, a docs/ directory, a test directory, a conftest.py at the root, tox.ini for the test matrix, and a .mailmap and a CODEOWNERS-style ownership file.

Where astroquery is the wrong choice

The first limitation is that every sub-package is a client for something the archive controls. A service can change its response format, tighten its rate limit, or change its query syntax without a breaking change release on the Python side, because the library version has not moved. The package cannot make a third-party web service reliable, and a pipeline built on it inherits that dependency.

The second is that the shared API is a lowest common denominator. The documentation says the API shows the standard suite of tools common to most modules, and the word most is doing work. A service with a capability that does not fit the common shape will expose it in a service-specific way, and the code that uses it stops being portable between archives. That is a reasonable design, and it means the abstraction helps most when your queries are ordinary.

The third is what the package is for. It returns tables of catalogue rows and query results, and it is a client, not a database. If you need the data to be reproducible years later, you need to store it, and the project is not in the business of promising that an archive will still answer the same question in 2036. The per-version Zenodo DOIs are about citing the code, not about archiving the data behind it.

The comparison with writing the requests yourself is honest. A single simple query against a single archive is a dozen lines of requests and a bit of parsing, and doing that means you own the retry behaviour, the rate limiting and the format changes. astroquery earns its cost when you are querying many services, when you want the astropy table types, and when you would rather not maintain a parser for every catalogue you touch.

Editorial conclusion

astroquery fits an astronomer or a data pipeline that needs to query several archives with one idiom rather than learning each service's own request format and response schema. It does not fit a project that needs guaranteed stability from a third-party web service, because every sub-package is a client for something the archive can change. Before depending on it, check that the specific service you need has a sub-package, read the API guidance for that module rather than assuming the common methods cover it, and cite it properly, since the project asks users to reference the paper in The Astronomical Journal.

Frequently asked questions

Official sources

1. astropy/astroquery on GitHub
2. License: BSD-3-Clause
3. Project website
4. README
5. Releases