# openFDA: the code behind api.fda.gov

> The openfda repository is the ETL and API server behind the api.fda.gov endpoints, built from Luigi pipelines that turn FDA public data into Elasticsearch indices and a Node.js Express service that serves them. It is CC0 licensed, it is a research project rather than a product, and running it locally is a matter of three containers.

**FDA/openfda** — openFDA is an FDA project to provide open APIs, raw data downloads, documentation and examples, and a developer community for an important collection of FDA public datasets.

- Repository: https://github.com/FDA/openfda
- Website: https://open.fda.gov
- Stars: 720 · Forks: 172
- Language: Python
- License: CC0-1.0
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/fda-openfda

## Three parts: pipelines, schemas, and a JSON API

openFDA is a research project to provide open APIs, raw data downloads, documentation and examples, and a developer community for an important collection of FDA public datasets. This repository holds the code that powers all of the api.fda.gov endpoints, and the README describes it in three parts.

The first part is Python pipelines written with Luigi for processing public FDA data sets, drugs, foods, medical devices and other, into a JSON format that can be loaded into Elasticsearch. The second is the Elasticsearch schemas for the available data sets. The third is a Node.js API server written with Express, Elasticsearch.js and Elastic.js that talks to Elasticsearch and provides the api.fda.gov JSON interface, which is documented in detail on the project site.

That is a clean separation and it is the shape of most data platforms. The transformation is separated from the schema and both are separated from the query layer, so you can replace the storage engine, extend a schema, or run a pipeline against a different source without rewriting the API.

The licence is CC0-1.0, which is the public domain dedication, and the primary language is Python. The default branch is master, the last push was on 2026-06-08, and there are no GitHub releases. The homepage is open.fda.gov.

The disclaimer at the top of the README is the first thing to read and it is unambiguous. Do not rely on openFDA to make decisions regarding medical care, always speak to your health provider about the risks and benefits of FDA-regulated products, and the project may limit or otherwise restrict your access to the API in line with its terms of service. That last clause matters for a dependency, since the access terms are not yours to set.

## Running the whole thing with three containers

If you intend to try running openFDA yourself, the repository ships a docker-compose configuration, and the README lists exactly what it brings up.

The Elasticsearch container comes first. The second is an API container exposing port 8000 for queries, and the third is a Python 3 container that runs the NSDE, CAERS, Substance Data, Device Clearance, Device PMA and Device Event pipelines and creates the corresponding indices in Elasticsearch.

The timing detail matters. The API container starts right away, but it will not serve any data until some or all of the pipelines have finished running, and you can check progress with curl http://localhost:8000/status to see which endpoints have become available as the pipelines progress or after they have completed. Once an endpoint is available it is queried with the standard openFDA query syntax, and the example given is a food event query filtering on the products industry name and a reaction, with a limit of ten.

The README is also candid about the scope of the bundled pipelines. Only the NSDE, CAERS, Substance Data, Device Clearance, Device PMA and Device Event pipelines run in the Python container, because most of those are relatively lightweight, except Device Event, and they require no access to internal FDA networks. More pipelines will be added in case there is substantial interest from the community. And the ones included are described as a good starting point for understanding openFDA internals or customising it.

So the local setup is a demonstration, not a mirror of production. That is the right scope for a project whose purpose is to show how the system works.

## Prerequisites, packaging and the Linux memory setting

The prerequisites are pinned versions rather than ranges. You need Elasticsearch 7, Python 3.10, and Node 16 or above.

Packaging is one command. Running bootstrap.sh downloads and sets up a virtualenv for the openfda python package and downloads and sets up the openfda-api node package, and the Dockerfile in the repository calls exactly that script after copying the source in.

The Docker image itself shows the shape of the environment. It is based on python:3.10.19, it installs Node 16 from the NodeSource setup script, and it adds netcat-openbsd and p7zip-full, which are the tools the pipelines need to reach the FDA servers and to unpack archives. The command at the end runs the scripts/all-pipelines-docker.sh script, so the container is the pipeline runner rather than the API.

There is one system tweak that will bite you on Linux. The vm.max_map_count needs to be increased before Elasticsearch can start successfully:

```shell
sudo sysctl -w vm.max_map_count=262144
echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf
```

The second line is the one that persists it, and the first only affects the running system.

Windows users get a different warning. You have to clone the repository with the core.autocrlf set to input, in the form git clone with the --config core.autocrlf=input option, in order to circumvent Docker issues with building images on a Windows computer. That is a line endings problem showing up inside a container build, which is a specific and memorable failure.

## The dependency pins, and what they tell you about the age of the code

The two dependency files are worth reading together, because they disagree, and the disagreement is informative.

The requirements.txt is a fully pinned list of more than seventy packages. arrow is 1.3.0, attrs 25.3.0, boto 2.49.0, elasticsearch 7.15.1, luigi 3.6.0, lxml 5.3.2, nose2 0.15.1, numpy 1.26.4, pandas 1.5.3, requests 2.27.1, urllib3 1.26.20, and so on. There are also packages that tell you what the pipelines do: chromedriver-binary-auto and selenium for scraping, pytesseract and pdf2image for reading documents, nltk for text, beautifulsoup4 for parsing HTML, and python-crfsuite for sequence labelling.

The setup.py, by contrast, is looser and partly stale. It declares version 1.0, requires Python 3.6 or newer while the README and the container say 3.10, and it caps elasticsearch at 7.15.1 and luigi at 2.8.13, whereas the requirements file pins luigi 3.6.0. Its install_requires also includes boto, the old library, alongside the modern boto3 and botocore in the requirements, and mock 2.0.0 alongside nose2.

The practical consequence is that the requirements file is the one to trust, because it is what the container installs, and the setup.py is a packaging artefact that has not kept up. A setup.py that still caps luigi at 2.x while the runtime uses 3.x will mislead anyone who installs the package with pip rather than with the bootstrap script.

Two other details in setup.py are unusual enough to be worth naming. The package lists a dependency link to a local tarball for python-crfsuite, with the file living in a dependencies directory in the repository, and the packages list includes openfda.annotation_table, openfda.faers and openfda.spl as subpackages, which is where the annotation table, the adverse event and the structured product label code live.

## Where openFDA is the wrong thing to build on

The first wrong case is anything clinical. The README says not to rely on openFDA to make decisions regarding medical care, and if your use case touches a treatment decision or a patient outcome, the disclaimer is not a formality. The data is public and the API is convenient, and neither fact makes it suitable for that.

The second is a dependency on the hosted service. The project states it may limit or otherwise restrict your access to the API under its terms of service, and the repository is a research project rather than a product with a support commitment. If your application needs openFDA to keep working, you are depending on a service whose terms can change and whose access can be restricted.

The third is a modern data platform. This is Elasticsearch 7 and Luigi 2 and 3, with the client library pinned below 8. Elasticsearch 8 changed the client and the security defaults, and this code does not target it. Building on the pipelines means building on a specific major version of a search engine that has moved on, and the compose file pins 7.14.1.

The fourth is a general ETL framework. Luigi is a good tool for batch dependency graphs and the pipelines here use it well, but if you want to load data continuously, or want to write the transformation in something other than Python, this is a specific stack rather than a foundation.

The alternative worth weighing is querying the hosted API directly and keeping the copy of the data you need in whatever system you already run, since the API is a documented JSON interface and the data it serves is also available as bulk downloads. That avoids the whole local deployment, at the cost of depending on the hosted service, which is a different dependency but a simpler one. Running the pipelines yourself is worth it when you need to customise the transformation, and the README says exactly that the included pipelines are a starting point for understanding the internals or customising openFDA.

## Conclusion

Adopt this repository if you need to run an openFDA-style API on your own data, or if you want to see how a large public dataset is turned into a queryable JSON index, since the pipelines, the schemas and the API server are all here under CC0. Keep away if you need a medical device decision, because the README says explicitly not to rely on openFDA for that. Before you run it, check the Elasticsearch version constraint, since the code pins the 7.x client and the compose file uses 7.14.1, and note the last push was on 2026-06-08 with no tagged releases.

## FAQ

### What is in the openFDA repository?

It holds the code behind all the api.fda.gov endpoints: Python pipelines written with Luigi that process public FDA datasets into JSON for Elasticsearch, the Elasticsearch schemas for those datasets, and a Node.js API server built with Express, Elasticsearch.js and Elastic.js that serves the JSON interface.

### How do I run openFDA locally?

Use the shipped docker-compose configuration, which starts an Elasticsearch container, an API container exposing port 8000 and a Python 3 container that runs the NSDE, CAERS, Substance Data, Device Clearance, Device PMA and Device Event pipelines. The API serves nothing until the pipelines finish, and you can check progress with curl http://localhost:8000/status.

### What are the prerequisites for openFDA?

Elasticsearch 7, Python 3.10 and Node 16 or above. On Linux, vm.max_map_count has to be raised to 262144 before Elasticsearch will start, and the README gives the sysctl command plus the line to add to /etc/sysctl.conf so it persists. On Windows you clone with core.autocrlf set to input to avoid Docker build issues.

### Can I rely on openFDA for medical decisions?

No. The README states plainly that you should not rely on openFDA to make decisions regarding medical care, and advises speaking to a health provider about the risks and benefits of FDA-regulated products. The project also reserves the right to limit or restrict your access to the API under its terms of service.

## Sources

- [FDA/openfda on GitHub](https://github.com/FDA/openfda)
- [Issues](https://github.com/FDA/openfda/issues)
- [License: CC0-1.0](https://github.com/FDA/openfda/blob/master/LICENSE)
- [Project website](https://open.fda.gov)
- [README](https://github.com/FDA/openfda/blob/master/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/fda-openfda