Docling Serve: a FastAPI wrapper around Docling, and the 11 GB problem its images create

Two install routes and three local URLs

Docling Serve exists to run Docling as an API service. There is no build step for the evaluation path, just an install and a run:

# Using the python package
pip install docling-serve
docling-serve run --enable-ui

# Using container images, e.g. with Podman
podman run -p 5001:5001 -e DOCLING_SERVE_ENABLE_UI=1 quay.io/docling-project/docling-serve

The UI is off by default and the flag or the environment variable turns it on, which is the right default for a service. Once running, three addresses matter: the API at http://127.0.0.1:5001, the generated API documentation at /docs, and the UI playground at /ui.

The stack underneath is a standard Python service stack. pyproject.toml pins fastapi[standard], uvicorn[standard], typer for the command line, pydantic and pydantic-settings for models and settings, python-multipart for uploads, websockets, and scalar-fastapi alongside the FastAPI documentation. Python 3.10 is the floor, with classifiers for 3.10 through 3.13, and the project declares itself Production/Stable and Typed.

The package itself depends on docling-slim, docling-core and docling-jobkit, the last of which carries the extras for rq, ray, vlm, azure, gcloudstorage and gdrive. That extras list is the job engine story: the same service can drive RQ on Redis, Ray, or remote storage backends.

A conversion request names its source, and the v1 path is stable

The API takes a JSON body listing sources. This is the request the README gives:

curl -X 'POST' \
  'http://localhost:5001/v1/convert/source' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "sources": [{"kind": "http", "url": "https://arxiv.org/pdf/2501.17887"}]
  }'

Two structural details are worth noting. The kind field makes the source type explicit rather than inferred, so http here means the server fetches the document rather than receiving an upload. And the path is prefixed with v1, which reflects a migration the README calls out as a note: Docling Serve now has a stable v1 API, with a dedicated migration document at docs/v1_migration.md.

That versioning is the difference between a script and a dependency. If you are integrating this into a product, read the migration guide before assuming an older endpoint shape still applies.

Image sizes are the real deployment decision

The container table is the part to read carefully, because the sizes are large and the architecture coverage is uneven.

The base image, published as ghcr.io/docling-project/docling-serve and quay.io/docling-project/docling-serve, installs everything from the official PyPI index and covers linux/amd64 and linux/arm64 at 8.7 GB on amd64 and 4.4 GB on arm64. The CPU variant, docling-serve-cpu, pulls torch from the PyTorch CPU index and is 4.4 GB on both architectures.

The CUDA images are where the weight lands. docling-serve-cu128 is a CUDA 12.8 build at 11.4 GB and covers linux/amd64 only. docling-serve-cu130 is a CUDA 13.0 build covering amd64 and arm64, with its size listed as TBD in the table.

A ROCm image for AMD ROCm 6.3 is supported but not published, because of its size. It can be built locally:

git clone --branch main [email protected]:docling-project/docling-serve.git
cd docling-serve/
make docling-serve-rocm-image

The README also notes that docling-serve-slim images are coming, which will reduce the size by skipping the model weights download. Until that ships, the model weights are why the images are measured in gigabytes.

CUDA images deliberately have no latest tag

There is a tagging policy for the CUDA images and it will bite you if you copy a docker pull line from elsewhere. Base images, docling-serve and docling-serve-cpu, are tagged with latest and main for convenience. The CUDA images, docling-serve-cu128 and docling-serve-cu130, are tagged only with explicit versions and main, and they intentionally exclude latest.

The reason given is PyTorch's CUDA lifecycle: CUDA versions are deprecated over time as PyTorch adds support for newer releases, so the CUDA images exclude latest to avoid pulling a deprecated version by accident. The recommended pattern is an explicit version:

# Recommended: Explicit version
docker pull quay.io/docling-project/docling-serve-cu130:v1.18.0

while the latest form is not available for those images. Practically this means your deployment manifests cannot use a floating tag for GPU deployments, and a pinned GPU tag is something you have to update deliberately.

Configuration comes from a file or the environment, with environment winning

The example environment file shows the shape of a deployment. The values are not just logging switches; they encode a sizing decision.

TESSDATA_PREFIX=/usr/share/tesseract/tessdata/
UVICORN_WORKERS=2
UVICORN_RELOAD=True

# Logging configuration (case-insensitive)
# DOCLING_SERVE_LOG_LEVEL=WARNING  # Options: WARNING, INFO, DEBUG (or warning, info, debug)

# RQ Engine Redis Connection Pool Configuration
# DOCLING_SERVE_ENG_RQ_REDIS_MAX_CONNECTIONS=50
# DOCLING_SERVE_ENG_RQ_REDIS_SOCKET_TIMEOUT=5.0
# DOCLING_SERVE_ENG_RQ_REDIS_SOCKET_CONNECT_TIMEOUT=5.0
# DOCLING_SERVE_ENG_RQ_QUEUE_NAME=convert

The Redis pool guidance is scale dependent and stated in the comments: small deployments of 1 to 4 workers use the defaults of 50 connections with no timeouts, medium deployments of 5 to 10 workers should set max_connections to 100, and large deployments of 10 or more workers should set 150 to 200. Timeout settings are only set if you hit connection issues.

Configuration can also come from a YAML or JSON file, and the precedence is documented:

# Environment variables take precedence over config file values
# DOCLING_SERVE_CONFIG_FILE=examples/config.yaml

The repository ships both examples/config.yaml and examples/config.json, so you can copy a starting point rather than writing one from nothing.

VLM presets are an allowlist, which is the security control that matters here

Document conversion in this service can be backed by a vision language model rather than by deterministic layout analysis, and the configuration treats the choice of model as something to restrict rather than to trust.

The default preset applies when a user asks for the preset named default:

DOCLING_SERVE_DEFAULT_VLM_PRESET=granite_docling

A list of permitted preset identifiers can be set, either as a JSON array or comma separated, and custom presets can be declared as a JSON object mapping an identifier to an engine and model, for example {"my_preset": {"engine": "openai", "model": "gpt-4-vision"}}. The environment variable names follow the same DOCLING_SERVE_ prefix pattern as everything else.

The reason to notice this is that the presets are user selectable in the API. An allowlist of preset IDs, with defaults separated from custom definitions, means a caller cannot pass an arbitrary model identifier through to a paid API. If you are deploying this for multiple users, that configuration is where the boundary is.

Observability examples are shipped, and so is a separate process example

The examples directory is more than sample payloads. It contains OpenTelemetry and Prometheus material: examples/OTEL.md, examples/docker-compose-otel.yaml, examples/otel-collector-config.yaml, examples/prometheus.yaml, examples/prometheus-scrape.yaml, examples/tempo.yaml and examples/grafana-datasources.yaml.

That is a complete tracing and metrics path rather than a stub. A compose file for OpenTelemetry, a collector configuration, Tempo and Grafana datasource definitions together describe a deployment where a conversion request can be followed end to end.

examples/split_processing.py is the one to read for architecture. A document conversion service has an obvious failure mode where the HTTP request thread is blocked by a long conversion, so separating request handling from processing is the standard answer, and the file suggests the job engines in the dependency list support it.

For reference, the README asks users citing Docling to cite the arXiv technical report, 2501.17887.

Maintenance state and licensing

Release cadence is fast and the version numbers track it: v1.35.0 on 2026-09-23, v1.36.0 on 2026-10-01 and v1.37.0 on 2026-10-13, with the last push on 2026-10-16. That is roughly one minor release a week, which is normal for a service layer tracking a fast moving core library, and it means you should pin rather than track the default branch.

The version in pyproject.toml is 1.37.0 and carries an inline comment marking it as automatically updated, so do not edit it by hand. The Makefile tags local builds from the current git revision and the current branch, and defaults to docker as the container runtime, overridable with CONTAINER_RUNTIME, which is how the README's Podman example is consistent with the build system.

Licensing is MIT for the codebase. The project is maintained under IBM and the author list in pyproject.toml carries both IBM and Red Hat affiliations, with maintainers listed separately.

One dependency caveat is worth reading before you commit: docling-slim is required at a version below 3.0.0 and docling-jobkit below 4, so both are on major version ranges that will eventually move.

Editorial conclusion

Adopt Docling Serve if you need document conversion as a service with an HTTP API, a generated API reference and a playground UI, and if your platform is one of the published image targets. Do not adopt the prebuilt CUDA images on a rolling tag policy, because the README states CUDA images are tagged only with explicit versions and main and intentionally exclude latest. Verify first which image matches your hardware, since CUDA 12.8 covers linux/amd64 only while the CPU image covers both amd64 and arm64, and confirm your Python version is 3.10 or newer against the declared classifiers.

Frequently asked questions

Official sources

1. docling-project/docling-serve on GitHub
2. Issues
3. License: MIT
4. README
5. Releases