The vLLM production stack's router ships a code formatter as a runtime dependency

The package is named for the router, not for the stack

The project is a stack: serving engines, a request router, and an observability layer. The Python distribution at the root is not the stack, it is the router, and it says so in its name and its one line description. Its version is not written in the file at all, being derived from the repository's tags and written into a module inside the source tree at build time, which is the modern arrangement but means the manifest cannot answer what version you have installed. The console command it installs is named for the router too. So a reader arriving at a deployment problem has to work out that the thing named in the manifest is one of the three components on the page, and a reader pinning a version has to look at the tags rather than at the package.

The dependency list is duplicated and the comment admits it

Above the dependency list there is a comment saying the list should be the same as the requirements file inside the router's source directory. That is a manual synchronisation obligation written into the manifest itself, which is unusual candour and an unusual risk: the two files drift the moment somebody adds a dependency in one place. The router package is not the only project in this repository, so a reader who edits the wrong requirements file will find that the container still installs the old set, and the failure will look like a version conflict rather than a stale file. The rest of the toolchain is tidier than that: a version derived from tags, a single console entry point, and a test configuration that puts asynchronous tests in automatic mode so test files do not each have to declare it.

A code formatter is a runtime dependency

One entry in the router's install requirements is a code formatter, and it is not in a development group, a lint group or an extra. It is in the list every installation of the router pulls. That is a packaging mistake with a real cost rather than a cosmetic one: a formatter and its own dependency tree ship inside production containers, in a service whose job is to sit in front of model servers, and a reader auditing that image has to decide why. It also explains one of the other entries, since a formatter and an import sorter are usually the same team's tooling. The manifest has separate groups for linting and for tests, so the omission is not that the group does not exist.

Two error reporting stacks and a tracing stack arrive by default

The core dependency list includes a hosted error reporting client with its framework integration, and it also includes the three pieces of an open telemetry stack: the application programming interface, the software development kit and the protocol exporter. Both error paths are therefore live in a default install, with no switch to turn either off. For a service that sits in front of every request to a model, that is a deliberate choice rather than an accident, and it is worth reading as one: whoever deploys this is sending request-level telemetry to somebody's project by default. The metrics the page advertises are a third path, the Prometheus client and a dashboard, and that is the one an operator is most likely to want, since the router exports request rates, latency to first token, queue depths and uptime per instance.

The roadmap says the router should be rewritten in another language

Three items are listed as coming: autoscaling driven by the engine's own metrics, support for splitting the prefill work from the decode work, and router improvements, which the page then glosses as a more performant router written in something other than Python, a cache aware routing algorithm and better fault tolerance. That third item is the interesting one, because the router is the component that decides which backend serves a request, and the project's own plan for it is a rewrite. Meanwhile the router's documented algorithms are round robin, session based, and prefix aware, and the last of those is marked work in progress on the same page. So the component with a rewrite on the roadmap is also the component whose most interesting mode is not finished.

Disaggregated prefill has examples before the roadmap says it exists

The roadmap lists support for splitting prefill from the rest of the request as forthcoming. The examples directory in the same repository already contains two directories for it, one plain and one described as orchestrated. That is not necessarily a contradiction in the strict sense, since example code can precede a supported feature and the roadmap wording is about support rather than about code existing. It does mean a reader has to guess which one they are looking at, and a reader who finds the example directory will reasonably assume the feature works. The tutorials list tells you where each stage of the setup lives, from installing a cluster through loading model weights from a persistent volume to launching several models, and it does not mention this feature anywhere in its seven steps. The chart commands that bracket a minimal install are the whole deployment story in three lines:

git clone https://github.com/vllm-project/production-stack.git
cd production-stack/
helm repo add vllm https://vllm-project.github.io/production-stack
helm install vllm vllm/vllm-stack -f tutorials/assets/values-01-minimal-example.yaml

The blog, the docs and the interest form live on another domain

The page's link row points at a blog, an interest form, a documentation site, and two separate chat invitations, one for this project and one for a caching project. Three of those four destinations are not on this project's own domain. The blog and the interest form sit on a domain belonging to the caching project, and the chat link for this project goes through a third party invitation service rather than a direct invite. The documentation is the exception, on the vendor's own site, and the readme also carries a configuration file for a second documentation host, so there are two places the documentation is built. None of it is a problem for a reader following a link, and all of it makes the project's own front door a little harder to find, which is an odd shape for something whose stated purpose is to be a reference others adopt.

Editorial conclusion

Use this stack if you are running the engine on more than one machine and need routing that reuses cached tokens plus metrics to see what is happening, because those two things are what it adds over running the engine directly. Three things to know before you build on it. The router is a Python service whose dependency list is almost entirely exact pins, so treat upgrades as deliberate and read the comments that explain the three unpinned ones. Two error reporting stacks and a formatter arrive with the install, which tells you what the maintainers prioritise and what a leaner deployment would drop. And treat the prefix aware routing and the disaggregated prefill examples as research rather than features, since the page marks one as unfinished and the other as forthcoming.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. vllm-project/production-stack on GitHub