sglang vs vllm: shared-prefix speed against serving breadth

Two engines with different bets

Both projects are Apache-2.0 licensed Python serving frameworks for LLMs and multimodal models, and both are under active development: each repository's last push was September 11, 2026. They are direct competitors, so the comparison is about where each engine puts its effort. SGLang's README and news stream emphasize shared-prefix caching and fast adoption of new models, with multiple posts about day-0 support for models such as DeepSeek V3 and R1, Kimi K3 and the Nemotron line. vLLM's README emphasizes breadth of another kind: efficient attention key and value memory management through PagedAttention, continuous batching, chunked prefill, prefix caching, speculative decoding options, and a long list of quantization formats and hardware targets. The two projects pursue the same job, serving open models at throughput, but they bet on different ways to win at it.

How each engine gets its throughput

SGLang's signature mechanism is shared-prefix caching across requests. Our analysis of the repository describes the main performance claims as depending on shared prefixes, with a radix cache that reuses computation and key-value state across requests; the practical effect is that workloads with long common prefixes, such as identical system prompts or large retrieval contexts, can see large gains, while requests with no overlapping prefixes get less benefit. vLLM attacks the same problem from the memory side: PagedAttention, described in the project's paper, manages attention key and value memory in pages to reduce wasted memory, and the README lists continuous batching, chunked prefill and prefix caching, plus CUDA and HIP graphs, torch.compile, and kernels such as FlashAttention, FlashInfer and FlashMLA. Both engines also support speculative decoding; vLLM's README lists n-gram, suffix, EAGLE and DFlash variants, and SGLang's news stream covers its own next-generation speculative decoding work under the names DFlash and Spec V2.

Getting each one running

Installation differs only in flavour. vLLM's README recommends install via uv with uv pip install vllm, or pip, and points to its documentation for building from source and for checking the list of supported models before you start. SGLang installs from PyPI too, and its README tells you to verify that your specific model architecture is supported before committing. The practical difference is in how you validate compatibility. vLLM documents that it supports 200-plus model architectures on Hugging Face, spanning decoder-only models, mixture-of-experts, hybrid attention and state-space models, multimodal models, embedding and retrieval models, and reward and classification models. SGLang's README does not list an equivalent count; instead its compatibility story is organized around day-0 support announcements for specific popular models. If your model is a long tail, vLLM's explicit supported models list is easier to check against at a glance. If your model is a headline release, SGLang may already have announced support for it.

Hardware and parallelism

vLLM documents the widest hardware net: NVIDIA GPUs, AMD GPUs, Intel GPUs, x86, ARM and PowerPC CPUs, plus hardware plugins for Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, MetaX GPU and more. For distributed serving it lists tensor, pipeline, data, expert and context parallelism. SGLang's documented focus is narrower: NVIDIA and AMD GPUs are the primary targets, and TPU support arrived as a native backend, described in the README news stream as SGLang running on TPU with the SGLang-Jax backend, with Google collaboration bringing full SGLang features to TPUs. For a team with Intel GPUs, CPU-only serving or an unusual accelerator, vLLM's support table is the safer starting point. For a team on NVIDIA or AMD GPUs that may later touch TPUs, SGLang's stack does not close that door either, but the TPU path is a separate backend rather than a mainstream configuration.

What each engine serves beyond plain chat

vLLM's README lists structured output generation with xgrammar or guidance, tool calling and reasoning parsers, streaming outputs, efficient multi-LoRA support for dense and MoE layers, and disaggregated prefill, decode and encode. It also covers non-chat model families: embedding and retrieval models such as E5-Mistral, GTE and ColBERT, and reward and classification models. SGLang's README news stream shows a different expansion path: SGLang Diffusion accelerates video and image generation, and the project reports agentic workloads, for example a blog post on serving GLM5.2 NVFP4 agentic workloads at high throughput. Multimodal models are in scope for both. If your serving layer must speak OpenAI-compatible, Anthropic Messages and gRPC endpoints, vLLM documents all three; SGLang's README excerpt does not enumerate its API endpoints, so check the documentation if you need a specific protocol.

Where each falls short

SGLang's weaknesses follow from its speed focus. Our analysis of the repository notes it is a fast-moving codebase with frequent releases, which is fine for teams that track it closely and painful for anyone wanting a stable long-term stack. The same analysis flags that the main performance claims depend on shared prefixes, so you should test the cache hit rate on your actual prompt patterns before trusting the headline numbers, and low shared-prefix workloads may not see the advertised advantage. vLLM's weaknesses follow from its breadth. The analysis of vLLM describes real operational overhead: configuration and monitoring need investment, and a simple single-model server with minimal setup is not where the engine shines. Compatibility is gated, because your exact model architecture must appear in the supported models list, and the analysis advises checking the latest release notes for breaking changes in the OpenAI-compatible API or parallelism options before upgrading.

Licence, maintenance and the choice

Both projects are Apache-2.0 and both were pushed on September 11, 2026, so licensing and maintenance health do not separate them. Release cadence is high on both sides: vLLM's recent releases include v0.28.0, v0.27.1 and v0.27.0 across a few weeks in August 2026, and SGLang released v0.5.18, v0.5.17 and v0.5.16 on a roughly two-week rhythm over the same period. Plan for frequent upgrades either way. The choice comes down to workload shape. Shared long prefixes and day-0 model access point to SGLang. Broad model coverage, hardware variety and a documented multi-protocol API point to vLLM, and its longer track record around a larger ecosystem makes it the default for teams that do not have a specific reason to pick SGLang.

Bottom line

Choose vLLM when you need one serving engine for many model architectures and hardware backends and you can invest in learning its configuration, and choose SGLang when your traffic shares long prefixes or a brand-new model release must be served immediately. Before committing either way, verify that your exact model architecture is on the supported list, and for SGLang measure the shared-prefix cache hit rate on your own prompts, since its headline performance claims depend on that pattern.

Sources

1. sgl-project/sglang repository
2. sgl-project/sglang README
3. vllm-project/vllm repository
4. vllm-project/vllm README