SGLang: one Python package for LLM, vision-language and diffusion serving, with the launch step left to its docs

Text, vision-language and diffusion models share one sglang package

SGLang describes itself as an inference framework for large language, vision-language and diffusion models, tuned for three workloads: agentic applications, large-scale serving and reinforcement learning rollouts. The diffusion side is not a plugin. SGLang Diffusion, the image and video generation engine, lives in this repository and ships inside the same `sglang` Python package as the language model server.

For an operator, that means one dependency to install and one engine to learn for chat models and image or video generators. It also means a change aimed at the diffusion engine and a change aimed at LLM serving arrive in the same release, so upgrading for one side moves the other side too. The repository layout reflects this breadth: alongside `python/` there are `rust/`, `sgl-router/`, `sgl-model-gateway/`, `proto/`, `benchmark/` and `docker/` at the top level.

Two install paths, and both point at moving targets

There are two ways in. The Docker image bundles SGLang with its dependencies:

docker pull lmsysorg/sglang:latest

The other path is a pip install inside an activated Python environment using uv:

uv pip install --prerelease=allow sglang

Both commands as written float. The image tag is `latest`, and the uv command explicitly allows prerelease packages during resolution. Releases arrive quickly: v0.5.21 on October 2, 2026, v0.5.22 on October 19, and v0.5.23 on November 2. Running the commands twice a few weeks apart can produce two different servers. If you need a reproducible deployment, record the image digest or the exact package version you got, rather than relying on `latest` or an unpinned install.

Installing does not tell you how to launch your model

After the install step, the repository front page stops. Launching a model is handed to two external sites: the Quickstart at docs.sglang.io, which walks through running a first model and sending a request, and the Cookbook at cookbook.sglang.io, where you pick a model and a hardware target and receive a ready-to-run launch command.

The consequence is practical. You cannot go from clone to a running endpoint using the repository alone, and the correct flags depend on the model and hardware combination you choose in the Cookbook. Teams that mirror dependencies internally or work in restricted networks should save the Cookbook command for their pairing along with the version it was written for, because the launch command is the part most tied to a specific release.

Wide hardware coverage, with TPUs and newer chips handled elsewhere

The Supported Hardware table covers NVIDIA (B200, B300, GB200, GB300, H100, H200, H800, H20, A100, RTX 30, 40 and 50 series, RTX 6000 Ada and PRO 6000, DGX Spark, Jetson Orin), AMD Instinct MI300X, MI325X, MI350X and MI355X, Intel Arc and Arc Pro B-Series GPUs plus Xeon CPUs, Apple Silicon Macs via Metal and MLX, Huawei Ascend A2, A3 and 950PR/DT NPUs, and Moore Threads MTT S5000.

Google TPU support is split off. SGLang-TorchTPU brings the engine to TPUs through PyTorch, torch_tpu and Pallas kernels, and supports LLM and vision-language serving on TPU7x (Ironwood) with tensor and expert parallelism across hosts. SGL-JAX is the JAX-based TPU path. Both are separate repositories, so a TPU deployment is not the same codebase as a GPU one.

AWS Trainium, Alibaba T-Head PPU, Cambricon MLU, Qualcomm QAIC, MetaX, Hygon HCU/DCU and Iluvatar CoreX are listed as integrations in progress. If your fleet runs on one of those, you are adopting unfinished support. The table lists chips and platforms rather than operating systems, and Windows does not appear in it.

Audio serving, draft-model training and cluster routing are separate projects

Several capabilities people associate with SGLang are not in the `sglang` package. SGLang Omni, which serves text-to-speech and automatic speech recognition models, is its own repository. SpecForge, which trains draft models for speculative decoding and then deploys them with SGLang, is another. For deploying and scaling inference services with routing, load balancing and cluster orchestration, the ecosystem table points to SMG, RBG, llm-d, Ray Serve and NVIDIA Dynamo.

For a reader planning a deployment, this means a speech workload needs a second install, a custom draft model needs a separate training tool, and a multi-node service needs a choice of orchestration layer on top of the engine. Budget for integrating those pieces rather than expecting one binary to cover them.

RL training frameworks call it for rollout generation

One of the three workloads SGLang names is RL rollouts, and the ecosystem table backs that up with five training frameworks that integrate it for rollout generation: Miles, slime, AReaL, Tunix and verl. In those setups SGLang is the component producing model outputs during post-training, while the framework handles the training loop. If you already use one of these frameworks, SGLang may arrive as a dependency before you choose it directly, and its release pace then affects your training stack too.

For learning how the engine is built, the project points to Mini-SGLang, the zero-to-sglang tutorial from datawhalechina, and a DeepLearning.AI short course on efficient text and image generation with SGLang.

Contributing starts from the dev container and an editable install

The development path begins with the `lmsysorg/sglang:dev` Docker image, which carries development tools and most dependencies. You clone or mount your checkout inside the container, then install it in editable mode from the repository root so tests pick up local Python changes:

pip install -e "python"

The Python package lives under `python/`, which is why the path argument is not the root. In an activated virtual environment, `uv pip install --prerelease=allow -e "python"` works instead. Before submitting, run `pre-commit run --all-files`, add regression coverage for fixes, and include benchmarks or accuracy evaluations in the pull request when relevant. Larger changes go to a GitHub issue or Slack first. The project is hosted by LMSYS, a non-profit open-source organization, and the most recent push landed on November 2, 2026.

Editorial conclusion

SGLang suits teams serving open-weight language, vision-language or diffusion models on NVIDIA, AMD, TPU, Intel, Ascend or Apple Silicon hardware, and RL teams whose training framework already uses it for rollouts. It is a poor fit if you need audio serving or cluster routing from one install, or if your chip sits on the in-progress integration list. Before adopting it, open the Cookbook entry for your exact model and hardware pair, and decide how you will pin versions given a release roughly every two weeks.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes