# LeaderWorkerSet makes a group of pods one unit of replication

> The LeaderWorkerSet API in kubernetes-sigs/lws deploys a group of pods as a single replica, with a leader pod and a set of workers that share a lifecycle, an index, and a gang scheduling guarantee, which is what multi host LLM inference needs when a model is sharded across nodes. A second API, DisaggregatedSet, layers on top for disaggregated serving, and the project reached v1.0.0 in October 2026.

**kubernetes-sigs/lws** — LeaderWorkerSet: An API for deploying a group of pods as a unit of replication

- Repository: https://github.com/kubernetes-sigs/lws
- Website: https://lws.sigs.k8s.io
- Stars: 893 · Forks: 199
- Language: Go
- License: Apache-2.0
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/kubernetes-sigs-lws

## A group of pods that has to behave like one pod

LeaderWorkerSet is an API for deploying a group of pods as a unit of replication, and the README is direct about the problem it addresses: common deployment patterns of AI and ML inference workloads, especially multi-host inference workloads where the LLM will be sharded and run across multiple devices on multiple nodes.

That is the key. When a model is sharded, its parts have a relationship: a rank zero process and the other ranks have to find each other, they start together or not at all, and they scale as a set. Treating each pod as an independent Deployment replica is the wrong abstraction, because the scheduler will happily place a group across nodes that cannot talk to each other, and the autoscaler will scale the parts independently.

The API therefore models something between a pod and a workload. The README calls the group a super pod, and the features list calls it a tightly managed group of pods. It supports a unique pod identity, where each pod in the group has a unique index from 0 to n-1. Pods in the group have the same lifecycle and are created in parallel. And multiple groups with identical specifications can exist, each of which is a single unit for rolling update and scaling, and each of which maps to a single exclusive topology for placement.

The audience is therefore a specific kind of platform engineer. Somebody running a distributed inference server on Kubernetes, where the serving topology is part of the correctness of the system rather than an implementation detail.

The project is under the kubernetes-sigs organisation, the licence is Apache-2.0, the primary language is Go, the homepage is lws.sigs.k8s.io, and the last push was on 2026-10-20, the same day as the v1.0.0 tag on 2026-10-16.

## One leader, a set of workers, and a template for each

The group has an internal structure, and the README describes it in a way that is closer to a replica set than to a Deployment.

A replica is a group of a single leader and a set of workers. The API allows you to specify a template for the workers and optionally to use a second one for the leader pod, and the features list calls this the dual template design. So the leader is not just the first pod by convention, it is a pod with its own specification, which is what you need when the leader does something different: it serves traffic, it holds a lease, it does the routing, and it may need a different resource profile from the workers that do the compute.

The lifecycle is shared. Pods in the group have the same lifecycle and are created in parallel, and the rollout semantics follow from that: rollout and rolling update happen at the group level, which means the groups are upgraded one by one as a unit, with the pods inside a group updated together. A rolling update that replaced pods inside a group independently would break the sharded model, and the API does not offer that.

Scaling follows the same logic. A scale subresource is exposed so that the HPA can dynamically scale the number of replicas, and a replica is a group, so the HPA scales groups rather than individual pods.

Two features are opt-in and are the ones to look at first for a real deployment. Topology-aware placement lets pods in the same group be co-located in the same topology, which for an inference workload is often the difference between a working and an unusable deployment, depending on the interconnect. And all-or-nothing restart is opt-in support for all pods in the group to be recreated if one pod in the group failed or one container in the pods is restarted, which keeps a partially broken group from serving.

The gang scheduling feature is the one to be careful with. Each replica with a group of pods can be scheduled in an all-or-nothing manner, and the README marks it as Alpha level with the note that the API may change in the future.

## DisaggregatedSet adds roles, ratios and coordinated rollouts

The second API in the repository, DisaggregatedSet, is for a harder version of the same problem, and the README is clear that it builds on LWS rather than replacing it.

LWS forms the core API for multi-node, while DisaggregatedSet builds on it to add advanced disaggregated workload deployment with support for autoscaling, rollouts and failure handling. The key features list is where the difference shows. Disaggregated architecture support is specifically designed for workloads where different phases, such as prefill and decode, run on separate infrastructure.

The rollout model is the most interesting part. Coordinated N-dimensional rollouts update multiple roles, between 2 and 10, in lockstep, preserving capacity ratios throughout the update process. That is a real constraint that a plain Deployment cannot express: if you have a prefill role and a decode role that must keep a 1 to 3 ratio, then updating one role at a time changes the ratio and the serving behaviour, so the two have to move together.

Unified lifecycle management manages multiple underlying LeaderWorkerSets as a single logical unit, and automatic service orchestration automatically creates and manages headless services for each role to facilitate discovery and revision-aware routing. The revision-aware part matters, because a headless service that returns endpoints from two different rollout revisions will send traffic to a mix of old and new.

Failure handling is also coordinated, with drain and restart policies across all the roles in the disaggregated set rather than per role.

The design context is the collaboration with llm-d, which the README describes as a CNCF sandbox project, a high-performance distributed inference serving stack optimised for production deployments. The two APIs are being co-designed with llm-d so that they are optimised for real-world serving frameworks and disaggregated architectures. The Kubernetes enhancement proposal for the DisaggregatedSet API lives in the keps/ directory of the repository, under keps/766-DisaggregatedSet.

## Support is a window of four Kubernetes minor versions

The compatibility section is precise, and for a SIG project that precision is the contract.

LeaderWorkerSet maintains support for the latest Kubernetes minor versions, currently tested on Kubernetes 1.34 through 1.37, and the installation section says it is officially supported and tested on Kubernetes 1.34 or newer, pointing at the installation guide for the details.

The test coverage is published on TestGrid, and the README lists three kinds. Unit tests and integration tests each have their own TestGrid presubmit job. End to end tests run on KIND, with one job per Kubernetes version, for 1.34, 1.35, 1.36 and 1.37.

That is a large matrix to run on every change, and it is also the reason to trust the compatibility claim. The e2e jobs are not theoretical, they are the mechanism by which the project verifies that a group of pods with a leader, an index, topology placement and a group level rollout actually behaves as described on each of those versions. There is a coverage badge for the unit tests as well.

The practical consequence for an adopter is that the support statement is tied to a window, not to a version. A cluster on 1.33 is outside the window, and a cluster on 1.38 may be, depending on when you read this.

For the rest of the documentation, the README points at the site: the documentation at lws.sigs.k8s.io/docs/, the installation guide, the examples, and a page for adopters, integrations and talks. The community channels are the Kubernetes ones, the sig-apps Slack channel and the kubernetes-sig-apps mailing list, and the project is under the Kubernetes Code of Conduct.

## The repository layout follows the Kubernetes project conventions

The file listing is the standard Kubernetes SIG layout, and reading it tells you where to look.

The api/ directory holds the API types, and client-go/ holds the generated client. The keps/ directory holds the enhancement proposals, which is where the design of a new API is argued out before the code is written, and the DisaggregatedSet proposal is filed there. The cmd/ directory holds the manager entry point, pkg/ holds the internal packages, and config/ holds the deployment manifests.

The charts/ directory is the Helm chart, so installation is a chart install rather than a pile of manifests to apply, and the site/ directory with netlify.toml is the documentation site source, which is what produces the pages the README links to. The test/ directory holds the test code, and hack/ holds the development scripts.

There is also a Dockerfile and a Makefile, a cloudbuild.yaml for Google Cloud builds, a CODEOWNERS file, a PROJECT file, a RELEASE.md, a SECURITY.md and a SECURITY_CONTACTS file, and two golangci configuration files, .golangci.yaml and .golangci-kal.yml.

The go.mod is where the technical choices are visible. The module is sigs.k8s.io/lws and it requires Go 1.26.0. The Kubernetes libraries are pinned at v0.37.1, matching the newest version in the support window, and the controller framework is sigs.k8s.io/controller-runtime at v0.25.2. The Ginkgo and Gomega test libraries are there for the integration and e2e tests, and the interesting dependency for an inference workload is volcano.sh/apis at v1.12.1, which is the batch scheduling project whose APIs this project builds on for the gang scheduling behaviour.

## The container image is distroless and built for four architectures

The build setup is the standard kubebuilder scaffold with a few choices worth noticing.

The Dockerfile takes its builder and base images as build arguments, golang 1.26 for the builder and the distroless static nonroot base for the runtime. The manager binary is compiled with CGO disabled, targeting linux and the target architecture, and the runtime stage copies only the binary, runs as the non root user 65532 and 65532, and sets the entrypoint to the manager.

That base image choice is a security decision with a practical consequence. A distroless static image contains no shell and no package manager, so there is nothing in the running container to attack and nothing to keep patched, but you also cannot exec into it for debugging, so the logs and the metrics endpoint become your only windows into a running controller.

The Makefile shows the release machinery, and two details are worth calling out. The Go version is taken from go.mod as a single source of truth, with the comment saying so explicitly, and the ENVTEST_K8S_VERSION variable pins 1.37.0, which is the kubebuilder assets version used by the envtest binary, so the controller tests run against a specific API server version.

The image is built for multiple platforms, with PLATFORMS defaulting to linux arm64, linux amd64, linux s390x and linux ppc64le, and the buildx command being the default. The comment explains that the target commands are only tested with Docker, and that you might want to replace it to use other tools such as podman. The image tag is derived from the date and the git describe output, and the Makefile documents the buildx requirement, the need for BuildKit, and the need to set a valid IMG value if you intend to push.

So the deliverable is a controller image you can run on amd64, arm64, s390x and ppc64le, which matters for an inference platform where the nodes are not all the same architecture.

## Conclusion

Adopt LeaderWorkerSet if you are running a sharded inference workload where the leader and its workers must be scheduled and scaled together and you need the whole group to be treated as one deployment unit. Do not adopt it for a workload whose pods are independent, since the group semantics and the all or nothing scheduling bring costs for no benefit, and do not adopt it on a cluster older than 1.34, since the project is officially supported and tested on 1.34 and newer. Before you deploy, read the installation guide and the examples, check whether your workload needs the DisaggregatedSet API rather than the core one, and remember that gang scheduling is at Alpha level with an API that may still change.

## FAQ

### What problem does the LeaderWorkerSet API solve?

It deploys a group of pods as a unit of replication, aimed at AI and ML inference workloads, especially multi-host inference where a model is sharded across devices on multiple nodes. Pods in the group share a lifecycle, each has a unique index from 0 to n-1, and the whole group is scaled and rolled out as one unit.

### How does DisaggregatedSet differ from LeaderWorkerSet?

LWS forms the core multi-node API, while DisaggregatedSet builds on it for advanced disaggregated deployment, with support for autoscaling, rollouts and failure handling. It coordinates N-dimensional rollouts across 2 to 10 roles in lockstep while preserving capacity ratios, and manages multiple LeaderWorkerSets as one logical unit.

### Which Kubernetes versions does LeaderWorkerSet support?

It is officially supported and tested on Kubernetes 1.34 and newer, currently tested on 1.34 through 1.37. The end to end tests run on KIND with one TestGrid job per minor version.

### Is gang scheduling in LeaderWorkerSet stable?

No. Each replica with a group of pods can be scheduled in an all-or-nothing manner at Alpha level, and the README notes that the API may change in the future for that feature.

### How do I install LeaderWorkerSet?

The README points at the installation guide in the project documentation and the examples page, and the repository holds a Helm chart in the charts/ directory. The project is Apache-2.0 licensed, reached v1.0.0 on 2026-10-16, and the last push was on 2026-10-20.

## Sources

- [kubernetes-sigs/lws on GitHub](https://github.com/kubernetes-sigs/lws)
- [License: Apache-2.0](https://github.com/kubernetes-sigs/lws/blob/main/LICENSE)
- [Project website](https://lws.sigs.k8s.io)
- [README](https://github.com/kubernetes-sigs/lws/blob/main/README.md)
- [Releases](https://github.com/kubernetes-sigs/lws/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/kubernetes-sigs-lws