scheduler-plugins gives Kubernetes scheduling new extension points

Out-of-tree, but still inside the scheduler framework

The scheduler-plugins repository is for out-of-tree scheduler plugins based on the scheduler framework. The description says these plugins are exercised in large companies, which is the honest way to describe the difference from the in-tree scheduler: the code is not part of kube-scheduler, but it plugs into the same extension points.

The scheduler framework is what makes that possible. It splits scheduling into a chain of extension points, filter, score, reserve, permit, pre-bind, bind, post-bind and pre-filter, and a plugin can implement the ones it cares about. A plugin that only scores is a policy, a plugin that filters is a constraint, and the framework composes them. That is why a scheduling requirement like capacity limits or topology awareness can be added to a cluster without forking kube-scheduler.

The repository states three ways to consume it. The plugins can be vendored as Go SDK libraries, which is the path for a team that wants to build its own scheduler binary. They can be used out of the box via the pre-built images or the Helm charts. And the repository incorporates best practices and utilities to compose a high-quality scheduler plugin, which is the path for a team writing their own.

The primary language is Go, and the module path is sigs.k8s.io/scheduler-plugins. It is an Apache-2.0 licensed project in the Kubernetes SIG space, with a SECURITY_CONTACTS file, a code of conduct, a PROJECT file, an OWNERS file and a RELEASE.md, all of which are the standard governance furniture of a Kubernetes subproject.

Seven production plugins and three samples

The plugin list is split into two tiers, and the distinction is explicit rather than implied.

The kube-scheduler binary includes seven plugins intended for real use. Capacity Scheduling handles bin packing and resource limits per group, for clusters where tenants need guarantees. Coscheduling groups pods that must run together. Node Resources and Node Resource Topology expose node level capacity and topology to the scheduler. Preemption Toleration changes which pods can be preempted. Trimaran provides load aware scheduling. Network-Aware Scheduling places pods with regard to the network.

Then there are three sample plugins, and the README says plainly that these are not intended for use in production environments. They are Cross Node Preemption, Pod State, which records the state of pods so the scheduler can reason about it, and Quality of Service, which factors QoS class into scheduling decisions. They are there as reference implementations and as a starting point for your own plugin, which is a reasonable thing to ship in a repository whose stated purpose includes demonstrating how to compose a good plugin.

Each plugin lives in its own directory under pkg/ with its own README, so the plugin boundaries are visible in the tree rather than only in a list. The whole set is enabled through scheduler profiles, which is a Kubernetes configuration surface for running several differently configured schedulers in one cluster, and the README points at the Kubernetes documentation for multiple profiles as the mechanism.

The two image names in the repository are kube-scheduler and controller, and the difference is worth understanding: the first is a scheduler binary that replaces or supplements the upstream one, the second is the controller that drives the out-of-tree plugins, such as the capacity controller for the capacity scheduling plugin.

Version numbers encode which Kubernetes they were built for

The compatibility matrix is the most practically useful thing in the README, and the numbering convention behind it is unusual enough to explain.

The minor version of scheduler-plugins matches the minor version of the Kubernetes client packages it is compiled with, so the v0.18.x releases are built with Kubernetes v1.18.x dependencies. The current table shows v0.35.7 compiled with v1.35.7, v0.34.7 with v1.34.7 and v0.33.5 with v1.33.5, while the go.mod on master instead requires k8s.io/api, apimachinery, apiserver, client-go, code-generator, component-base, component-helpers and kube-scheduler all at v0.36.4 along with k8s.io/kubernetes v1.36.4.

The patch versions come in two varieties. Single digit patch versions, such as v0.18.9, exactly align with the Kubernetes client package versions the plugins are built with. Three digit patch versions, such as v0.18.800, are built on demand and indicate that the Kubernetes client package versions have not changed since the previous release, only the plugin code did.

That convention matters before you pick a version. A three digit patch release is not a Kubernetes patch bump, it is a plugin-only release, and if you assume otherwise you will end up with a plugin set built against a different client API than you think.

The images for each release are in the official registry, registry.k8s.io/scheduler-plugins, with a kube-scheduler image and a controller image per version. The current releases are built for four architectures, linux/amd64, linux/arm64, linux/s390x and linux/ppc64le, and the Makefile confirms that set as the PLATFORMS default. Older releases in the collapsed table drop to two architectures, which is a practical constraint if you run s390x or ppc64le on an older release.

Installing means running a custom scheduler image

The install path is to pull the images and point a scheduler at them. The README's install section is short:

docker pull registry.k8s.io/scheduler-plugins/kube-scheduler:$TAG
docker pull registry.k8s.io/scheduler-plugins/controller:$TAG

There are two images, one for the kube-scheduler and one for the controller, and the compatibility matrix gives the complete list of images with their tags. The README links a separate install document, doc/install.md, for how to install a release image.

The build system tells you more about how the images are produced. The Makefile derives the Go version from the go.mod, which currently declares go 1.26.0, and uses golang:$(GO_VERSION) as the build base with gcr.io/distroless/static:nonroot as the distroless base image. The registry default is the staging registry rather than production, and a release version is composed of a date and a git describe, so an automated build for an untagged commit and a tagged release differ only in the describe suffix.

The repository layout is the standard Kubernetes subproject shape. There is pkg/ for the plugin implementations, cmd/ for the command entry points, apis/ for the API types the plugins introduce, manifests/ for the deployment YAML, config/ for the scheduler configuration, build/ and hack/ for the build and update scripts, kep/ for the enhancement proposals, doc/ for the documentation, site/ for the docs site with a netlify.toml, test/ for the integration tests, and a cloudbuild.yaml for the image build pipeline.

The dependencies reveal what the plugins actually reach for. There is gonum for the numerical work the load aware and topology plugins need, go-cache for the in-memory caching, diktyo-io's appgroup-api and networktopology-api for the network aware plugin, the node resource topology working group's API and podfingerprint packages, paypal/load-watcher for the Trimaran load metrics, containers/common and the security profiles operator for the sandboxed or policy aware paths, and controller-runtime for the controller side.

Where the default scheduler is the better answer

The first limit is that most clusters do not need this. The default kube-scheduler handles the overwhelming majority of workloads, and the plugins here solve specific, hard problems. If your pods fit, you have no reason to run a second scheduler.

The second is the operational cost. Adopting a plugin means running a custom scheduler binary in addition to, or instead of, the upstream one. That means a second Deployment, its own configuration, its own profiles, and the need to keep the version matched to the Kubernetes minor version. Every Kubernetes upgrade becomes a coupled operation between the cluster version and the plugin version, which the compatibility matrix makes explicit rather than hiding.

The third is the maturity of the individual plugins. The repository says these are exercised in large companies, and the sample plugins are explicitly not for production, but between those two statements there is a range of readiness that the README does not itemise. Capacity scheduling and coscheduling are long-established, and the load aware and network aware plugins are newer and depend on external metric and topology APIs, which means they depend on infrastructure you also have to run.

The fourth is the split of the two images. The controller is not a small sidecar. For capacity scheduling it runs a controller that manages the resource groups and their limits, and that controller is a piece of stateful infrastructure with its own failure modes. The plugin is the scheduler extension, and the controller is the thing that makes the plugin's model true, so the full cost of the feature is more than the scheduler change.

There is also a licensing point worth noting for anyone building on this. The repository is Apache-2.0, which is permissive and includes a patent grant, and a team vendoring these plugins as Go SDK libraries into its own scheduler binary inherits that permissiveness. Vendoring, however, means owning the merge of upstream Kubernetes changes, which is a different commitment from running the published image.

Editorial conclusion

scheduler-plugins suits a cluster operator whose scheduling problem is not solvable with the default kube-scheduler, for example a shared cluster that needs capacity guarantees or a gang-scheduled batch workload. It is the wrong tool for a cluster whose pods fit comfortably on the default scheduler, and it is the wrong tool if you cannot accept running a custom scheduler binary. Before adopting a plugin, check the compatibility matrix so the version matches your Kubernetes minor version, and remember that the plugins are configured through scheduler profiles rather than by editing the default scheduler.

Frequently asked questions

Official sources

1. Issues
2. kubernetes-sigs/scheduler-plugins on GitHub
3. License: Apache-2.0
4. README
5. Releases