sriov-network-device-plugin lets a pod claim an SR-IOV virtual function as a resource

Three kinds of device, one plugin

The SR-IOV Network Device Plugin is a Kubernetes device plugin for discovering and advertising networking resources on a Kubernetes host, and the resources come in three forms.

The first is SR-IOV virtual functions, VFs. The second is PCI physical functions, PFs. The third is auxiliary network devices, in particular Subfunctions, SFs.

That third category is the newest thing here and the one with the narrowest support. The feature list says it handles PCI backed auxiliary network devices, at the moment SFs only, so the subfunction support is deliberately scoped rather than general.

The feature list also covers the cases that normally go wrong in device plugins. It handles devices that are SR-IOV capable and devices that are not, NICs and accelerators alike. It supports devices with both kernel and userspace drivers, UIO and VFIO. It allows resource grouping using selectors, and the resourceName is user configurable.

Two operational behaviours are worth calling out because they are the difference between a plugin that works in a cluster and one that does not. It detects kubelet restarts and auto-re-registers, which matters because a device plugin that loses its registration after a kubelet restart silently disappears from the node's allocatable resources. And it detects link status for Linux network devices and updates the associated VFs health accordingly, so a VF whose link goes down is not handed out as though it were healthy.

The extension story is stated too: extensible to support new device types with minimal effort if not already supported.

The project is Go, Apache-2.0 licensed, and part of the Kubernetes Network Plumbing Working Group, with a weekly meeting on Mondays at 3pm GMT.

It only advertises, other components do the plumbing

The plugin does not attach a network device to a pod on its own, and the README is precise about the division of labour.

To deploy workloads with an SR-IOV VF, an auxiliary network device or a PCI PF, the plugin needs to work together with two CNI components.

The first is any CNI meta plugin supporting device plugin based network provisioning, which means Multus CNI or DANM, and its job is to retrieve the allocated network device information of a pod.

The second is a CNI capable of consuming the network device allocated to the pod. For SR-IOV VFs that is the SR-IOV CNI, which during pod creation plumbs the allocated SR-IOV VF into the pod's network namespace using the VF information given by the meta plugin, and on pod deletion resets and releases the VF from the pod. For PCI PFs it is the host device CNI, which does the same with the allocated physical device.

So there are four moving parts, and each has to be present: the device plugin, the meta plugin, the CNI, and the virtual functions themselves.

The Quick Start spells out the order. Before starting the plugin you need to create the desired network functions on your system, with a separate document for creating SR-IOV virtual functions at docs/vf-setup.md and another for creating subfunctions at docs/subfunctions/README.md. Then the SR-IOV CNI is installed, supported from SR-IOV CNI release 2.0 and later. Then the plugin image. Then the plugin itself. Then the meta plugin.

That ordering is not bureaucratic, it is the dependency order, and skipping a step leaves you with a plugin that advertises devices nothing will attach.

Installing it, step by step

The plugin image comes either from GitHub or a local build.

 docker pull ghcr.io/k8snetworkplumbingwg/sriov-network-device-plugin:latest

Or built on each node with the Makefile.

 make image

On a successful build a docker image with the tag ghcr.io/k8snetworkplumbingwg/sriov-network-device-plugin:latest is created, and the README notes you need to build this image on each node, or use a local docker registry to host it.

Then a ConfigMap defining the SR-IOV resource pool configuration.

 kubectl create -f deployments/configMap.yaml

The reminder attached to that step matters: make sure to update the config.json entry in the configMap data to reflect your resource configuration for the device plugin, because the shipped ConfigMap is a template, not a working configuration for your hardware.

Then the DaemonSet itself.

 kubectl create -f deployments/sriovdp-daemonset.yaml

And finally a compatible CNI meta plugin. The README says one is required for the SR-IOV CNI plugin to be able to get the allocated VF's deviceID in order to configure it, and then offers two options, Multus or DANM, with a pointer to the Multus quickstart installation guide.

The Makefile behind the image build is worth a look for the details it exposes. The binary is named sriovdp and is built from the cmd/sriovdp directory, the default build tags are no_openssl, and there is a STATIC option that sets CGO_ENABLED=0 and adds static linker flags for a fully static binary. The lint tooling is pinned, with golangci-lint at v2.12.2 installed into the local bin directory if it is not already there. The coverage settings use atomic mode with a 15 second test timeout.

The NICs it was tested against

The supported hardware list is explicit, and the wording is careful. The following NICs were tested with this implementation, however other SR-IOV capable NICs should work as well.

The Intel list covers three generations. The Ethernet 800 Series (E810), the 700 Series (XL710, X710, XXV710), and the 500 Series (82599, X520, X540, X550).

The Mellanox entries are broader: ConnectX-4, the Connectx-4 Lx EN Adapter, ConnectX-5, ConnectX-5 Ex, ConnectX-6, ConnectX-6 Dx, and BlueField-2.

And there is Broadcom NetXtreme-E Series Adapters.

The framing of that list is worth reading carefully, because it is a statement about what was validated rather than what is supported. The feature list says it handles SR-IOV capable and not-capable devices, NICs and accelerators alike, so the plugin does not care what the device is. What the list tells you is where to look first if something does not enumerate.

BlueField-2 in that list is also a reminder that the audience is not only networking teams. SmartNICs and DPUs are increasingly what people are trying to schedule onto pods, and a device plugin is the mechanism for getting a DPU or its functions into a container.

Virtual deployments, IOMMU and multi-architecture

Three README sections cover deployment situations that are not a plain bare metal cluster, and they are the ones most likely to matter to you if you are running Kubernetes inside a virtualisation platform.

Virtual deployments support covers configuring device plugin extended selectors in virtual environments, and the feature list mentions that the plugin works within virtual deployments of Kubernetes that do not have virtualized-iommu support, with VFIO No-IOMMU support.

That is a specific and valuable capability. A passthrough device in a virtual machine normally needs an emulated IOMMU, and platforms differ in whether they provide one. If yours does not, the usual answer is that VFIO cannot be used with an I/O memory management unit, and SR-IOV functions are unusable. This plugin's answer is that it can work in that configuration, which widens the set of platforms where SR-IOV scheduling is possible.

There are also separate sections on CNI plugins in virtual environments and on virtual environments with no iommu, which suggests the differences are handled per component rather than only in the plugin.

Multi architecture support has its own section, which matters because the hardware you are scheduling is usually x86 but the machine running the cluster might not be.

And there is a section on the Container Device Interface, which is the newer standard that lets a plugin advertise devices through a CDI rather than the original device plugin API. The go.mod shows the container-device-interface dependency at v0.5.4, so both paths are present in the code.

The Go dependency list, and what it tells you

The go.mod is short enough to read as a description of what the plugin touches, and it is built on the working group's own libraries rather than on generic tooling.

The Go directive is 1.26.0. The direct dependencies include github.com/Mellanox/rdmamap for RDMA memory mapping, jaypipes/ghw for hardware discovery and jaypipes/pcidb for the PCI device database.

Then there are three libraries from the same working group. github.com/k8snetworkplumbingwg/govdpa at v0.1.4 is the vendor-agnostic device plugin API, the network-attachment-definition-client at v1.7.7 is the client for the NetworkAttachmentDefinition CRDs that Multus uses, and github.com/k8snetworkplumbingwg/sriovnet at v1.3.0 handles the SR-IOV netlink operations.

The Kubernetes side is apimachinery and kubelet, both at v0.36.2, with grpc at v1.84.0 for the device plugin server protocol, and vishvananda/netlink for the low level kernel networking calls.

The test dependencies are Ginkgo and Gomega, the pair used for the table driven suites, plus testify.

The repository layout follows the standard Go layout with cmd/ for the entry point, pkg/ for the packages, deployments/ for the manifests, docs/ for the setup guides, images/ for the Dockerfile, test/ for the suites and a .golangci.yml for linting, plus a .coderabbit.yaml for automated code review.

Where the SR-IOV Network Device Plugin is the wrong choice

Four cases argue against it.

The first is a cluster without the hardware. This plugin does not create virtual functions, it advertises ones that already exist. If your nodes are generic cloud instances with a software bond and no SR-IOV capable NIC, there is nothing for it to do.

The second is a workload that does not need a dedicated function. The point of SR-IOV is that the pod gets the physical function or the virtual function directly, bypassing the virtual switch, and that is what gives you predictable latency, RDMA and hardware offload. If your application is fine with an eBPF datapath or a virtio interface, the complexity of this stack buys nothing, and the ordinary CNI plus a network policy is the right answer.

The third is a team that wants a single component. Four pieces have to be installed and kept in step, the plugin, the meta plugin, the CNI and the virtual function setup on every node. And the config.json in the ConfigMap has to describe your actual resource layout, which means the plugin is not deployable without per-hardware configuration.

The alternative in the same direction is the SR-IOV CNI with Multus directly, or a hardware vendor's own scheduler integration. The difference in approach is that the device plugin is the Kubernetes-native way to make a function schedulable, so the scheduler accounts for it, the allocation is tracked and released with the pod, and the resource appears in the node's allocatable list. Without the plugin, Multus can still hand a VF to a pod, but the cluster does not know the VF is in use, two pods can be given the same function, and nothing prevents the scheduler from placing a pod on a node that cannot satisfy the request.

Editorial conclusion

sriov-network-device-plugin fits a cluster whose nodes have SR-IOV capable hardware and a workload that genuinely needs a dedicated function or physical function, since it turns those devices into a first class schedulable resource with selectors. It does not fit a general cluster, because the hardware, the virtual function setup, the meta plugin and the CNI all have to be in place before the plugin has anything to advertise. Create the virtual functions first following docs/vf-setup.md, install the SR-IOV CNI at release 2.0 or later, then apply `deployments/configMap.yaml` and `deployments/sriovdp-daemonset.yaml`, and check the supported NIC list before buying hardware.

Frequently asked questions

Official sources

1. Issues
2. k8snetworkplumbingwg/sriov-network-device-plugin on GitHub
3. License: Apache-2.0
4. README
5. Releases