nvidia_gpu_exporter: Prometheus GPU Metrics Without the Datacenter Stack

The Gap Between Consumer GPUs and Datacenter Monitoring

Datacenter GPU monitoring assumes you can install the NVIDIA GPU Operator and run DCGM-exporter. On a GeForce card in a gaming rig, a vGPU guest, a MIG slice or a locked-down container, that stack is often unavailable, and the deeper GPU counters are not exposed anyway. The README lists these as the primary use cases: consumer and prosumer GPUs where nvidia-smi is the only uniform source of utilization, memory, power and temperature; small Kubernetes clusters, edge boxes and homelabs that want GPU metrics without the Operator; and mixed fleets of old and new cards that need one exporter behaving the same everywhere. The project is explicit that if you run datacenter cards on Kubernetes with the GPU Operator already installed, DCGM-exporter is probably the better fit. That framing is unusually honest for a README and tells you the author knows exactly where the tool stops being useful.

How the Exporter Collects Metrics: nvidia-smi First, NVML Optional

The default backend shells out to the nvidia-smi binary, parses its output and converts it into Prometheus metric families. Because the only dependency is the binary, it runs on Linux, Windows and macOS, on bare metal, in Docker or on Kubernetes, with no C bindings. The README also notes it does not need to run on the monitored machine: it can be configured to execute the nvidia-smi command remotely, which is useful when you cannot install an agent on the GPU host itself. Metric field discovery is automatic, so new fields nvidia-smi exposes in future driver versions are picked up without a code change. Two optional modes change the collection pattern: per-process GPU metrics, which show which process uses how much GPU memory, and background collection, which runs nvidia-smi on a timer instead of on every scrape. That second option matters if the binary is slow or the scrape interval is short. On Linux, an experimental native backend reads metrics directly from the NVIDIA driver library via NVML instead of parsing nvidia-smi. The README states that every metric the default backend serves stays identical in name, labels and value, so existing dashboards and alerts keep working, and the NVML backend adds families nvidia-smi cannot provide: per-MIG-instance metrics, XID error counters, a total energy counter and PCIe throughput. The official Grafana dashboards have panels for those, which sit empty on the default backend.

Running It: Docker, Demo Mode and the NVML Flavor

The quick start assumes a Linux machine with the NVIDIA driver and the NVIDIA Container Toolkit. The README gives this example: docker run -d --name nvidia_gpu_exporter --restart unless-stopped --gpus all -e NVIDIA_DRIVER_CAPABILITIES=utility -p 9835:9835 utkuozdemir/nvidia_gpu_exporter:latest, followed by curl http://localhost:9835/metrics. The port is 9835 and the driver capability is set to utility, which is all nvidia-smi needs. If you have no GPU at hand, the flag --collect.backend demo serves realistic synthetic metrics on any machine. According to the README, demo mode by default simulates two H200 GPUs with fluctuating values, a MIG topology and an XID error history, including the NVML-only families, with no GPU, driver or even Linux required. That makes it a reasonable way to validate your scrape config and dashboard before touching production hardware. The NVML backend ships as its own release flavor: grab a -nvml archive from the releases page, or use a -nvml image tag such as utkuozdemir/nvidia_gpu_exporter:latest-nvml with the same docker run flags. The README points to docs/CONFIGURE.md for the full backend comparison and current limits, and to docs/INSTALL.md for Windows, macOS, packages, Kubernetes and non-Docker installs. Windows and winget are mentioned in the badge list, so those install paths exist in the docs rather than in the README body.

The NVML Backend Is Experimental, and That Matters

The README marks the native NVML backend as experimental, and the stated reason is testing coverage: it needs more testing across driver versions and GPU generations. The author asks users who try it to open an issue about how it went, good or bad, and says that is what will get it past the experimental label. Read that as a real constraint rather than marketing hedging. If you depend on per-MIG-instance metrics, XID counters, the total energy counter or PCIe throughput, you are relying on the least-tested path in the project, across a driver matrix the maintainer has not fully covered. The default nvidia-smi backend is the safer choice for anything you cannot afford to debug, and the README's own framing supports that: the NVML flavor is a separate archive or image tag precisely so that adopting it is a deliberate act. A second limitation is structural rather than technical. The README carries a warning that this is a side project maintained in spare time, and that issues or PRs might take a long time to be looked at, or not be looked at at all. For a homelab dashboard that is acceptable. For a monitoring dependency in a production fleet, it is a risk you should price in before adopting, especially if you plan to rely on the experimental backend. The project is not archived and releases have been frequent, but the maintainer's own statement about response times is the relevant fact for planning.

DCGM-exporter and the Difference in Approach

The README names DCGM-exporter directly and says it is probably the better fit if you run datacenter cards on Kubernetes with the GPU Operator already installed. The difference is not just packaging. DCGM-exporter is part of the NVIDIA datacenter stack and depends on DCGM to reach the GPU through the driver's profiling interfaces, which is why it can expose counters that nvidia-smi does not surface. nvidia_gpu_exporter takes the opposite route: it treats nvidia-smi as the interface, which means it inherits nvidia-smi's limits but also its portability. That is why it runs on Windows without Docker or Linux, and why it can execute the command remotely against a host where you cannot install an agent. The NVML backend narrows the gap by adding per-MIG-instance metrics, XID errors, energy and PCIe throughput, but it is Linux-only and experimental, while DCGM-exporter is the supported datacenter path. If your environment already has the GPU Operator, adding this exporter buys you nothing and gives you a second thing to maintain. If your environment cannot have the GPU Operator, DCGM-exporter is not an alternative at all, which is precisely the case this project is built for.

Licence and Maintenance Cost

The project is MIT licensed, which is permissive and imposes no copyleft obligation on how you deploy or redistribute it. The usual MIT condition applies: the copyright notice and permission notice must be included in copies or substantial portions of the software, so if you vendor or repackage the exporter, keep the LICENSE file with it. That is a description of the licence text, not legal advice, and if you are redistributing it commercially you should read the LICENSE in the repository yourself. On maintenance, the practical cost is low but not zero. There is no database, no agent to keep in sync, and no C bindings to rebuild against new drivers, which is the main reason the nvidia-smi approach survives driver upgrades that break compiled integrations. Upgrades are a container tag or a binary swap, and the README says the NVML backend keeps metric names, labels and values identical to the default backend, so switching between flavors does not require dashboard changes. The cost that does not disappear is the maintainer's availability, stated plainly in the README warning. If you need a guaranteed response to a bug in the experimental NVML path, this project does not offer one. Budget for pinning a known-good version rather than tracking latest.

Who Should Adopt This, and What to Check First

Adopt it if your GPUs are consumer or prosumer cards, if you run a small Kubernetes cluster, edge box or homelab without the GPU Operator, if you are in a vGPU guest, MIG slice or locked-down container where nvidia-smi still answers but deeper counters do not, or if you have a mixed fleet that needs one exporter behaving the same on old and new cards. Do not adopt it if you run datacenter cards on Kubernetes with the GPU Operator installed; the README points you to DCGM-exporter and that is the right call. Do not adopt the NVML backend for anything you cannot afford to troubleshoot across driver versions, given its experimental status. Before deploying, verify three things. First, that nvidia-smi returns data on the target host, or that the -nvml flavor loads against your driver version. Second, that your scrape path works end to end by running nvidia_gpu_exporter --collect.backend demo and pointing Prometheus at port 9835. Third, whether you need per-process metrics or background collection, since both change how often the underlying command runs and therefore what your scrape interval can be. The two Grafana dashboards referenced in the README, per-GPU detail and multi-GPU overview, are the intended visualization layer and are worth importing before you write your own.

Editorial conclusion

Adopt nvidia_gpu_exporter if you run consumer or prosumer GPUs, small Kubernetes clusters or vGPU guests where nvidia-smi already works and the NVIDIA GPU Operator is not installed. Do not adopt it if you run datacenter cards on Kubernetes with the GPU Operator, where the README itself points to DCGM-exporter. Before deploying, verify that nvidia-smi answers on the target host (or that the NVML flavor loads against your driver), set --collect.backend demo to confirm the metrics endpoint and dashboards work end to end, and check whether per-process or background collection modes fit your scrape interval.

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. utkuozdemir/nvidia_gpu_exporter on GitHub