# GPUtil: picking a free NVIDIA GPU by parsing nvidia-smi output > GPUtil is a small MIT licensed Python module that shells out to nvidia-smi, parses the output into per device load and memory figures, and returns the ids of GPUs that look free. It is written for deep learning workloads that need to pick a device, and it carries the usual caveats of a text scraping approach. **anderskm/gputil** — A Python module for getting the GPU status from NVIDA GPUs using nvidia-smi programmically in Python - Repository: https://github.com/anderskm/gputil - Stars: 1,215 · Forks: 132 - Language: Python - License: MIT - Published: 2026-10-19 · Updated: 2026-10-19 · Language: en - Canonical page: https://hysenlabs.com/projects/anderskm-gputil ## GPUtil scrapes nvidia-smi because there is no other channel The mechanism behind GPUtil is not an API. The documentation is blunt about it: GPUtil uses the program nvidia-smi to get the GPU status of all available NVIDIA GPUs, and nvidia-smi is expected to be installed automatically with the NVIDIA driver. On a Linux box the module runs a subprocess, captures the text output, and parses it into a list of GPU objects. The stated purpose is narrow and specific. GPUtil locates all GPUs on the computer, determines their availability, and returns an ordered list of available GPUs. Availability is based on current memory consumption and load of each GPU. The documentation says the module is written with GPU selection for deep learning in mind, while also noting it is not task or library specific and can be applied to any task where identifying available GPUs is useful. So the library exists to fill a gap that appears in any script that trains on a machine with more than one card. Without it, you either hardcode device 0 and hope, or write your own parser of nvidia-smi output, which is exactly the repetitive, brittle work GPUtil packages. The value is convenience and a stable call signature, not a superior data source. The dependency list in the documentation is entirely standard library: subprocess, distutils, math, random, time, os, sys and platform. The presence of distutils matters later, because setup.py in this repository imports it at the top level, and the documented support claim covers both Python 2.X and 3.X. The documentation says it was tested on CUDA driver version 390.77 with Python 2.7 and 3.5. That is a narrow and dated test matrix, and the distutils import is a concrete warning about running the package install path on an interpreter whose standard library no longer bundles that module. ## Installing with pip and confirming the module sees your cards The documented install is a single command in a terminal: ```sh pip install gputil ``` The documentation then tells you to test the install from a folder other than the GPUtil folder, so that a local checkout does not shadow the installed package, and to start a Python console: ```python import GPUtil GPUtil.showUtilization() ``` The expected output is a small table, described as depending on the number of GPUs and their current usage: ``` ID GPU MEM -------------- 0 0% 0% ``` The three columns are the device id, the GPU load percentage and the memory usage percentage. If this call prints an empty list, the module found no parsable devices, and the first place to look is whether nvidia-smi is on your PATH and whether the driver is present at all. There is an older installation route as well. You can download or clone the repository and add the GPUtil folder to PYTHONPATH in your bashrc, so the example given in the documentation is: ``` export PYTHONPATH="$PYTHONPATH:/home/anderskm/github/gputil" ``` That path is what you would use if you need to run against a working tree rather than the published package. The repository layout is small, a single GPUtil package directory alongside setup.py, setup.cfg, MANIFEST.in and LICENSE.txt, so a checkout is quick to inspect. The packaging metadata in setup.py is where the version question is settled. It declares name = 'GPUtil', version = '1.4.0', packages = ['GPUtil'], license = 'MIT', and a download_url pointing at the v1.4.0 tarball. The setup script calls distutils.core.setup rather than setuptools. If you are installing on a modern Python where distutils is no longer bundled, expect the install step itself to be the thing that breaks, not the library logic. ## getAvailable returns ids, and every ordering you can ask for The main entry point returns device ids, not GPU objects: ```python deviceIDs = GPUtil.getAvailable(order = 'first', limit = 1, maxLoad = 0.5, maxMemory = 0.5, includeNan=False, excludeID=[], excludeUUID=[]) ``` Availability is a threshold test. A GPU is considered available if its current load is below maxLoad and its current memory usage is below maxMemory. The defaults are 0.5 for both, so a card already half busy or half occupied is skipped. limit caps the number of ids returned and defaults to 1, which is the common case of wanting one card for one training job. The order argument determines the shape of the result and takes five documented values. 'first' orders by ascending id and is the default, 'last' orders by descending id, 'random' shuffles, 'load' orders by ascending load, and 'memory' orders by ascending memory usage. The last two are the ones that matter when you want the emptiest card rather than the lowest numbered card, and they only order devices that already passed the availability threshold. Two exclusion arguments let you pin specific devices out of the pool. excludeID takes a list of GPU ids and excludeUUID does the same job using the device UUID instead, which is the more reliable handle when device enumeration order can change between boots. Both default to an empty list. includeNan is the argument that shows the module knows it can fail. It is a flag for whether to include GPUs where either load or memory usage is NaN, meaning the usage could not be retrieved. It defaults to False, so unreadable devices are skipped rather than handed to your job. A NaN device is not a busy device, it is an unreadable device, and including it by default would hand your training script a card whose state you know nothing about. If you want the object rather than the id, the documentation points to the GPU class description for the identifiers used by excludeID, and offers helper functions beyond the two main ones. The section on helper functions is where the remaining API lives, and the examples section shows the same calls in context. ## getFirstAvailable throws instead of returning None The second main function is a thin wrapper with one behavioural difference that matters in practice: ```python deviceID = GPUtil.getFirstAvailable(order = 'first', maxLoad=0.5, maxMemory=0.5, attempts=1, interval=900, verbose=False) ``` With the default arguments it behaves the same as calling getAvailable(order = 'first', limit = 1, maxLoad = 0.5, maxMemory = 0.5) and taking the single element. The difference is the failure mode. The documentation states that if no available GPU is found, an error is thrown. There is no None return, no empty list, and no graceful degradation. The attempts and interval parameters are the answer to that. attempts is the number of times the function retries before it gives up, and interval is how long it waits between tries. The default of attempts = 1 means a single look with no waiting, so the default behaviour of getFirstAvailable is to raise as soon as every card is busy. Raising is defensible for a training script that has no alternative work to do, and it is a poor fit for a service that should queue or hand the task to another machine. Set attempts above one and the function becomes a polling loop over the same nvidia-smi query, which is a reasonable way to wait for a card to free up in a single node script. The verbose flag exists to report what happens during those attempts. The caveat to keep in mind is that the whole call is a text parse of a subprocess. Between the moment the state is sampled and the moment your job allocates memory, another process can take the card. There is no reservation, no lock and no atomicity, so two scripts calling getFirstAvailable at the same time can be handed the same id. The documentation does not describe any mechanism to prevent that, which means GPUtil is safe to use as a hint and unsafe to use as a scheduler. ## The examples in the documentation are Caffe and TensorFlow era code The usage examples in the documentation are the parts most likely to be dated. The table of contents lists three of them: selecting the first available GPU in Caffe, occupying only one GPU in TensorFlow, and monitoring GPU in a separate thread. The Caffe and TensorFlow references place the library in a specific era of deep learning tooling, and the requirement that it was tested on CUDA driver version 390.77 with Python 2.7 and 3.5 places the code itself around that time. The examples are still instructive about the intended usage pattern, because they show the module being used to set an environment variable for a third party framework rather than to allocate a device directly. The pattern is: ask GPUtil which device is free, then point the framework at that device by id. That indirection is the module's actual job, and it applies to any framework that takes a device index, whether or not it is the one named in the docs. The third example, monitoring in a separate thread, is worth a second look because it implies the module can be called repeatedly and safely from a watcher. The documentation does not, however, describe any caching or throttling, so a monitoring loop built on top of it queries the tool on every poll. The releases tell their own story. v1.4.0 in December 2018 added Windows support and removed GPU temperature and the numpy dependency. v1.3.0 in April 2018 added support for 'not supported' fields and GPU exclusion. v1.2.3 in February 2017 corrected GPU utilization. The v1.3.0 entry is telling: supporting 'not supported' fields is a concession that nvidia-smi output can contain placeholders rather than numbers, which is the same problem includeNan addresses at the API level. The exclusion arguments and the NaN flag were added because the underlying text is inconsistent, not because the data is clean. ## Where GPUtil stops: no per process view, no AMD or Apple hardware Three boundaries are visible in what the documentation covers. First, NVIDIA only. Every entry point is built on nvidia-smi, and the requirement is an NVIDIA GPU with the latest NVIDIA driver installed. There is no path for AMD, Intel or Apple silicon GPUs, and no fallback to a vendor agnostic interface such as ROCm. On a Mac without an NVIDIA card, the module has nothing to talk to. This is not a criticism so much as a definition of scope, but it does mean the library is one of several things you would need to abstract behind if you support more than one accelerator vendor. Second, the module reports device state, not process state. Load and memory figures come from nvidia-smi and describe the card as a whole. If you need to know which Python process is holding 30 GB and whether to kill it, this is the wrong tool. There is no per process accounting, no per process memory query, and no way to attribute usage to a PID through the documented API. Third, the availability model is a threshold, not a lock. Nothing in the documentation describes reserving a device, coordinating between processes, or a lease that expires. A job scheduler that runs many workers on one box needs that coordination, and reading aggregate state twice in a row is not a substitute for it. The maintenance picture deserves a date rather than an adjective. The repository is not archived, and the last push was on 2026-07-18, so work is still landing on the branch. The tagged releases, however, stop in December 2018 with v1.4.0, and setup.py still declares version = '1.4.0'. The gap between a recent push and a 2018 version number means the package you install from PyPI and the code on the branch are not the same thing, and that the maintained version of GPUtil is something you would have to read the source of to understand. The licence is MIT, which is permissive, and the code is small enough that vendoring it into your own repository is a realistic option if the parse behaviour needs to change for your driver version. ## Conclusion GPUtil is a reasonable choice for a single node training script that has to pick one of several NVIDIA GPUs and has no appetite for a larger monitoring stack, and it is a poor fit for a scheduler, a cluster or anything that needs per process accounting, since it reads aggregate device state from nvidia-smi and offers no locking between callers. Before adopting, check that the nvidia-smi query format on your driver still parses, since the last tagged release v1.4.0 dates from 2018 and the package version in setup.py is still 1.4.0, and confirm the pip install gputil path resolves the version you expect. ## FAQ ### How do I install GPUtil and check that it sees my GPUs? Run pip install gputil, then from a folder other than the source checkout run import GPUtil followed by GPUtil.showUtilization(), which prints an ID, GPU and MEM table. The module needs nvidia-smi on the machine, which comes with the NVIDIA driver. ### How do I pick the GPU with the most free memory in GPUtil? Call getAvailable with order set to 'memory', which orders available device ids by ascending memory usage, and set maxMemory to the maximum relative memory usage a card may have and still count as available. The function returns ids, not GPU objects. ### What happens when no GPU is free in GPUtil? getFirstAvailable throws an error if no available GPU is found. Passing attempts above the default of 1 makes it retry on an interval instead of failing on the first check. ### Does GPUtil work on non NVIDIA GPUs or on modern Python versions? It is NVIDIA only, because it reads nvidia-smi output. The setup script imports distutils from the standard library, and the documented test matrix is CUDA driver 390.77 with Python 2.7 and 3.5. ## Sources - [anderskm/gputil on GitHub](https://github.com/anderskm/gputil) - [Issues](https://github.com/anderskm/gputil/issues) - [License: MIT](https://github.com/anderskm/gputil/blob/master/LICENSE) - [README](https://github.com/anderskm/gputil/blob/master/README.md) - [Releases](https://github.com/anderskm/gputil/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/anderskm-gputil