NBD-VRAM: Using Idle GPU Memory as a Linux Swap Tier

The Problem: VRAM Sitting Idle While RAM Fills Up

Hybrid-graphics laptops pair an AMD or Intel integrated GPU for the display with a discrete NVIDIA card for compute. On these machines the display runs through the integrated GPU, which leaves the NVIDIA card idle most of the time, its VRAM unused. Meanwhile the system's soldered RAM has no upgrade path. When parallel compiles, browser tabs, or data jobs push the machine into swap, the only destination is the SSD.

SSDs handle swap poorly. A page fault to NVMe can stall the requesting process for milliseconds. On a machine where every NAND write cycle counts, swap traffic accelerates wear. nbd-vram redirects that overflow to the GPU's VRAM instead. The README describes its tested setup as an AMD/ATI plus RTX 3070 Laptop combination with 16 GB of RAM and 8 GB of VRAM; allocating 7 GB to swap produced approximately 46 GB of total addressable memory across RAM, VRAM, zram, and SSD.

The README describes the latency difference plainly: a page fault answered from VRAM takes around 250 microseconds. The same fault answered from NVMe waking from a power-saving sleep takes milliseconds. The difference does not show up as a throughput number but as the absence of stutter when switching back to an application left open for hours.

The Architecture: NBD, CUDA, and No Kernel Code

The daemon allocates VRAM using the CUDA driver API, then exposes it as a block device through the Linux Network Block Device protocol over a Unix socket. The kernel's built-in nbd driver connects to that socket and presents /dev/nbdX as an ordinary block device. From there the kernel swap subsystem treats it like any other partition.

The data path is: kernel swap subsystem to /dev/nbdX, nbd kernel driver over a Unix socket, nbd-vram daemon, then cuMemcpyHtoD and cuMemcpyDtoH for the actual transfers to and from GPU VRAM.

The README explains why this path was chosen over two alternatives. The NVIDIA P2P API (nvidia_p2p_get_pages_persistent), which would pin VRAM pages in BAR1 for direct CPU access, returns EINVAL on consumer GeForce GPUs at the driver level regardless of driver version. The second alternative, mapping BAR1 physical addresses with ioremap_wc without the P2P API, fails because the GPU's internal page tables expose only about 16 MiB of BAR1 (the display framebuffer), so the rest reads as zeros and mkswap appears to succeed while swapon fails. The NBD path works on any CUDA-capable GPU with no special permissions because cuMemcpyHtoD and cuMemcpyDtoH are available to all CUDA consumers.

The build requires only gcc and make. The daemon links against libcuda.so.1 but does not require the full CUDA toolkit, only the driver library that ships with the standard NVIDIA driver package. The Makefile compiles nbd-vram.c with optimization level O2 and links lpthread for the worker threads.

Installing nbd-vram and Verifying the Swap Device

The repository provides an install.sh script that handles building, deploying the systemd service, and setting thread counts automatically. Clone the repository, then run the installer with sudo:

git clone https://github.com/c0dejedi/nbd-vram
cd nbd-vram
sudo ./install.sh
sudo systemctl start vram-swap-nbd

After the service starts, confirm the device is active:

swapon --show
# NAME       TYPE      SIZE USED PRIO
# /dev/nbd0  partition   7G   0B 1500

The service is enabled on install and comes up automatically on subsequent boots. The installer also enables vram-swap-nbd-suspend.service, which tears down the swap device before the machine sleeps and rebuilds it on resume. The installer will ask whether to enable power-aware management, which stops the service on battery and restarts it on AC power.

The prerequisites are an NVIDIA GPU with CUDA support, the NVIDIA driver with libcuda.so.1 present, the nbd-client package, gcc, and make. Linux kernel 5.6 or later is recommended because the swap-deadlock protection relies on PR_SET_IO_FLUSHER, which landed in that release. Older kernels still run the daemon but with that protection disabled.

Configuring VRAM Size, Thread Count, and Swap Priority

Configuration lives in the systemd service file at /etc/systemd/system/vram-swap-nbd.service as environment variables:

Environment=VRAM_SETUP_SIZE_MB=7168
Environment=VRAM_SWAP_PRIORITY=1500
Environment=VRAM_NBD_THREADS=8
Environment=VRAM_NBD_CONNECTIONS=8

VRAM_SETUP_SIZE_MB is a ceiling, not a hard requirement. The daemon tries the requested allocation and backs off in 512 MiB steps if the display compositor has already claimed some VRAM, so it grabs as much as available up to the limit.

VRAM_NBD_THREADS and VRAM_NBD_CONNECTIONS are auto-set to the output of nproc at install time and should be kept equal to each other. The README states that the benefit of more connections saturates around the physical core count, and single-stream workloads do not use parallelism at all. VRAM_SWAP_PRIORITY at 1500 places VRAM swap above any typical SSD swap device, ensuring it fills before the SSD is touched.

After editing the file, reload and restart with:

sudo systemctl daemon-reload && sudo systemctl restart vram-swap-nbd

The power management configuration lives in /etc/nbd-vram.conf and can be changed after install. Changes take effect within 60 seconds or immediately on the next AC plug or unplug event.

Suspend, Resume, and Shared GPU Workloads

The suspend service (vram-swap-nbd-suspend.service) is ordered before systemd's sleep services and before NVIDIA's power-down hooks. This ordering ensures the swapoff completes while the GPU context is still live, VRAM is freed before the GPU powers off, and a fresh CUDA context is built on resume.

There is one failure mode the README documents explicitly: if swap is heavily used at suspend time and the paged-out data cannot fit back into RAM, the pre-sleep swapoff fails with ENOMEM. The README states this is intentional, as forcing the evacuation could cause a kernel panic. On a heavily swapped machine, freeing memory before suspending is the user's responsibility. There is no automatic workaround.

On a machine running CUDA inference, game rendering, or any other GPU workload simultaneously, nbd-vram competes directly for VRAM. The daemon tries to allocate up to its configured ceiling; whatever it claims is unavailable to the other application. The README does not document a dynamic sizing mode, so the allocation is static for the lifetime of the service. Users running occasional GPU workloads can stop the service manually with sudo systemctl stop vram-swap-nbd; the README states that a manual stop is always respected and the power management service will not restart it.

Limitations and When to Reach for zram Instead

nbd-vram is the wrong tool in several common situations. On a desktop with socketed RAM slots, adding physical memory costs less and removes the PCIe latency entirely. On a laptop without a discrete NVIDIA GPU, there is no VRAM to allocate. On machines where the GPU runs continuous workloads (ML training, video encoding, gaming), the VRAM budget is already spoken for.

The NBD path carries PCIe latency that VRAM-direct access would not. The README frames this as acceptable given that the alternative is NVMe, but PCIe round-trips are still measurably slower than DRAM. For workloads that take many simultaneous page faults, the thread-per-connection model helps, but the README notes that single-stream workloads gain nothing from multiple connections.

zram, which compresses pages in CPU memory, is a different trade-off. zram adds CPU load for compression and decompression; nbd-vram adds PCIe round-trip latency and GPU context overhead. The project's tested configuration stacks both: VRAM absorbs the overflow from RAM, zram compresses what VRAM cannot hold, and SSD catches the rest. Using zram alone is a reasonable choice on a machine without a discrete GPU or when the CUDA driver is not installed.

The repository has no GitHub releases. Build requirements (gcc, make, nbd-client) are standard but not always present by default on minimal installs.

Maintenance Status and Licence

The repository is not archived and its last push was on 2026-07-17, which is within the past six months. The project is licensed under the MIT licence, which permits use, modification, and redistribution with attribution.

The daemon is a single C file (nbd-vram.c) linked with libdl and libpthread. There is no dependency on CUDA headers or the full CUDA toolkit, only on the runtime library libcuda.so.1. Because no kernel symbols are used and no out-of-tree module is compiled, the daemon survives kernel and driver updates without rebuilding. Driver updates that change the CUDA ABI could in principle break the dlopen path, but this has not been documented as a known issue.

There is no upgrade mechanism beyond pulling the repository and rerunning install.sh. Configuration files written by the installer (/etc/systemd/system/vram-swap-nbd.service and /etc/nbd-vram.conf) are not overwritten automatically on reinstall; the README does not document whether the installer preserves or replaces them. Checking both files after an update is prudent.

Editorial conclusion

nbd-vram is the right choice for a Linux user running a hybrid-graphics laptop with soldered RAM, an idle NVIDIA RTX or GTX card, and real swap pressure from parallel builds, browser tabs, or virtual machines. It is the wrong choice on machines where the GPU is also running CUDA inference or gaming workloads at the same time, since it consumes VRAM that those applications would otherwise use. Before committing, verify that your driver exposes libcuda.so.1 and that your kernel is 5.6 or later; without kernel 5.6 the swap-deadlock protection that relies on PR_SET_IO_FLUSHER is silently disabled. The last push to the repository was on 2026-07-17.

Frequently asked questions

Official sources

1. c0deJedi/nbd-vram on GitHub
2. Issues
3. License: MIT
4. Project website
5. README