NVIDIA RMM and the interface under every GPU allocation in RAPIDS

Three things RMM promises, and all three are about control

The overview explains the problem RMM exists to solve. Achieving optimal performance in GPU-centric workflows frequently requires customizing how host and device memory are allocated, and the two examples given are specific. Using pinned host memory for asynchronous host to device memory transfers, and using a device memory pool sub-allocator to reduce the cost of dynamic device memory allocation.

Both examples point the same way. The default CUDA allocation path is general purpose, and a workload with a known shape can do better with something tuned. A pool amortises the cost of a fresh allocation across many smaller ones. Pinned host memory lets a copy overlap with compute instead of waiting for the transfer to complete.

The stated goal is then threefold: a common interface that allows customizing memory allocation on device and host, a collection of implementations of that interface, and a collection of data structures that use the interface for memory allocation.

That third item is the part that matters for an adopter. RMM is not only a set of allocators you can call. It is the allocator that the containers and buffers of the wider RAPIDS stack are built on, which means adopting RMM changes how those structures behave under allocation pressure. The readme lists the available device resources and the device data structures as separate sections, and the documentation site carries the reference documentation, tutorials and topic guides for both the Python and the C++ API.

The design walkthrough is on the NVIDIA Developer Blog, in a post titled Fast, Flexible Allocation for NVIDIA CUDA with RMM, which is where a reader who wants the reasoning behind the interface rather than the API will find it.

The package suffix is a decision, not a detail

The installation section makes a point that is easy to skim and expensive to get wrong. Stable releases of `librmm` and `rmm` are available on PyPI, and you should match the package suffix to the major CUDA version supported by your installed driver.

The commands are straightforward:

# CUDA 13
pip install librmm-cu13
pip install rmm-cu13

# CUDA 12
pip install librmm-cu12
pip install rmm-cu12

There are two packages rather than one, and the split is worth understanding. `librmm` is the C++ library, the shared object that a compiled extension links against. `rmm` is the Python package, the bindings and the Python side of the API. They are versioned and installed separately because the C++ library can be consumed by projects that never touch Python at all, and because a Python-only install still has to load a compiled library at run time.

The conda route mirrors the pip one, with the `rapidsai` channel for stable and `rapidsai-nightly` for development builds:

# Stable
conda install -c rapidsai -c conda-forge librmm
conda install -c rapidsai -c conda-forge rmm

# Nightly
conda install -c rapidsai-nightly -c conda-forge librmm
conda install -c rapidsai-nightly -c conda-forge rmm

The nightly pip installs need an extra index, and the command is written with a line continuation to keep the flag and the package name together:

# CUDA 13
pip install --pre \
  --extra-index-url=https://pypi.anaconda.org/rapidsai-wheels-nightly/simple \
  librmm-cu13

The note at the top of the readme also matters if you are reading a tagged copy: for the latest stable README you should be on the `main` branch, since release branches carry their own readme.

Building from source needs a specific toolchain and a conda env

The source build path is documented in detail, and the requirements are exact enough to be worth reading before you start.

The compiler requirements are `gcc` version 13.3 or newer, `nvcc` version 12.9 or newer, and `cmake` version 4.0 or newer. The CUDA and GPU runtime requirements are CUDA 12.2 or newer, and the GPU support statement is a hard boundary: RMM is tested and supported only on Volta architecture and newer, meaning Compute Capability 7.0 and up. The Python side needs `rapids-build-backend`, available from PyPI or the `rapidsai` conda channel, plus `scikit-build-core`, `cuda-python` and `cython`.

The build itself starts with the conda environment, created from a file whose name embeds the CUDA version and the architecture:

# create the conda environment (assuming in base `rmm` directory)
$ conda env create --name rmm_dev --file conda/environments/all_cuda-133_arch-$(uname -m).yaml
# activate the environment
$ conda activate rmm_dev

Then `librmm` is configured and built with CMake and make. The comments in the readme note that CMake depends on the `nvcc` executable being on the path or defined in the `CUDACXX` environment variable:

$ mkdir build                                       # make a build directory
$ cd build                                          # enter the build directory
$ cmake .. -DCMAKE_INSTALL_PREFIX=/install/path     # configure cmake ... use $CONDA_PREFIX if you're using a conda environment
$ make -j                                           # compile the library librmm.so ... '-j' will start a parallel job using the number of physical cores available on your system
$ make install                                      # install the library librmm.so to '/install/path'

There is a `build.sh` alternative at the repository root that does the same thing, and it takes the target names as arguments. It can build `librmm` alone, `rmm` alone, or both, and with or without installing:

$ ./build.sh -n librmm                              # Build librmm without installing
$ ./build.sh -n rmm                                 # Build rmm without installing
$ ./build.sh -n librmm rmm                          # Build librmm and rmm without installing
$ ./build.sh librmm rmm                             # Build and install librmm and rmm

The Python package is then built and tested from the `python` folder, with `pytest -v` at the end.

ABI versioning is what keeps the RAPIDS components compatible

There is a section on ABI versioning that reads like a footnote and is really the compatibility contract for the whole stack.

RMM symbols are placed in an inline namespace derived from the RMM major and minor version. The public API remains available through the `rmm::` namespace, while static RMM libraries built for different ABI versions can coexist in one process. Process-global state is shared by RMM copies with the same ABI version and kept separate across different ABI versions.

This is a deliberate design for a component that many separately built libraries link against. If two static copies of RMM end up in one process, and they disagree about process-global state, the failure mode is the kind that produces wrong results rather than a link error. The inline namespace makes the version part of the symbol name, so the linker separates them, and the explicit rule about process-global state answers the question of what happens when two versions coexist.

The practical consequence for a user is a versioning constraint that is stricter than it looks. RAPIDS libraries are released together on a cadence, and a mismatch between the RMM version one component was built against and the version another expects is the kind of problem that shows up at link time or as an ABI error rather than as a clean message.

The release numbering reflects that discipline. The three most recent releases are v26.06.00 on 2026-06-03, v26.08.00 on 2026-08-05, and v26.10.00 on 2026-10-07, so the version carries the year and the month rather than a semver patch counter. The last push was on 2026-10-19, and the repository is not archived.

Third-party dependencies are cached rather than rebuilt

The build pulls in dependencies such as CCCL, GoogleTest and GoogleBenchmark, and it uses CPM.cmake to handle them.

The behaviour is what you would expect from a C++ dependency manager: if CMake finds an appropriate version on your system it uses it, and you can point it at a local installation by setting `CMAKE_PREFIX_PATH`. Otherwise the dependencies are downloaded as part of the build. In general the readme says you will not have to worry about it.

The one piece of advice worth acting on is about repetition. If you frequently start new builds from scratch, consider setting the environment variable `CPM_SOURCE_CACHE` to an external download directory to avoid repeated downloads of the third-party dependencies. This is a small operational point that pays off immediately if you work on RMM itself or on a downstream project that vendors it, because a full dependency download on every clean build is a tax paid every time.

The repository layout supports the picture of a project with a substantial build system around it. There is a `cmake/` directory, a `conda/` directory, a `python/` directory for the Python package, a `cpp/` directory for the C++ library, `ci/` for the continuous integration setup, `docs/` for the documentation sources, and `dependencies.yaml` as the dependency manifest. There is also `print_env.sh` for environment reporting, a `VERSION` file, a `RAPIDS_BRANCH` file, an `AGENTS.md` and a `CHANGELOG.md`.

Where RMM is the wrong layer to solve the problem

There are two clear cases where reaching for RMM is the wrong move, and one where it is a partial answer.

The first is a workload that allocates a few buffers and frees them. RMM's value is in tuning allocation behaviour for a hot path, and the machinery of an interface, a set of backends and a set of data structures costs you a dependency, a build step and an ABI compatibility constraint. If you allocate once and keep the pointer, plain CUDA allocation is simpler and there is nothing to tune.

The second is hardware. The statement that RMM is tested and supported only on Volta architecture and newer, Compute Capability 7.0 and up, is a support statement, not a performance one. On older hardware the library may or may not work, but the project is not claiming it does, and a component that RAPIDS libraries depend on is not somewhere to find out.

The partial answer case is the most interesting. RMM is a memory management library, and it does not address the other things that dominate GPU application performance. It does not schedule work, it does not overlap kernels, it does not manage a CUDA stream, and it does not tell you whether your algorithm is the bottleneck. The readme's own framing is narrow and worth repeating: the goal is a common interface for customizing memory allocation on device and host, plus implementations and data structures that use it. If the problem you are trying to solve is that a data frame operation is slow, the allocator is one candidate cause, and the fastest way to rule it in or out is to look at where the time actually goes before adding an allocator abstraction.

There is also a dependency direction to consider. RMM lives in the RAPIDS ecosystem, and the RAPIDS release train moves together. Depending on RMM directly ties you to that cadence even if you never import another RAPIDS library, which is a cost for a general purpose allocator.

Against raw CUDA allocation, the difference is policy in one place

The realistic alternative is to call the CUDA allocation APIs directly, and the difference in approach is about where policy lives.

With raw CUDA, allocation policy is distributed across your code. Every call site that needs a pool has to know about the pool, every call site that wants pinned memory has to remember to ask for it, and every call site has to be consistent about lifetime. That works fine for a small library and becomes a problem when several libraries in one process each have their own idea of how memory should be handled.

With RMM, the policy is a single object that the rest of the code uses. The interface is common, so the same call works for host and device, and the backends are swappable, so a pool allocator and a plain allocator are the same shape to the caller. The data structures that use the interface then inherit the behaviour, which is why RMM's third goal matters: the containers do not have to be told about the allocator because they already know about RMM.

The cost is that this is a real abstraction with a real dependency. You are choosing an allocator layer for the whole process, and you are accepting its ABI rules, its supported hardware floor and its release cadence. A project that needs one special allocation pattern is better off writing the one allocator it needs.

The pragmatic path is to look at what the RAPIDS stack you already use does, because if you use cuDF or cuML you are already using RMM underneath, whether you asked for it or not. In that case the question is not whether to adopt RMM but which backend to configure and what the default means for your memory profile. The readme's sections on available device resources and device data structures are where that question is answered.

Editorial conclusion

RMM fits a data science team already using RAPIDS, or writing a CUDA library that needs allocation behaviour consistent with RAPIDS rather than raw cudaMalloc calls. It is the wrong tool for a project that allocates a handful of buffers and frees them once, and it is the wrong tool on hardware older than Volta, which the project states it does not support. Before adopting, verify three things: that the package suffix you install matches the major CUDA version your driver supports, since the packages are published per CUDA major, that your GPU is Compute Capability 7.0 or newer, and which allocator your workload should use, because the default and the pool behave differently under memory pressure. The code is Apache-2.0, the current release is v26.10.00 from 2026-10-07, and the last push was on 2026-10-19.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. rapidsai/rmm on GitHub
4. README
5. Releases