NVIDIA RAFT: CUDA Primitives for People Who Write the Algorithms
RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.
At a glance
- What is it?
- RAFT is a header-only C++ template library of CUDA-accelerated primitives for machine learning and information retrieval, with optional shared-library and Python layers. It is infrastructure for library authors, not a toolkit for data scientists.
- Who is it for?
- Adopt RAFT if you are building a CUDA library or a product that needs reusable primitives for dense and sparse linear algebra, solvers, statistics and distance computation, and you are willing to take on a CUDA toolchain plus RMM and cuVS as dependencies.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Cuda, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap RAFT fills is maintenance, not raw speed
Most GPU projects end up rewriting the same small set of kernels: a pairwise distance, a reduction, a k-selection, a sparse matrix transpose. Each rewrite is a chance to get stream handling, workspace sizing or allocator behaviour subtly wrong. RAFT's stated goal is to centralise those computations so that a fix or an optimisation lands once and benefits every algorithm built on top. The README frames this as three outcomes: faster algorithm construction, a smaller maintenance burden through reuse, and a single place for core reusable computations. That is a library-author argument, not a user-facing one. If you are the person writing the algorithm, the value is that you stop owning the primitives underneath it. If you are the person calling the algorithm, RAFT is invisible to you, and the README is explicit that data scientists looking for discovery and experimentation should look at the RAPIDS website instead.
Header-only core, optional shared library, thin Python wrappers
RAFT is a C++ header-only template library with an optional shared library. The shared library exists for two reasons given in the README: it can speed up compile times for common template types, and it provides host-accessible runtime APIs that do not require a CUDA compiler to use. That second point is the interesting one. It means a team can consume RAFT functionality from a non-CUDA translation unit, which matters when the GPU code and the application code are built by different toolchains. On top of that sit two Python packages. pylibraft is described as lightweight wrappers around the host-accessible runtime APIs, accepting any object that supports __cuda_array_interface__, with CuPy's ndarray given as the example. raft-dask is a separate concern: a multi-node, multi-GPU communicator infrastructure for building distributed algorithms on the GPU with Dask. Note the asymmetry. The C++ side is the full surface. The Python side is a wrapper over the runtime subset, and the README says the number of algorithms exposed there is continuing to grow, which is a polite way of saying it is incomplete.
mdspan and mdarray as the data contract
RAFT does not invent its own array type for inputs. It accepts mdspan, the multi-dimensional array view, and also ships a corresponding owning mdarray structure that manages allocation in host or device memory. The mdarray is a convenience layer over RMM, and the README gives three constructors: raft::make_device_scalar, raft::make_device_vector and raft::make_device_matrix, each taking a handle plus dimensions. Choosing a non-owning view as the primary interface is a deliberate constraint. The library never assumes it owns your memory, so you can pass a slice of a larger buffer, a view into memory allocated by another framework, or a region of a pooled allocation, without a copy. The cost is that lifetime and layout are your problem. RAFT will not detect that the buffer behind a view was freed, and it will not reallocate for you. For a primitives library that is the correct trade, but it shifts a class of bugs into the caller's code.
device_resources is the object you thread through everything
Most RAFT primitives take a raft::device_resources object. The README describes it as managing resources that are expensive to create: CUDA streams, stream pools, and handles to other CUDA libraries such as cuBLAS and cuSolver. The intended pattern is to construct one and pass it down, rather than letting each call create its own stream. The C++ example in the README shows the full shape of a call sequence. It constructs a handle, then builds views over raw pointers with make_device_matrix_view and make_device_vector_view, calls raft::random::make_blobs to generate clustered input and labels, and then calls cuvs::distance::pairwise_distance with a DistanceType of L2SqrtExpanded to fill an n_samples by n_samples output matrix. Two things stand out. First, the distance computation is not a RAFT function at all; it comes from cuVS, the NVIDIA library the README links to. RAFT is positioned as the layer that manages resources and data layout, while the nearest-neighbour and distance work increasingly lives next door. Second, the example leaves the pointer allocation as an ellipsis comment, which tells you the intended reader already knows how to allocate device memory with RMM.
RMM is not optional in practice
The README states that RAFT relies heavily on the RAPIDS Memory Manager, which eases configuring different allocation strategies globally across libraries that use it. Read that carefully: the value proposition is global configuration. If your application and RAFT both allocate through RMM, one setting controls the pool for both, and you avoid the classic failure where a library allocates through the default CUDA allocator while your application uses a pool, producing unpredictable peak memory. If you are not already using RMM, adopting RAFT means adopting RMM as well, and that is a change to how your whole process allocates, not a local dependency. This is the single largest integration cost in the project, and it is stated in one sentence rather than in a migration guide.
Building it means committing to the CUDA toolchain
The repository points to a dedicated build document, docs/source/build.md, for installation and building instructions, and to docs/source/quick_start.md for getting started. Those are the authoritative sources for exact commands, and the README does not inline them. What the README does establish is the shape of the commitment: a CUDA compiler for the header-only path, and the option of the shared library if compile times for common template types become a problem or if you need the host-accessible runtime APIs from code that cannot include CUDA headers. The releases follow a dated versioning scheme, with v26.08.00, v26.06.00 and v26.04.00 listed at roughly two-month intervals. That cadence is the real upgrade cost. A library that ships every two months will move its API, and code pinned to an older tag will drift from the version that RAPIDS libraries in the same stack expect. Plan to track the cadence or to pin the whole RAPIDS stack together.
Where RAFT is the wrong tool
The README answers this directly: RAFT contains low-level primitives, and is not intended to be used directly by data scientists for discovery and experimentation. Take that at face value. If your task is to cluster a table, run a nearest-neighbour search over an embedding set, or evaluate a model, you want the higher-level RAPIDS libraries, not RAFT. The second limitation is the Python surface. pylibraft wraps the host-accessible runtime APIs, which is a subset of the C++ template surface, and the README says the number of exposed algorithms is still growing. A Python user who needs a primitive that exists only as a template in C++ has no path except writing an extension. The third limitation is the cuVS boundary. The README's own example reaches into cuvs::distance for pairwise distances, which means a reader looking for distance and vector-search primitives should be reading cuVS documentation, not RAFT's, and should expect the two to be versioned together.
The realistic alternative, and the actual difference
The alternative to RAFT for a GPU-accelerated application is not a single library but a habit: writing your own CUDA kernels, or leaning on thrust and CUB for the generic pieces and hand-rolling the rest. The difference in approach is where the abstraction sits. Thrust and CUB give you parallel primitives over iterators and ranges, and they are largely domain-agnostic: a scan is a scan. RAFT gives you domain primitives for machine learning and information retrieval, expressed over multi-dimensional views with a resource handle threaded through, and it assumes an RMM allocator underneath. In practice you would use both, since RAFT itself is built on the CUDA ecosystem rather than replacing it. The choice is whether you want to own the domain-level kernels (pairwise distances, k-selection over a matrix, sparse symmetrisation) or whether you want a library that owns them and updates them on a two-month cadence. Owning them costs you maintenance and buys you control over the exact memory layout and stream behaviour of your hot path.
Licence and the cost of tracking releases
RAFT is Apache-2.0, which permits commercial and closed-source use and includes a patent grant. That is a permissive licence, and it is the same one used across much of the RAPIDS stack, so there is no licence conflict introduced by mixing RAFT with typical application code. The practical obligation is attribution and preserving the licence notice for the portions you redistribute. This is not legal advice; check with counsel if you are redistributing modified sources. The ongoing cost is version alignment. RAFT sits in a stack with RMM and cuVS, and the README's example imports headers from both RAFT and cuVS in the same translation unit. That means a RAFT upgrade is rarely a RAFT-only upgrade. Budget for moving the pinned versions together, and read the release notes for v26.08.00, v26.06.00 and v26.04.00 before assuming a drop-in replacement.
Editorial conclusion
Adopt RAFT if you are building a CUDA library or a product that needs reusable primitives for dense and sparse linear algebra, solvers, statistics and distance computation, and you are willing to take on a CUDA toolchain plus RMM and cuVS as dependencies. Do not adopt it if you want a Python surface for discovery work: the README states plainly that RAFT is not intended to be used directly by data scientists, and pylibraft exposes only the host-accessible runtime APIs, a subset that the README says is still growing. Before committing, verify on your own hardware that the pinned CUDA version, the RMM allocator configuration and the mdspan-based API compile against your existing kernels, and check whether the primitives you need are already in cuVS rather than in RAFT itself.
Community notes