tiny-cuda-nn: A Self-Contained CUDA Framework for Fused MLPs and Hash Encodings

The problem tiny-cuda-nn is built to solve

Most deep learning frameworks are organized around large models trained over long horizons. tiny-cuda-nn is organized around the opposite case: small networks that must be evaluated millions of times per second inside a rendering or simulation loop. The README frames it as a small, self-contained framework for training and querying neural networks, and the two headline components are a fully fused multi-layer perceptron and a multiresolution hash encoding. Both are aimed at the same workload. A NeRF-style scene representation, for instance, needs a coordinate network that maps positions and directions to color and density, and it needs that mapping to run fast enough to keep a real-time renderer fed.

The audience is therefore narrow and specific. You are writing CUDA, or you are willing to call into a library that does. The README's usage example is C++: it includes tiny-cuda-nn/common.h, builds a nlohmann::json config, and calls create_from_config. There is a PyTorch binding, and the documentation shows tcnn.NetworkWithInputEncoding, but the README is explicit that JIT fusion gives a lower speed-up through the Python path, particularly during training, because the JIT compiler cannot see the whole compute graph. That sentence tells you where the project's center of gravity sits. If your work lives entirely in Python, you are using the secondary interface.

Fully fused MLP and hash encoding as the two core mechanisms

The fully fused MLP is the piece the README calls out first, linking to a diagram and a technical paper. The term fused refers to how the network's layers are evaluated: rather than dispatching a separate kernel per layer and materializing intermediate activations in global memory, the computation is arranged so the network runs as one unit. That is the mechanism behind the performance chart in the README, which compares fully fused networks against TensorFlow v2.5.0 with XLA on 64 and 128 neuron wide MLPs on an RTX 3090, generated by benchmarks/bench_ours.cu and benchmarks/bench_tensorflow.py using data/config_oneblob.json. The benchmark is small and specific: two layer widths, one GPU. Treat it as a demonstration of the design intent rather than a general claim.

The second mechanism is the multiresolution hash encoding, also linked to its own paper. In the config example it appears as an encoding block with otype HashGrid and parameters n_levels, n_features_per_level, log2_hashmap_size, base_resolution, and per_level_scale. The structure implied by those keys is a set of grids at increasing resolutions, each backed by a hash table of bounded size, with per_level_scale controlling how resolution grows between levels and log2_hashmap_size capping the table. This is what lets a small MLP represent high-frequency spatial detail without a proportional increase in network width. The encoding absorbs the detail; the MLP stays shallow.

The config also exposes loss and optimizer as sibling blocks, here L2 and Adam with a learning rate of 1e-3. The README mentions support for various other input encodings, losses, and optimizers without enumerating them, so the set is wider than the example but not fully documented in the supplied material.

Getting a model running: the C++ path and the JSON config

The README gives a complete training and inference sketch. You include tiny-cuda-nn/common.h, declare a config as nlohmann::json with loss, optimizer, encoding, and network entries, then call tcnn::create_from_config(n_input_dims, n_output_dims, config). The returned object has a trainer member and a network member. Training is a loop calling model.trainer->training_step(training_batch_inputs, training_batch_targets, &loss), and inference is model.network->inference(inference_inputs, inference_outputs). Inputs and outputs are GPUMatrix<float> objects sized by dimension and batch size.

Two constraints are stated in the code comments rather than in prose, which is easy to miss. The first: batch_size must be a multiple of tcnn::BATCH_SIZE_GRANULARITY. The second appears in the manual JIT example, where threads must be a multiple of 32 for neural networks to work, and all 32 threads of the warp must be active at the point where the model is called. These are hard requirements, not tuning suggestions, and they shape how you structure the surrounding kernel.

For JIT fusion there are two routes. The automatic one is a single call, model->set_jit_fusion(tcnn::supports_jit_fusion()), or in Python model.jit_fusion = tcnn.supports_jit_fusion(). The README states that if JIT compilation errors, a warning is emitted and JIT mode is turned off automatically, falling back to the 1.X code path. Your code keeps running. The manual route uses tiny-cuda-nn/rtc_kernel.h and tcnn::CudaRtcKernel, where you write your kernel as a string, prepend model->generate_device_function("model_fun") via fmt::format, and launch with fused_kernel.launch(blocks, threads, shmem_size, stream, ...). The README points to Instant NGP's src/testbed_nerf.cu and include/neural-graphics-primitives/fused_kernels/render_nerf.cuh as reference implementations, and states that Instant NGP achieves a 5x speedup by fusing the entire NeRF ray marcher into a single kernel. That figure is the project's own, for that specific integration.

Where JIT fusion backfires, according to the README

The most useful part of the documentation is the section that argues against its own feature. JIT fusion is described as almost always recommended for a 1.5x to 2.5x performance boost depending on model and GPU, with larger speedups on newer GPUs. Then the exceptions. If your model has very large hash grids, roughly 20 million or more parameters, or MLPs with layer sizes larger than 128 neurons, JIT fusion can slow down training. On an RTX 3000 series GPU or earlier, the same applies. Inference is rarely affected, but rarely is not never.

The recommended workaround is to enable JIT fusion separately for training and inference and measure. This is a genuine design trade-off rather than a bug, and the README says so plainly, inviting users to open an issue if they hit a slowdown outside these cases. The pattern to notice is that the feature pays off when the model is small and the GPU is recent, which is exactly the regime the library targets. Step outside that regime and the fusion overhead stops being amortized.

A second limitation is structural. The Python bindings cannot reach the same speed-up as C++, because the JIT compiler does not have access to the whole compute graph. If you are in Python and expecting the headline numbers, the README tells you not to.

How it differs from training the same network in PyTorch

The obvious alternative for a small MLP is PyTorch, and the README's own benchmark uses TensorFlow with XLA as the comparison point, which makes the contrast concrete: a general-purpose tensor framework compiles and dispatches operations through a graph, while tiny-cuda-nn generates a kernel specialized to one network configuration. The trade is flexibility for latency. In PyTorch you get autograd, a large optimizer library, distributed training, and a debugging experience that does not involve reading generated CUDA. In tiny-cuda-nn you get a network that can be inlined into a ray marcher.

The difference is sharpest at the manual JIT fusion boundary. PyTorch cannot express fusing a NeRF ray marcher and its coordinate network into one kernel, because the ray marcher is not a tensor operation. tiny-cuda-nn's generate_device_function plus CudaRtcKernel exists precisely to make that possible. If your problem does not have that shape, if the network is the whole program rather than a component inside a larger kernel, the case for tiny-cuda-nn weakens considerably and a standard framework will be easier to live with.

A middle path worth noting: the PyTorch binding lets you keep the training loop in Python and still use the hash encoding and fused MLP, accepting the reduced JIT benefit the README describes.

Version 2.0, maintenance, and the licence question

The release history is uneven. v1.5 arrived in April 2022, v1.6 in December 2022, and v2.0 in July 2025, with the repository's last push in April 2026. That is a long gap between minor releases and then a major one that introduced JIT fusion as an optional layer on top of the existing code path. The fallback behavior matters here: when JIT compilation fails, the README states the library reverts to the 1.X path automatically. That design keeps the 2.0 upgrade low-risk for existing users, since the new code is additive rather than a replacement.

Upgrade cost is mostly a measurement exercise. The API surface shown in the README is small, and set_jit_fusion is a single call, but the README's own warnings mean you cannot assume the upgrade is free on older hardware or on large models. The honest position is that you should benchmark before and after on your specific configuration, because the documentation gives no general rule beyond the parameter thresholds it names.

The licence is the unresolved item. The repository metadata reports NOASSERTION, which means GitHub could not match the licence file to a known identifier. The README does not state a licence. If you are planning to ship tiny-cuda-nn inside a product, read the LICENSE file in the repository directly and get your own advice; nothing in the supplied material tells you what terms apply.

Editorial conclusion

Adopt tiny-cuda-nn if you are writing CUDA-side code that needs a small MLP or hash-grid encoding evaluated inside your own kernels, particularly if you can use the manual JIT fusion path to fold the model into a larger kernel. Do not adopt it if your model is a conventional deep network, if you need a stable Python-first training loop, or if you are targeting hardware older than the RTX 3000 series without measuring first, since the README states JIT fusion can slow training on those GPUs. Before committing, verify three things on your own hardware: whether tcnn::supports_jit_fusion() returns true, whether enabling jit_fusion separately for training and inference helps or hurts, and whether your batch size is a multiple of tcnn::BATCH_SIZE_GRANULARITY, because the training example requires it.

Official sources

1. Issues
2. NVlabs/tiny-cuda-nn on GitHub
3. README
4. Releases