RTNeural: neural inference for real-time audio, without a runtime

An inference engine with no training and no runtime dependency

RTNeural is a lightweight neural network inferencing engine written in C++, designed with real-time systems in mind, and specifically real-time audio processing.

That single sentence sets its boundaries. It is not a training framework and it does not compete with TensorFlow or PyTorch. It takes a network that has already been trained, loads the weights from that network, and runs inference. The value is that the inference path has no dependency on a large runtime, which is what lets it sit inside an audio callback where allocation, locking and unpredictable dispatch are not acceptable.

The layer set is deliberately narrow and matches what appears in signal processing work: Dense, GRU, LSTM, Conv1D, Conv2D, BatchNorm1D and BatchNorm2D are supported, while MaxPooling is the single entry left unchecked in the README's list. Activations cover tanh, ReLU, Sigmoid, SoftMax, ELu and PReLU.

That list is the honest scope of the library. It covers recurrent and convolutional architectures that show up in audio and signal work, and it does not attempt to be a general neural network toolkit.

Supporting material surrounds the library: an API reference hosted at ccrma.stanford.edu, the reference paper on arXiv at 2106.03037, a Discord server, an example plugin repository, a comparison benchmark repository and an experimental extensions repository.

Weights cross the boundary as JSON

The training and inference sides meet at a JSON file. Neural networks are typically trained using Python libraries including Tensorflow or PyTorch, and once trained you export the network weights to a json file that RTNeural can read.

The repository provides an implementation of the export process for a sequential Tensorflow model in python/model_utils.py:

# import dependencies
import tensorflow as tf
from tensorflow import keras
from model_utils import save_model

# create Tensrflow model
model = keras.Sequential()
...

# train model
model.train()

# export model weights
save_model(model, 'model_weights.json')

For PyTorch there is an example script at python/gru_torch.py. The README does not document a general purpose PyTorch exporter beyond that example, so a PyTorch user with an unusual architecture will be writing their own export code.

The consequence for a team is worth stating plainly. The JSON file becomes an interface between two worlds, and it is an interface that nothing checks for you. If the training code changes a layer order and nobody regenerates the export, the C++ side silently runs the old architecture. Keeping the export in the build pipeline, and versioning it, is the part of the work that the library does not do for you.

A dynamic model at run time, or a type at compile time

There are two ways to hold a model, and the README is direct about the trade-off.

The dynamic path parses the JSON at run time and creates the inferencing engine from it:

#include <RTNeural.h>
...
std::ifstream jsonStream("model_weights.json", std::ifstream::binary);
auto model = RTNeural::json_parser::parseJson<double>(jsonStream);

The compile-time path defines the model architecture as a C++ type. If the model architecture is fixed at compile time, the README says it may be preferable to use this API, and that it can significantly improve performance:

// define model type
RTNeural::ModelT<double, 8, 1,
    RTNeural::DenseT<double, 8, 8>,
    RTNeural::TanhActivationT<double, 8>,
    RTNeural::DenseT<double, 8, 1>
> modelT;

The layers become template arguments, so the network shape is part of the type. That is why the static version can be faster: there is no parsing step, no dynamic dispatch over layer objects, and the compiler can see the whole graph.

The cost of that speed is a rebuild whenever the architecture changes. A static model with eight inputs and a fixed layer stack is a different type from one with ten inputs, so an experiment that changes the shape means recompiling. Teams that iterate on architecture during development usually run the dynamic path and switch to the static path once the shape is settled.

Stateful models need an explicit reset before inference begins, which the README calls out as a recommendation. The example calls model->reset() on the dynamic model and modelT.reset() on the static one, and the reason is that a recurrent layer carries hidden state between calls, so a fresh stream or a new buffer needs the state cleared deliberately.

PyTorch layers load into a static model with torch_helpers

The compile-time API and the PyTorch path are connected by a helper namespace. The RTNeural namespace RTNeural::torch_helpers provides helper functions for loading layers exported from PyTorch, and the README shows loading a single layer into a static model:

// load model weights from json
std::ifstream jsonStream("model_weights.json", std::ifstream::binary);
nlohmann::json modelJson;
jsonStream >> modelJson;

// load a layer from a static model
RTNeural::ModelT<float, 1, 1, RTNeural::DenseT<float, 1, 1>> model;
RTNeural::torch_helpers::loadDense(modelJson, "name_of_layer.", model.get<0>());

Two things are visible there. The JSON is read into an nlohmann::json object, so the project depends on that JSON library for this path, and a layer is loaded by name into a specific position of the model, which means the Python exporter and the C++ template have to agree on layer names. The example passes a trailing dot in the layer name, which is a detail that will bite anyone writing their own exporter.

More examples live in the examples/torch directory, and the wider examples tree includes hello_rtneural, rtneural_dynamic_model, rtneural_static_model, conv1d_stateless_example and custom_layer_model, which tells you that custom layers are an expected extension point rather than an accident.

Three backends, and a licence caveat attached to the choice

RTNeural supports three backends: Eigen, xsimd, or the C++ STL. You choose one with a CMake flag, and the README states that by default the Eigen backend will be used:

set(RTNEURAL_XSIMD ON CACHE BOOL "Use RTNeural with this backend" FORCE)
add_subdirectory(modules/RTNeural)

The flags are -DRTNEURAL_EIGEN=ON, -DRTNEURAL_XSIMD=ON and -DRTNEURAL_STL=ON.

The performance guidance in the README is nuanced and worth repeating accurately. The Eigen backend typically has the best performance for larger networks, while smaller networks may perform better with XSIMD. The README then advises you to measure the performance of your network with all the backends available on your target platform, and points to the benchmark results published through the Bench workflow. So the guidance is not a table to memorise but an instruction to run your own measurement, which is the only honest advice for a library whose workloads range from a single dense layer to a convolutional stack.

There is a licensing consequence too. The README states that you must abide by the licensing rules of whichever backend library you choose. The library itself is BSD-3-Clause, but shipping an Eigen build means shipping under EIGEN's terms as well, and that decision belongs to whoever owns the distribution.

Building, testing and the shape of the repository

RTNeural is built with CMake, and the README says the easiest way to link is to include it as a submodule:

...
add_subdirectory(RTNeural)
target_link_libraries(MyCMakeProject LINK_PUBLIC RTNeural)

Projects that do not use CMake are pointed to separate instructions further down the README.

The repository carries its own quality apparatus. Four GitHub Actions workflows are badged at the top: Tests, Bench, Examples and RADSan, plus a codecov badge for coverage on the main branch. Running the examples in CI is a meaningful signal for a library whose value depends on the examples actually compiling and running against the current headers.

The top level of the repository shows the split between library, tooling and fixtures: RTNeural/ holds the headers, examples/ the sample projects, tests/ the test suite, bench/ the benchmarks, python/ the export helpers, modules/ for the backend modules, models/ for model files, test_data/ for fixtures, cmake/ for CMake helpers, and doxygen/ for the documentation build. The .clang-format and .clang-tidy files at the root set the style rules, and .gitmodules is present for the submodule setup the README recommends.

The repository is not archived and the last push was on 2026-08-20, though it publishes no GitHub releases. For academic use, the README asks that the library be cited as the arXiv preprint 2106.03037 from 2021.

Editorial conclusion

RTNeural fits audio plugin developers and other real-time C++ code that must run a trained network inside a hard deadline, and it fits people who are willing to export their model to JSON and keep that export in version control. It is the wrong choice for a training workflow, for a service that can afford a general purpose runtime, or for architectures outside the listed layers, since MaxPooling is the one entry still unchecked in the supported layer list. Before adopting it, measure your own network on the backends you can ship, because the README states that Eigen tends to win on larger networks and xsimd on smaller ones, and keep the JSON export step in your pipeline so the C++ side and the training side cannot drift apart.

Frequently asked questions

Official sources

1. Issues
2. jatinchowdhury18/RTNeural on GitHub
3. License: BSD-3-Clause
4. README