Magenta RealTime 2: An Open-Weights Model That Streams Music on Apple Silicon
Magenta RealTime 2: An Open-Weights Live Music Model
At a glance
- What is it?
- MRT2 ships a JAX and MLX inference library, a C++ engine, and AUv3 and standalone examples for live music generation. The catch is the hardware table: only a subset of Apple Silicon chips can stream the larger model in real time.
- Who is it for?
- Adopt MRT2 if you are building on Apple Silicon and want an open-weights model with a C++ engine and ready-made AUv3 and standalone examples to start from. Skip it if your target is Windows, Linux, or an NVIDIA GPU and you need real-time streaming, because the repository's own device table limits streaming to Apple Silicon and the README notes that only offline inference is available elsewhere.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 3 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap MRT2 fills for live music generation
Most open music generation models are built for offline rendering. You give them a prompt, wait, and receive a finished clip. That workflow does not fit a DAW session or a performance, where audio has to keep arriving while the performer plays. MRT2 is aimed squarely at that second case. The README describes it as an open-weights model for real-time music generation, and the repository is organised around streaming: a Python inference library, a C++ inference engine for efficient streaming audio generation on Apple Silicon MacBooks, and a set of example applications built on that engine. The audience is developers who want to embed music generation into a plugin, a standalone app, or something new, rather than researchers producing clips in a notebook. The README also states that future updates will support supervised fine-tuning, which tells you the current scope is inference and deployment, not training.
Two model sizes and the hardware table that decides your product
The repository offers two checkpoints. mrt2_small has 230M parameters and the README claims it runs in real time on any Apple Silicon Mac, including Air models. mrt2_base has 2.4B parameters, produces higher quality, and requires a Pro Max chip for real-time streaming. The device table is the most consequential part of the documentation because it is a compatibility matrix, not a suggestion. M5 Max, M3 Max, M2 Max, and M4 Pro support both models. M2 Pro, M1 Pro, M4 Air, M3 Air, and M1 Air support only mrt2_small. If your application needs the base model, the supported machine list is short and skews toward Max-class silicon. Both models can also run offline on any Apple Silicon Mac or NVIDIA GPU through the Python library, according to the README, but offline is a different product category. A plugin that generates a loop after a pause is not the same as one that generates while the user plays.
The inference stack: JAX, MLX, and a C++ engine
MRT2 is not a single runtime. The Python library, installed as magenta-rt, offers JAX and MLX backends. The C++ library, magentart::core, exists for efficient streaming on Apple Silicon. The example applications sit on top of the C++ engine rather than the Python path, which suggests the intended production route is the compiled engine and the Python package is for prototyping and offline work. The model architecture is split across components the README names explicitly: a style model and a codec model, identified as MusicCoCa and SpectroStream, plus the streaming model you select. The CLI reflects that split. mrt models init downloads the style and codec resources, and mrt models download fetches the streaming model. That separation matters operationally: the shared resources are downloaded once, while the streaming checkpoint is the large, swappable piece. The repository layout also includes examples/mrt2/auv3 for an all-in-one AUv3 plugin, examples/mrt2/standalone for a macOS app, examples/jam for note control, and examples/collider for prompt space, plus a notebooks directory for trying the Python API.
Getting it running: the commands in the README
The quickstart targets Apple Silicon and uses uv. The README gives this sequence: install uv with the shell script from astral.sh, create a virtual environment with uv venv --python 3.12, activate it, then run uv pip install "magenta-rt[mlx]". After that, mrt models init downloads MusicCoCa and SpectroStream, mrt models download fetches the streaming model, and generation looks like mrt mlx generate --prompt "disco funk" --duration 4.0 --model=mrt2_base. Swapping to mrt2_small is a flag change. For local development the README says to clone with git clone --recurse-submodules, change into the directory, and install editable with uv pip install -e ".[mlx]". The submodule flag is not optional decoration; the C++ core appears to live in a submodule, so a plain clone will leave you without it. For C++ work the README installs a pinned CMake with uv pip install "cmake<3.28", configures with cmake . -B build, builds the target with cmake --build build --target hello_mrt2 -j10, and runs the resulting binary with a model path, a resources path, a numeric argument, and a --prompt flag. The README does not explain what that numeric argument controls, and the material supplied here does not let me tell you.
Where MRT2 is the wrong tool
The hardware matrix is the first limitation, and it is a hard boundary rather than a soft one. Real-time streaming requires Apple Silicon. There is no Windows path and no Linux path for streaming in the documentation. The README notes that offline inference works on NVIDIA GPUs through the Python library, so a Linux server can generate music with MRT2, but it cannot do so live. For a web product or a cross-platform plugin, that removes the streaming mode entirely. The second limitation is the quality ceiling on smaller machines. If you are on an M2 Pro or an M1 Pro, the base model is off the table for real time, and you are shipping the 230M model whether or not it meets your quality bar. The third is the state of the project: the README says supervised fine-tuning is a future update, so anyone hoping to adapt the model to a specific artist, genre, or instrument cannot do that yet through the documented path. Treat the model as fixed and build around it.
How MRT2 differs from Stable Audio Open and similar offline models
The obvious comparison is Stable Audio Open, which is also an open-weights music generation model, though I am describing it from general knowledge rather than from material supplied here, so treat the contrast as directional. The difference in approach is architectural. Stable Audio Open is designed around offline generation: you submit a prompt and a duration, and you get a file. MRT2 is designed around a streaming loop, with a C++ engine, a device compatibility table, and example applications that are plugins and standalone apps rather than scripts. That is a real engineering difference, not a marketing one. It shows up in what the repository contains: an AUv3 target, a standalone macOS app, and a latency benchmark document. A model built for offline rendering does not need a latency benchmark, because latency is not a constraint. If your use case is batch generation of clips, a streaming-first model is more machinery than you need, and the Apple Silicon requirement is a cost you are paying for a capability you will not use.
Maintenance, licence, and what to verify before you build
The repository is Apache-2.0 and not archived, with a release tagged v2.0.3 in July 2026 and a push in September 2026. The README points to a CLI test workflow badge, so there is at least a command-line test suite in CI, though the supplied material does not describe what it covers. The Apache-2.0 licence covers the code in the repository; the model weights are hosted separately on Hugging Face, and the README links to them without restating terms. That distinction matters for anyone shipping a commercial product, because a permissive code licence does not automatically settle the question of model weights or generated output. I am not giving legal advice here, and you should read the licence files in both places yourself. On upgrade cost, the split between shared resources and the streaming checkpoint is helpful: mrt models init is a one-time download, while mrt models download is the piece that changes when a new model version lands. The pinned cmake<3.28 constraint in the C++ instructions is worth noting, since it suggests the build has not been validated against newer CMake, and that is a constraint you inherit if you build the engine yourself.
Editorial conclusion
Adopt MRT2 if you are building on Apple Silicon and want an open-weights model with a C++ engine and ready-made AUv3 and standalone examples to start from. Skip it if your target is Windows, Linux, or an NVIDIA GPU and you need real-time streaming, because the repository's own device table limits streaming to Apple Silicon and the README notes that only offline inference is available elsewhere. Before committing, run mrt models init and mrt models download, then check the latency benchmark documentation against your exact chip, since the difference between mrt2_small and mrt2_base on an M2 Pro or M1 Pro decides whether your product can stream at all.
Community notes