MuScriptor: Multi-Instrument Transcription from Audio to MIDI and Sheet Music
A multi-instrument music transcription model developed by Kyutai and Mirelo.
At a glance
- What is it?
- MuScriptor is Kyutai and Mirelo's transformer-based transcription model that turns recordings into quantized MIDI and engraved scores. The code is MIT, the weights are not, and the accuracy depends heavily on whether your recording has a steady tempo.
- Who is it for?
- MuScriptor fits engineers and music technologists who want a self-hosted transcription pipeline with both MIDI and notated output, and who can accept the non-commercial CC BY-NC 4.0 weights and a MuseScore dependency. It is the wrong tool for commercial products that need permissive model licensing, and for rubato or heavily improvised recordings where grid quantization breaks down.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 15 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 19, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What MuScriptor actually solves, and for whom
Transcription tools have traditionally been either single-instrument or locked behind a hosted API. MuScriptor targets the middle ground: a recording with several instruments at once, transcribed into MIDI and into notation, running on your own machine. The README describes it as a multi-instrument music transcription model that "turns a recording into MIDI and into sheet music" and claims it is the most accurate open-source transcription model, a claim the paper is meant to back up.
The audience is narrower than the phrase suggests. Because the weights are CC BY-NC 4.0, this is a tool for research, personal projects, and internal experimentation, not for a shipping commercial product. It is also not a real-time tool: the CLI works on files, and the web UI is a server you host. If you want to transcribe a live stream as it plays, nothing in the README suggests that is the intended path.
A decoder-only transformer with three weight sizes
The architecture is stated plainly: a transformer decoder only. There is no encoder branch and no separate instrument-classification stage described in the README. The model consumes audio and emits a token sequence that the surrounding code turns into MIDI events, and from there into MusicXML when sheet output is requested.
Three variants are published under the MuScriptor HuggingFace organization. small is 103M parameters with 14 layers and dimension 768; medium is 307M with 24 layers and dimension 1024; large is 1.4B with 48 layers and dimension 1536. medium is the default. The README positions small as the practical choice on CPU-only machines, medium as the speed/accuracy trade-off, and large as the most accurate but one that "really wants a GPU". On Apple Silicon the model runs on Metal (MPS) automatically.
Anywhere a model is selected, whether in load_model(), the CLI's --model, or serve --model, you pass the bare size keyword and the weights are downloaded and cached on first use. That is a deliberate simplification, and it means you cannot point the loader at an arbitrary local checkpoint path without reading the source.
Installing MuScriptor and running a first transcription
The README requires HuggingFace authentication before anything runs. You accept the model license on the model page for small, medium or large, and access is granted automatically. Then authenticate on the machine, either interactively or by exporting a token.
uvx hf auth loginAlternatively, create a token at huggingface.co/settings/tokens and set it as an environment variable. The weights are then downloaded and cached locally on first use.
export HF_TOKEN=hf_...With authentication in place you can run MuScriptor without cloning the repository. The base command differs by platform, and the README gives a table. Linux and Apple Silicon macOS use the plain form; Windows needs an explicit CUDA backend because the default PyTorch backend there is CPU; Intel Macs need a pinned Python because PyTorch stopped shipping x86_64 wheels after torch 2.2.2.
uvx muscriptor serveuvx --torch-backend=cu128 muscriptor serveuvx --python 3.12 muscriptor serveserve gives you the same UI as the hosted version at muscriptor.kyutai.org, with a different look. For a single file, the CLI is shorter. Point it at a WAV and it writes the transcription.
uvx muscriptor transcribe path/to/audio_file.wavIf you install with pip or uv instead of uvx, use Python 3.10 to 3.12 on Intel Macs. The package is on PyPI, so `uv add muscriptor` or `pip install muscriptor` both work.
Sheet music output needs MuseScore 4 and a steady tempo
The interesting feature is not the MIDI, it is the notation. Passing --format sheets engraves the transcription as readable notation instead of writing a single MIDI file.
muscriptor transcribe audio.wav --format sheets --output score/The README shows the resulting directory: a quantized score.mid, a score.musicxml, a full_score.pdf with every instrument on one system, then one PDF per instrument, plus a tablature PDF for fretted instruments. Drum kits get their own PDF and no tab. This is a genuinely useful layout for anyone who wants to hand a part to a player rather than open a piano roll.
The dependency is real. Sheet music output needs MuseScore 4 or newer installed separately, with downloads at musescore.org/en/download. If it lives somewhere unusual, set $MUSCRIPTOR_MUSESCORE. Without MuseScore, everything except that download still works, so the failure is contained rather than fatal.
The tempo constraint is the part worth reading twice. The README states it works best with a steady tempo, such as playing with a metronome, because that allows quantization of the notes onto a grid. Rubato recordings "will work significantly worse". That is not a bug report, it is a design boundary, and it rules out a large share of expressive solo piano repertoire.
Where MuScriptor is the wrong choice
Two limitations stand out. The first is licensing. The repository code is MIT, but the weights on HuggingFace are CC BY-NC 4.0, which the README glosses as non-commercial use. A tool whose output quality depends entirely on those weights is effectively non-commercial regardless of what the LICENSE file in the repo says. Anyone evaluating this for a product should treat the model license, not the code license, as the binding constraint.
The second is the quantization assumption. Because the pipeline snaps notes to a grid, anything without a stable pulse degrades. Free-tempo performances, spoken-word-adjacent material, and recordings with heavy rubato are the cases the README itself flags. A tool built around beat tracking rather than grid quantization would behave differently here, but MuScriptor does not offer that mode.
There is also a deployment cost the README does not dwell on. The Dockerfile installs fluidsynth for MIDI auralization, then downloads a MuseScore AppImage and extracts it, along with five system libraries the AppImage expects. That is a heavier image than a pure Python transcription service, and it exists because the notation and playback features are first-class rather than optional add-ons.
How MuScriptor differs from Basic Pitch
The closest well-known open alternative in this space is Spotify's Basic Pitch, which is also a polyphonic audio-to-MIDI model with an open release. The difference in approach matters more than the difference in accuracy claims.
Basic Pitch is oriented around note event detection and MIDI output, with instrument-agnostic pitch estimation as the core task. MuScriptor is built as a decoder-only transformer language model over music tokens, and its sheet music path is a documented first-class output: MusicXML, per-instrument PDFs, tablature for fretted instruments, and a full score. It also ships a hosted web UI and a self-hostable equivalent, plus a CLI and a Python API, rather than a library alone.
The trade-off runs the other way on licensing and weight. Basic Pitch's model release is permissive; MuScriptor's weights are CC BY-NC 4.0. If you need to ship commercially, that single fact decides the comparison before any accuracy discussion begins.
Upgrading, maintenance and licence boundaries
The last push to the repository was on 2026-09-04, and the most recent release is v0.3.0 from 2026-08-05, following v0.2.2 and v0.2.1 in July 2026. The project is not archived. The pyproject.toml marks the development status as Beta, which matches the release cadence: three releases in roughly two months, with version numbers still in the 0.x range.
Upgrade cost is dominated by the PyTorch dependency matrix rather than by MuScriptor's own API. The pyproject.toml carries two branches for torch and numpy, one for Intel macOS and one for everything else, with a comment explaining that uv would otherwise unify both branches on the old pin instead of forking the resolution. If you build your own environment rather than using uvx, you inherit that problem. The Intel Mac branch caps torch below 2.3 and numpy below 2, which in turn caps Python at 3.12.
On licensing, the split is worth stating precisely. The repository code is MIT. The model weights are CC BY-NC 4.0. The MuseScore General SoundFont downloaded for playback is distributed under its own MIT license. This is not legal advice, but the practical reading is that the MIT grant covers the Python package, not the numbers you actually run inference with.
Editorial conclusion
MuScriptor fits engineers and music technologists who want a self-hosted transcription pipeline with both MIDI and notated output, and who can accept the non-commercial CC BY-NC 4.0 weights and a MuseScore dependency. It is the wrong tool for commercial products that need permissive model licensing, and for rubato or heavily improvised recordings where grid quantization breaks down. Verify first that your audio has a steady tempo, that your hardware matches the model size you pick, and that you have accepted the license on the exact HuggingFace repo for small, medium or large before you write any integration code.
Frequently asked questions
What is a transcriptor, and how does MuScriptor relate to that term?
MuScriptor is a multi-instrument music transcription model from Kyutai and Mirelo. It converts a recording into MIDI and, with the right options, into engraved sheet music.
How do I install MuScriptor on a Mac?
On Apple Silicon, run uvx muscriptor serve after authenticating with HuggingFace, and the model runs on Metal automatically. On Intel Macs, PyTorch stopped shipping x86_64 wheels after torch 2.2.2, so you must pin the Python version with uvx --python 3.12 muscriptor serve.
Does MuScriptor need a HuggingFace account?
Yes. You must accept the CC BY-NC 4.0 license on the model page for small, medium or large, then authenticate with uvx hf auth login or set HF_TOKEN. The weights are downloaded and cached on first use.
Can MuScriptor produce sheet music instead of MIDI?
Yes, with muscriptor transcribe audio.wav --format sheets --output score/. This writes a quantized score.mid, a score.musicxml, a full score PDF, and one PDF per instrument, plus tablature for fretted instruments. It requires MuseScore 4 or newer installed separately.
Community notes