MidiTok: turning MIDI and abc files into token sequences for Transformer models

The gap MidiTok fills between a MIDI file and a model input

A Transformer does not consume notes. It consumes integer sequences. Between a .mid file and a training batch there is an encoding decision that determines what the model can and cannot learn: how time is represented, whether velocity is bucketed, whether instruments are separate streams, whether chords are labelled. MidiTok exists to make that decision a configuration object rather than a codebase you write yourself.

The README states the package can tokenize MIDI and abc files, converting them into token sequences ready to be fed to models such as Transformer, for any generation, transcription or MIR task. The audience is therefore narrow and specific: researchers and engineers training sequence models on symbolic music, and people reproducing published music generation work. The README notes the package was introduced at the ISMIR 2021 late-breaking demos, which tells you it originated in the music information retrieval community rather than in general NLP tooling.

The scope is deliberately one-directional in emphasis. MidiTok is a tokenizer library, not a model library and not a dataset. The README points to a Colab notebook for generating music with Hugging Face models, but the model itself comes from elsewhere.

Ten tokenizations behind one TokenizerConfig

The central design claim in the README is that MidiTok features most known music tokenizations and is built around the idea that they all share common parameters and methods. The list is REMI, REMI+, MIDI-Like, TSD, Structured, CPWord, Octuple, MuMIDI, MMM and PerTok, each with a link to its original paper. In practice this means you pick a class (REMI in the examples) and pass it a TokenizerConfig, rather than adapting your pipeline to each scheme's idiosyncratic output format.

That shared-interface decision is the real product. The differences between these schemes are substantial. Octuple packs attributes into tuples per note. CPWord nests compound words. MIDI-Like emits note-on and note-off style events without explicit beat structure. A library that abstracts over all of them has to pick a lowest common denominator somewhere, and the config keys visible in the README (num_velocities, use_chords, use_programs) are that denominator made explicit. Setting num_velocities=16 quantizes velocity into sixteen buckets; use_chords=True adds chord tokens; use_programs=True adds program (instrument) tokens for multitrack handling. Those three keys alone change the vocabulary size and the musical information available to the model, so they are not cosmetic.

The README also notes tokenizers can be trained with Byte Pair Encoding, Unigram and WordPiece, with BPE and Unigram backed by Hugging Face tokenizers for what the README calls superfast encoding. That is a subword layer sitting on top of the musical event vocabulary, which is how a 30,000-token vocabulary in the example is reachable at all.

Installation and the round-trip call pattern

Installation is a single command, pip install miditok. The README states MidiTok uses Symusic to read and write MIDI and abc files, so that dependency arrives with the package rather than being something you wire up.

The usage pattern is unusual enough to be worth stating plainly: the tokenizer object is callable in both directions. The README example constructs a Score from a path, calls tokenizer(midi) to get tokens, then calls tokenizer(tokens) to get a MIDI back. The comment in the example notes that calling the tokenizer will automatically detect MIDIs, paths and tokens, and that PyTorch, Tensorflow and Numpy tensors are supported as input. That polymorphic dispatch is convenient but also means a bug in your input type can silently route down the wrong branch, so type-checking your inputs is worth the effort.

A minimal working setup looks like this, taken directly from the README:

from miditok import REMI, TokenizerConfig from symusic import Score config = TokenizerConfig(num_velocities=16, use_chords=True, use_programs=True) tokenizer = REMI(config) midi = Score("path/to/your_midi.mid") tokens = tokenizer(midi) converted_back_midi = tokenizer(tokens)

Note the import path: REMI and TokenizerConfig come from the top-level miditok package, while dataset utilities live under miditok.pytorch_data and helpers under miditok.utils. That split is visible in the README's training example and is the shape of the v3 API.

Training a tokenizer and feeding a PyTorch DataLoader

The README's training example is where the package stops being a converter and becomes part of a pipeline. Four steps, in order.

First, collect file paths and train the subword vocabulary:

files_paths = list(Path("path", "to", "midis").glob("**/*.mid")) tokenizer.train(vocab_size=30000, files_paths=files_paths) tokenizer.save(Path("path", "to", "save", "tokenizer.json"))

Second, optionally push it to the Hub, which the README says can be pulled back with from_pretrained:

tokenizer.push_to_hub("username/model-name", private=True, token="your_hf_token")

Third, split long MIDIs into fixed-length chunks, because a tokenized piece will not fit a context window:

split_files_for_training( files_paths=files_paths, tokenizer=tokenizer, save_dir=Path("path", "to", "midi_chunks"), max_seq_len=1024, )

Fourth, wrap the chunks in a DatasetMIDI and a DataCollator:

dataset = DatasetMIDI( files_paths=list(dataset_chunks_dir.glob("**/*.mid")), tokenizer=tokenizer, max_seq_len=1024, bos_token_id=tokenizer["BOS_None"], eos_token_id=tokenizer["EOS_None"], ) collator = DataCollator(tokenizer.pad_token_id, copy_inputs_as_labels=True) dataloader = DataLoader(dataset, batch_size=64, collate_fn=collator)

Two details carry weight. The special tokens are looked up by name, tokenizer["BOS_None"], not by hardcoded integer, which is what makes swapping tokenizations tractable. And copy_inputs_as_labels=True in the collator sets up the labels for a standard next-token objective, so the collator is doing more than padding. Note that max_seq_len appears in both the splitter and the dataset; if those two values diverge, the dataset will be truncating what the splitter already sized.

What the tokenizer does not cover: MusicXML, Control Change, and the missing Rust path

The README's own todo list is the most honest part of the document, and it should be read before adoption. Three items matter.

Support for MusicXML files is listed as a todo. If your source material is MuseScore or Finale exports rather than MIDI or abc, MidiTok does not currently take it, and you will need a conversion step outside the library. Control Change messages are also listed as a todo, which means sustain pedal, modulation and similar continuous controller data are not part of the token vocabulary. For piano performance modelling where pedalling carries expression, that is a real loss, not a rounding error. A no_duration_drums option is listed as well, which implies that today drum notes carry duration tokens that some users would rather discard.

The fourth todo is a speed item: speeding up global and track event parsing with Rust or C++ bindings. The README does not give a number, and I have not measured one, so treat the parsing cost as unknown until you profile your own corpus. The subword layer is already backed by Hugging Face tokenizers, so the likely bottleneck is the event extraction stage, which is exactly the part still written in pure Python.

A second limitation is structural rather than listed. Because all ten tokenizations share one config object, a scheme-specific parameter that does not fit the common surface has nowhere obvious to go. The v3.0.6 release notes mention Bar and Position support added for PerTok specifically, which is a small illustration of the pattern: new capabilities arrive as targeted additions rather than as a general time-representation overhaul.

MidiTok against music21 and pretty_midi

The obvious comparison is with music21 and pretty_midi, the two long-standing Python libraries for reading and manipulating symbolic music. The difference is not quality, it is the output type.

pretty_midi gives you a PianoRoll and note objects. music21 gives you a full analytical object model with streams, chords, keys and Roman numeral analysis. Both are designed for inspection, analysis and transformation of music as music. MidiTok is designed for one output: an integer sequence with a vocabulary you can train and save to tokenizer.json. If your task is key detection, score comparison or notation processing, music21 is the right layer and MidiTok adds nothing. If your task is training a Transformer, music21 gives you no vocabulary, no special tokens and no DataLoader, and you would be writing the encoding yourself.

A closer comparison is the tokenizer code shipped inside individual research repositories. Many published music generation papers release a tokenizer alongside the model, tightly coupled to that model's training script. MidiTok's difference in approach is to decouple the tokenizer from the model and expose the published schemes as interchangeable classes. That buys you reuse across projects and Hub distribution via push_to_hub. It costs you the ability to make a scheme-specific hack without working through the shared config, and it means you inherit the library's release cadence for a component that a paper's own repository would have frozen at publication time.

Licence, releases, and what you are taking on

MidiTok is MIT licensed, which is permissive and places few constraints on commercial or closed-source use. The dependency situation is where the practical questions sit: Symusic handles file I/O and Hugging Face tokenizers backs BPE and Unigram. The README does not state their licences, so if licence compatibility matters to your organisation, check each dependency directly rather than assuming MIT propagates through the tree. Nothing here is legal advice.

On maintenance, the repository is not archived and the last push recorded is 2026-09-08. The release cadence visible in the material is patch-oriented: v3.0.6 in June 2025, v3.0.6.post1 in July 2025, and v3.0.5.post1 before that in February 2025. The v3.0.6 notes describe minor fixes plus Bar and Position support for PerTok; the post1 release is labelled a PerTok patch. That pattern suggests active but incremental work concentrated on specific tokenizations.

The upgrade cost is the part to plan for. The API shown in the README is the v3 surface: TokenizerConfig as a separate object, callable tokenizers, DatasetMIDI and DataCollator under miditok.pytorch_data. Code written against earlier major versions will not match this shape. Because a trained tokenizer is a saved artifact, an upgrade can invalidate a tokenizer.json you have already trained and pushed, and retraining a 30,000-token BPE vocabulary over your corpus is not free. Pin the version you train with, and treat a major bump as a retraining event rather than a routine dependency update.

Editorial conclusion

Adopt MidiTok if you are training or fine-tuning a symbolic music model and want a tokenizer whose vocabulary matches a published tokenization, or if you need to move an existing tokenizer to the Hugging Face Hub with push_to_hub and from_pretrained. Do not adopt it if you need MusicXML input, Control Change messages, or a model rather than a tokenizer: the README lists the first two as open todos, and the package has never contained the second. Before committing, verify on your own corpus that detokenization round-trips acceptably for your tokenization and config, and check the release notes for the v3.0.x line, since the API shown here is the v3 surface.

Official sources

1. License: MIT
2. Natooz/MidiTok on GitHub
3. Project website
4. README
5. Releases