MSAF segments a recording into large-scale sections and labels them by similarity

What the framework is built to do

MSAF is a Python framework to analyze music structure, and the setup.py description is more precise than the README title: a Python module to discover the structure of music files.

The long description in the setup file spells out the intent. A Python module to segment audio into all its different large-scale sections and label them based on their acoustic similarity.

Three ideas in that sentence do the work. Large-scale sections means the unit of analysis is a verse or a chorus or an instrumental passage, not a note or a bar. Label them means the output is not just a segmentation but a set of labels attached to the segments, which is what makes the output analysable rather than merely visual. Acoustic similarity is the criterion, which means two passages are the same section if they sound alike, which is the definition most people mean by structure in popular music.

That combination is the reason the framework exists as a separate layer rather than as one script. A researcher who wants a different feature extractor or a different segmentation algorithm needs the algorithms as functions they can swap, and the example scripts in the repository are named for exactly those variations.

The framework is MIT licensed, the primary language is Python, and the topic is music-information-retrieval. The documentation is at msaf.readthedocs.io and the README points there for a complete reference manual and introductory tutorials. There is a readthedocs badge, so the docs build is part of the project's continuous integration rather than an afterthought.

The author is Oriol Nieto, and the project has a research citation attached: Nieto and Bello, Systematic Exploration Of Computational Music Structure Research, at the 17th International Society for Music Information Retrieval Conference in 2016. Citing a paper of your own in the README is a small thing, but in a research framework it tells you the code and the paper are meant to be read together.

The recent releases are MSAF Beta v0.1.80 from 2023-06-25, v0.1.70 from 2018-06-16 and v0.1.61 from 2017-10-02, and the last push was on 2026-05-13.

Installing it, and what the pinned requirements imply

The installation instruction is one line. From the root folder, type:

    pip install .

The README adds a note: you may need to create and activate a Python virtual environment with python -m venv .venv and source .venv/bin/activate first, depending on your system configuration.

That is a plain source install rather than an install from PyPI, and the package is on PyPI as well, so both routes exist. The source route is the one documented, and the README's phrasing suggests it is the tested path.

What deserves attention is the requirements.txt, because the pins tell you the age of the codebase:

jams==0.3.2
joblib==0.11
scipy==1.1.0
mir_eval==0.4
pandas==0.23.1
librosa==0.6.1
llvmlite==0.32.1
six==1.11.0
numpy==1.19.5
matplotlib==3.3.4
scikit_learn==0.19.1

These are exact pins on versions from 2017 to 2021. numpy 1.19.5 and scipy 1.1.0 are old enough that they predate Python 3.10, and librosa 0.6.1 is from the same era.

The setup.py tells the same story from the other end. Its classifiers list Programming Language :: Python :: 2 and Python :: 2.7, then Python :: 3 and versions 3.4, 3.5 and 3.6. The development status is 4 - Beta and the licence classifier is MIT.

So the framework as packaged was written for a Python generation that has since moved on, and whether it installs cleanly today depends on which interpreter you point it at. That is not a criticism of the pins, which are what a research code release from 2023 would look like, but it is the first thing to check.

The setup.py also carries a dependency list beyond what requirements.txt shows, and the visible portion names audioread for reading audio files, cvxopt, decorator, enum34, jams at 0.3.0 or newer, joblib, librosa at 0.6.0 or newer, mir_eval, matplotlib at 1.5 or newer, numpy at 1.8.0 or newer, and pandas. The jams dependency is the important one, since jams is the annotation format this field uses for structural annotations.

Two details in the packaging are worth knowing. The version is not written in setup.py but imported from msaf/version.py through importlib, so the version lives in one place. And package_data includes msaf: ["algorithms/olda/models/*.npy"], with the same glob listed in data_files, which means the algorithm that needs pre-trained models ships those .npy files inside the installed package.

The examples, which are the actual documentation

The examples directory is the most informative part of the repository layout, because the file names describe the extension points.

There is Run MSAF.ipynb, which the README points to as a thorough example on the Titanic, which is the audio recording used throughout the music information retrieval literature as a test case with known structure. A demonstration notebook that runs a full analysis of a piece with a known ground truth is a good default because you can see whether the output looks right.

Then there is compute_features.py, custom_feature.py and custom_segmenter.py. Three separate scripts, and the names tell you the framework has three pluggable stages: feature extraction, segmentation, and labelling. custom_feature.py shows how to substitute your own feature representation, custom_segmenter.py shows how to substitute the algorithm that decides where the boundaries are, and compute_features.py is the straightforward path where you use the built-in one.

That structure is the framework's argument. A structure analysis pipeline has at least three decisions in it, and each of them has a different set of sensible choices, so the framework exposes each as a separate seam.

Then run_msaf.py, which runs the whole pipeline, and run_mirex.py, which runs against the MIREX dataset, the standard benchmark collection for music structure analysis with ground truth annotations. And run_sweep.py, which sweeps parameters over a corpus.

The segmenter/ subdirectory inside examples/ suggests that the custom segmenters are grouped as a set rather than being one-off scripts.

Together those scripts cover the three things a researcher actually wants to do: run the standard pipeline on one file, run it over a benchmark with ground truth to measure how it does, and try your own feature or segmentation idea against the same harness. That is a well-designed set of examples, and it is the closest thing the project has to a tutorial.

The repository also has a datasets/ directory at the top level and a tests/ directory, plus a .coveragerc with a Coveralls badge in the README and a pre-commit configuration, so the project has both coverage measurement and a commit-time hook.

The jams annotation format and the model files that ship with the package

Two details in the packaging deserve a closer look, because they say more about how the project is meant to be used than the README does.

The first is jams, which appears both in the dependency list and in the requirements file, pinned at 0.3.2. Jams is the annotation format built for music information retrieval, where a structural annotation is a set of labelled time segments rather than a list of timestamps. That is why the framework's output is described as sections that are labelled: the natural representation for a section is an annotation with a label, a start time and an end time, and that is what jams stores.

The choice of jams as the interchange format means MSAF output can be compared directly against the ground truth annotations that ship with MIREX, which is what the run_mirex.py example is for. A segmentation you cannot compare to a benchmark is a segmentation you cannot improve on, and the fact that the annotation format is a dependency rather than an internal structure is what makes that comparison possible.

The second detail is the package_data entry. The setup includes msaf: ["algorithms/olda/models/*.npy"] and lists the same glob under data_files, so the algorithm that needs pre-trained models ships those .npy files inside the installed package rather than downloading them at runtime.

That is a deliberate choice with a consequence. Shipping the models means a pip install is self-contained and works offline, which matters for a research tool that has to run reproducibly on someone else's machine a year from now. It also means the models are versioned with the package, so a model is never silently out of step with the code that loads it. The cost is that the wheel is larger than it would otherwise be, and that a change to the models requires a new release rather than a download.

The names of the example scripts map onto the pipeline stages, and the segmenter subdirectory suggests that the custom segmenters are grouped as a set rather than being one-off scripts. Together those scripts cover the three things a researcher actually wants to do: run the standard pipeline on one file, run it over a benchmark with ground truth to measure how it does, and try your own feature or segmentation idea against the same harness.

Where MSAF is the wrong tool

Three limits, and the first is about the dependency situation.

The first is that the pinned dependency set is old enough to be a problem in itself. numpy 1.19.5, scipy 1.1.0, pandas 0.23.1 and librosa 0.6.1 were current in 2019 to 2021, and the setup classifiers still advertise Python 2.7 and 3.4. For a research framework published in 2023 and used by a small community, that is normal, and the pins exist because the numerical stack underneath is not guaranteed to be backward compatible across versions. But it means the install is likely to need an older interpreter or a set of adjustments to the pins, and the release tags run v0.1.61 in 2017, v0.1.70 in 2018 and v0.1.80 in 2023, so there is a long gap between the second and third of those as well.

The second is that this is a research framework, not a product. The setup description says discover the structure of music files, and the deployment story is a source install into a virtual environment. There is no service, no API, no queue of work and no configuration beyond the requirements file. That is appropriate for what the project is, and it means the code is structured for experimentation rather than for throughput: the algorithms are numpy and librosa calls over whole files, and the parallelism comes from joblib rather than from a task queue.

The third is the scope of the analysis itself. MSAF works on large-scale sections, which is the verse and chorus level. It does not give you note-level transcription, chord recognition, beat tracking or key detection, and the segmentation is by acoustic similarity rather than by anything harmonic or lyrical. If your question is where the chorus starts, this is the right tool. If your question is what note is being played at 1:23, it is not, and a different family of tools answers that.

The alternative approach for many of these needs is to use a feature extractor and a segmentation library directly and write the twenty lines of glue yourself. The difference in approach is that MSAF gives you the full pipeline with the decisions already made and a benchmarking harness against MIREX, while writing the glue gives you control over every stage and no dependency on a 2019-vintage dependency tree. For a paper reproduction or a comparison against published results, using the framework that the published results used is the point. For a production system running on a current Python, the glue is the better bet.

Editorial conclusion

Adopt MSAF if you are doing music information retrieval research and want the segmentation and labelling algorithms as callable Python rather than as a black box, because the example scripts cover custom features, custom segmenters and sweeps over corpora. Do not adopt it for production work without checking the dependency pins, because the requirements file pins versions from 2018 to 2021 and the setup classifiers still list Python 2.7 and 3.4 through 3.6. Verify first that the versions in requirements.txt install on your Python, that librosa 0.6.1 and the pinned numpy and scipy versions still work together, that the model files under the algorithm's models directory are present in the installed package, since the setup includes them as package data, and that the release you pin is one whose numbering you can read, since the tags run v0.1.61, v0.1.70 and v0.1.80.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. urinieto/msaf on GitHub