Asteroid: a PyTorch toolkit for audio source separation research

The gap Asteroid fills between a paper and a working training run

Source separation papers usually ship as a repository tied to one dataset and one architecture. Asteroid takes the opposite route and collects many of them under a single installable package. The README describes it as a toolkit that "enables fast experimentation on common datasets" and states that it comes with source code supporting a large range of datasets and architectures plus "a set of recipes to reproduce some important papers". That sentence is the whole product thesis. The intended user is a researcher who has read Conv-TasNet or DPRNN and wants a training loop that already knows how to load WSJ0-2mix, rather than someone who wants a model file to drop into an app. The repository is organised around that split: asteroid/ holds the reusable library (models, data, losses), while egs/ holds per-dataset, per-architecture experiment folders. Support is stated for speech separation, speech enhancement and music separation, and the topics list includes pretrained models, so the project is not purely a training framework.

How the library and the egs recipes divide the work

The architecture visible in the README is a two-layer design. The first layer is the importable package. Models live as individual modules under asteroid/models/, for example sudormrf.py, dptnet.py, dccrnet.py, dcunet.py and x_umx.py, and dataset loaders sit under asteroid/data/ with files such as musdb18_dataset.py, fuss_dataset.py, avspeech_dataset.py and kinect_wsj.py. The second layer is egs/. Each recipe is a directory like egs/wham/ConvTasNet or egs/wsj0-mix/DeepClustering, and the README's instruction for running one is to change into that directory and source its run.sh script. That means the experiment configuration, the dataset path assumptions and the training entry point all live in the recipe folder, not in the installed package. The tutorials follow the same separation. Notebooks such as 03_PITLossWrapper.ipynb and 02_Filterbank.ipynb exercise single library components in isolation, while 04_ProcessLargeAudioFiles.ipynb addresses a practical constraint of separation work: audio files that do not fit comfortably in memory during inference. If you are deciding whether to adopt this, the useful question is which layer you need. The library layer is reusable across projects. The recipe layer is closer to a snapshot of how a specific paper was trained.

Installing it: three paths and what each one gives you

The README lists three installation routes. The first is a clone followed by an editable install: git clone https://github.com/asteroid-team/asteroid, cd asteroid, then pip install -e . The comment in the README notes that python setup.py develop is the dev-mode equivalent. The second is conda: conda env create -f environment.yml followed by conda activate asteroid. The third skips the repository entirely with pip install asteroid from PyPI. These are not interchangeable. The PyPI route gives you the library, so you can import models and dataset classes, but the egs recipes are not part of the installed package, so you cannot run a recipe from a plain pip install. Running recipes also needs a separate step: the README says recipes "requires additional packages in most cases" and recommends pip install -r requirements.txt from the repository root. So the realistic setup for experimental work is clone, install requirements, then install the package itself. The conda path exists because audio dependencies are frequently easier to resolve outside pip, though the README does not say which packages motivate it. There is no documented configuration file for dataset locations; the recipes' run.sh scripts are where that lives.

What is actually implemented, and what the README admits is missing

The recipe list is the most honest part of the documentation because it uses checkboxes. Marked as available: ConvTasNet, TasNet, Deep clustering, Chimera++, DualPathRNN, two-step learning, SudoRMRFNet, DPTNet, DCCRNet, DCUNet, CrossNet-Open-Unmix and Multi-Decoder DPRNN. Marked as not done: Open-Unmix and Wavesplit, both labelled "coming". The contributing section repeats the gap and lists Wavesplit, FurcaNeXt and DeepCASA as things the project is "missing", and it explicitly frames this as a place for new contributors to start. That is worth reading carefully, because it tells you the project is maintained as a community effort rather than a finished product, and the README says so directly: "Asteroid is intended to be a community-based project so hop on and help us!" The dataset list is broader than the model list in one respect: WSJ0-2mix and WSJ0-3mix, WHAM, WHAMR, LibriMix, the Microsoft DNS Challenge, SMS_WSJ, MUSDB18, FUSS, AVSpeech and Kinect-WSJ are all checked. Note the asymmetry. Datasets marked as supported are not the same as datasets with a recipe for every architecture. If your combination of dataset and model is not in the egs tree, you are writing the recipe yourself.

Version drift is the main practical risk

The release history shows a project whose recent work is dominated by dependency upgrades rather than new models. v0.6.0 is titled "Upgrade to newer PyTorch-Lightning versions (1.5+)". v0.6.1 is titled "Prepare upgrade to Torch 2.x and Lightning 2.x". v0.7.0, the most recent release listed, is titled "Upgrade to Torch 2.x and Lightning 2.x". Three consecutive releases are framework migrations. For a research toolkit this matters more than it would elsewhere, because recipes encode training behaviour that can shift when the training loop library changes underneath them. The documentation badge points at a versioned docs build for 0.7.0, which suggests the project keeps versioned documentation rather than only a latest build. The last push to the default branch is dated 2026-05-13, so the repository is not dormant, but the release list supplied here stops at October 2023. If you are pinning Asteroid for a long experiment, pin PyTorch and PyTorch-Lightning alongside it and expect the recipe scripts, not the library, to be where breakage surfaces first.

Where Asteroid is the wrong choice

Asteroid is a research toolkit, and the README never claims otherwise. Several concrete cases argue against it. If you need a separation model inside a latency-sensitive application, nothing in the supplied material describes an inference runtime, model export path or serving component; the library is PyTorch modules and dataset classes. If you need a model for a task outside the supported dataset list, the value proposition drops sharply, because the recipe scaffolding is dataset-specific and you would be using the models as ordinary PyTorch code. If you want a stable API surface, the release titles show active migration across major PyTorch and Lightning versions, which is the opposite of stability for downstream code. And if you are looking for music separation specifically, MUSDB18 is listed as a supported dataset, but the recipe list is dominated by speech datasets (WHAM, WHAMR, WSJ0-mix, LibriMix, SMS_WSJ), so the depth of music support is not established by the material here. The pretrained models are the other soft spot: the README points to a separate document at docs/source/readmes/pretrained_models.md rather than listing checkpoints inline, so you cannot tell from the README alone whether a pretrained model exists for your target task.

Asteroid against SpeechBrain, and what the difference costs you

SpeechBrain is the closest comparison for someone choosing a PyTorch audio toolkit, and the difference is in what the repository is organised around. Asteroid is organised around reproduction: the primary unit is an egs recipe that corresponds to a published paper, and the README's framing is reproducing "some important papers" on "common datasets". SpeechBrain is organised around a general speech processing library with a recipe collection spanning tasks beyond separation, including speech recognition and speaker tasks. The practical consequence is where you spend your time. With Asteroid, the shortest path to a working system is to find an egs directory matching your dataset and model, source run.sh, and adapt the hyperparameters in that folder. With a broader toolkit, you more often assemble a pipeline from library components and write the training loop yourself. Neither is strictly better. Asteroid's approach means the recipe is only as current as the paper it came from, and the migration-heavy release history shows what that costs. A general toolkit's approach means more assembly work before your first training run, but less coupling to one paper's configuration. If your goal is a specific published result on a specific dataset, Asteroid's structure is a real shortcut. If your goal is a production pipeline that happens to include separation, the recipe layer is overhead you will end up discarding.

Licence, maintenance and what to check before you commit

The repository is MIT licensed, which is permissive and imposes few conditions on redistribution or modification. That covers the code in this repository. It does not automatically cover the datasets the recipes target: WSJ0-2mix, WHAM, WHAMR, LibriMix, SMS_WSJ, MUSDB18, FUSS, AVSpeech and Kinect-WSJ each come from their own source with their own terms, and several are derived from corpora that require a separate agreement. The README links each dataset to its originating paper or page but says nothing about dataset licensing, so that check is on you. On maintenance cost, the honest reading of the material is that keeping a working Asteroid setup means tracking two upstream frameworks. The v0.6.x and v0.7.0 release titles are all PyTorch and PyTorch-Lightning migrations, and the project ships a versioned docs build per release, which implies users are expected to stay near a specific version rather than float. Budget for that. The concrete first checks: confirm your dataset and architecture combination appears in the egs tree, read the pretrained models document to see whether a checkpoint exists for your task, and decide early whether you need the library only or the recipes as well, because that choice determines whether pip install asteroid is sufficient or whether you need the clone plus requirements.txt.

Editorial conclusion

Adopt Asteroid if you are reproducing a separation paper or training a model on WSJ0-2mix, WHAM, WHAMR, LibriMix, SMS_WSJ, MUSDB18, FUSS, AVSpeech or Kinect-WSJ and you want the recipe already wired to the dataset. Do not adopt it as the inference layer of a product: the pretrained models live in a separate document, the default install is a repository clone, and the recipes need extra packages from requirements.txt. Before committing, run pip install -e . inside the clone, confirm the egs directory for your target dataset exists, and check the pretrained model list to see whether a checkpoint for your task is actually published.

Official sources

1. asteroid-team/asteroid on GitHub
2. License: MIT
3. Project website
4. README
5. Releases