vocal-remover: A PyTorch U-Net That Splits Songs Into Instrumental and Vocal WAVs

The two-file output contract

The problem this project addresses is narrow and concrete. You have a mixed stereo song. You want the instrumental without the singing, or the isolated vocal, and you do not want to pay a service per track or upload your audio anywhere. The README states the goal directly: it is a tool for extracting the instrumental track from your songs. Given one input file, the documented behaviour is that the command separates it into instrumental and vocal tracks saved as *_Instruments.wav and *_Vocals.wav. Two files out, both WAV, named by suffix from the input path. That naming convention is the whole interface contract, and it is worth knowing before you script around the tool, because anything downstream has to match those suffixes. The audience is a Python user with a GPU who is comfortable running a script from a checkout rather than installing a package. There is no Homepage field in the repository metadata and no published package install step in the README; the installation section tells you to download the latest version from the releases page, then cd into the directory and run pip install -r requirements.txt. That is a source checkout workflow, not a library import.

Spectrogram masking, not waveform regression

The architecture is not described in the README beyond the title, which says Deep Neural Networks, and the reference list. That list is the clearest statement of the design lineage. It cites Jansson et al. on singing voice separation with deep U-Net convolutional networks, two Takahashi et al. papers on multi-scale multi-band DenseNets and on combining convolutional and recurrent networks for audio source separation, Choi et al. on a deep complex U-Net for phase-aware speech enhancement, Jansson et al. again on learned complex masks, and the 2016 Signal Separation Evaluation Campaign. Read together, those references point at a time-frequency masking approach: transform the mixture to a spectrogram, have the network predict a mask over that representation, apply the mask, and invert back to audio. The phase-aware and complex-mask citations suggest the project has moved toward handling phase rather than only magnitudes, which matters because magnitude-only masking is where a lot of separation artifacts come from. I cannot confirm from the supplied material which of these papers the current v6.0.0b4 checkpoint actually implements, or how the U-Net blocks are configured. What the repository layout does tell you is that training and inference are separate entry points, inference.py and train.py, so the model definition is shared between them and the checkpoint is the artifact that carries the trained weights. The topics list includes segmentation and spectrogram alongside pytorch, which is consistent with the masking reading but is metadata, not documentation.

Running inference on CPU or GPU

The commands are short. On CPU: python inference.py --input path/to/an/audio/file. On GPU: python inference.py --input path/to/an/audio/file --gpu 0. The --gpu flag takes a device index, so a multi-GPU machine would pass 1, 2 and so on. There is no batch flag documented and no output directory flag in the README, so the outputs land beside the input according to the suffix rule. Two advanced flags exist. --tta performs Test-Time Augmentation, which the README says improves separation quality; the usual meaning of that term is that the input is transformed several ways, the model runs on each, and the results are combined, which costs proportionally more compute. --postprocess masks the instrumental track based on the vocal volume, and the README carries a warning block calling it an experimental feature and asking you to disable it if you encounter problems. That warning is not boilerplate. A flag that modifies one output based on a measurement from the other output can fail quietly, producing an instrumental that sounds fine in isolation but has holes where the vocal was loud. Test it on a track you know before you put it in a pipeline. The exact set of accepted audio formats is not enumerated in the README; the examples show wav and mp3, and the training dataset example also shows mp3 files, so the decode path handles at least those two.

Training needs paired stems you probably do not have

The training path is where the project stops being a tool and becomes a research codebase. The README gives a dataset layout with exactly two directories: instruments/ containing files named like 01_foo_inst.wav and mixtures/ containing files named like 01_foo_mix.wav, with the numeric prefix pairing them. The training command is python train.py --dataset path/to/dataset --mixup_rate 0.5 --reduction_rate 0.5 --gpu 0. Two hyperparameters are exposed on the command line. mixup_rate at 0.5 suggests interpolation between training examples, and reduction_rate at 0.5 suggests some form of dimensionality or band reduction, but the README does not define either one, and guessing at their effect from the name alone is how you waste a day of GPU time. The practical constraint is the dataset itself. You need aligned instrumental and mixture recordings of the same performance. Public music datasets with that structure exist, but you cannot build one from finished commercial releases, because you would need the instrumental stem, which is the thing you are trying to produce. This is the circular dependency at the center of every supervised separation project, and the README does not address it. There is also no documented validation split, no evaluation metric, and no checkpoint selection guidance. You run train.py and you get weights; whether they are better than the released ones is something you determine yourself.

Where it is the wrong tool

The failure modes worth naming are the ones that follow from the design. First, this is offline file processing. There is no streaming API, no real-time mode, and no documented latency figure, so live monitoring or a DAW plugin is out of scope. Second, the output is two tracks, not many. If you need drums, bass and other separated as well, this project does not do that, and the reference list is about singing voice separation specifically. Third, the model is trained on music with a lead vocal in a fairly conventional mix position. Material that departs from that, such as dense choral arrangements, heavy vocal processing, or spoken word over music, is outside what the training data likely covered, and the README makes no accuracy claims you could hold it to. Fourth, --postprocess is flagged experimental by the maintainers themselves, which is an unusually direct signal that the feature is not ready for unattended use. Fifth, the version situation. The most recent release listed is v6.0.0b4 from July 2024, and the b4 suffix means beta. The stable line visible in the release list is v5.1.1. If you need something you can pin and forget, the beta channel is not that. The repository metadata shows a last push in September 2026 and the default branch is develop, so the develop branch is where the beta work lands.

How it differs from Demucs and Spleeter

The obvious alternative in the same space is Demucs, and the difference in approach is worth stating precisely. Demucs is a waveform-domain source separation system that outputs four stems by default, drums, bass, vocals and other, and it is distributed as an installable package with a command-line interface and a Python API. vocal-remover is a spectrogram-masking project with a two-output contract and a checkout-based workflow. That difference shows up in practice. With Demucs you install a package and get a multi-stem result; with vocal-remover you download a release, install requirements, and get instrumental plus vocal. Spleeter is the other common comparison point, a TensorFlow-based separator from Deezer that also offers multi-stem output and a packaged install. If your goal is only the karaoke-style instrumental and you want to stay in PyTorch, vocal-remover is the smaller dependency surface: PyTorch plus the requirements file, no framework conversion. If your goal is stems, or you want a maintained package with a stable API, the other two are the better fit. I am not claiming vocal-remover is more accurate than either; the repository supplies no benchmark table, and the 2016 SiSEC citation is a campaign reference, not a result for this model.

Maintenance cost and the MIT licence

The maintenance picture is mixed. On the plus side, the interface is small: two scripts, a handful of flags, a requirements file. There is little surface area to break. On the minus side, the dependency that matters is PyTorch, and PyTorch checkpoints are not portable across arbitrary version changes. A model saved under one PyTorch release may need code changes or a conversion step under another, and the README does not state which PyTorch version the released checkpoints were produced with. It only links to the PyTorch GET STARTED page and tells you to install the other packages from requirements.txt. That means the first thing to verify after install is that the checkpoint loads and inference produces audio, before you build anything on top. Upgrades are release downloads rather than pip upgrades, since there is no package install step documented, so moving from v5.1.1 to v6.0.0b4 means fetching a new archive and re-checking the checkpoint. The licence is MIT, which is permissive and permits commercial use and modification provided the copyright notice and permission notice are retained; that is a summary of the licence identifier in the repository metadata, not legal advice, and if you are shipping this inside a product you should read the LICENSE file in the release you download. The references in the README point at third-party papers, and nothing in the supplied material indicates whether the released weights carry terms separate from the code. That is a question to resolve before commercial distribution.

Editorial conclusion

Adopt vocal-remover if you want an offline, MIT-licensed Python script that turns a song file into an instrumental and a vocal WAV on your own machine, and you are willing to supply a GPU and a paired dataset if the pretrained checkpoint is not good enough for your material. Do not adopt it if you need real-time separation, stem-level control beyond two outputs, or a supported training pipeline, because the repository ships a research-oriented train.py with a fixed dataset layout and no documented evaluation loop. Verify three things before committing: that the released checkpoint loads against your installed PyTorch version, what --postprocess does to your specific tracks given the README labels it experimental, and whether your input format survives the decode step, since the README only shows wav and mp3 examples.

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. tsurumeso/vocal-remover on GitHub