Adaptive-MultiSpeaker-Separation

The core idea is a sparse linear autoencoder that reads raw audio and replaces spectrograms, wrapped around established separation models like Deep Clustering and L41.

The centerpiece is an Adaptive Layer built as a sparse linear autoencoder. Instead of working from spectrograms, it deals directly with raw audio files, which is a different route than most separation pipelines take. The autoencoder is added around existing architectures rather than replacing them wholesale, so it reads as a wrapper that changes the input representation of models that were already doing separation.

The repository names the models the layer attaches to: a Deep Clustering Model, with and without an enhancing layer, and the L41 model, also called Magnolia, again in both variants. That gives four combinations in the README's framing. The point is that the adaptive layer is an add-on that can be tested against a current architecture, not a whole system built from scratch.

Those setups are compared against traditional STFT approaches running the same architectures. Two more entries appear as placeholders: a Dense Model and a PIT Model, both marked TODO and both limited to two speakers. So the comparison table is honest about what has actually been run and what is still planned, which keeps expectations about the current state realistic.

The pretraining section exposes a set of flags. You can mix men voices or women voices, set the number of speakers in a mix, or disable random picking so the mixing follows a regular man, woman pattern. Then there are the loss controls: beta for the sparsity loss based on KL divergence, a regularization coefficient for L2 weight decay, an overlap coefficient for the overlapping loss, and a choice of loss between l2, sdr, or both. The MIT license and the small star count suggest a research project more than a maintained tool.

• Source 1 · Project repository
• Source 2 · README