Adaptive-MultiSpeaker-Separation swaps spectrograms for a sparse autoencoder

A sparse autoencoder over raw audio, not spectrograms

The Adaptive Layer is the contribution, and its defining property is where it sits in the signal path. It is a sparse linear autoencoder that replaces the use of spectrograms and deals directly with raw audio files. Sparsity is not incidental: the argument list exposes a coefficient called beta that controls a sparsity loss expressed as a KL divergence, alongside a separate coefficient for L2 regularisation on the weights and another for an overlapping loss. So the layer is trained to produce a compressed representation of raw audio with an explicit information bottleneck, rather than operating on a time-frequency image of it. Two architectural hyperparameters control its shape: filters, described as the number of autoencoder bases, and max_pool, the max pooling size along the time axis.

Two architectures, each with and without the layer

The layer is not a model on its own, and the README is explicit that it is added around existing architectures. Two receive it: the Deep Clustering model and the L41 model, also referred to as Magnolia. Each is run in two configurations, with and without the enhancing layer, which is what makes the comparison meaningful rather than decorative. Those same two architectures are then compared against traditional STFT approaches, so the design produces a small grid: the same model, with or without the adaptive layer, against a spectrogram-based baseline. Two further entries in the comparison are marked TODO: a Dense model with 2 speakers and a PIT model with 2 speakers. Neither is implemented, so any result from this repository is a comparison against two of four baselines rather than four.

The Focus Layer is the half that was never finished

The repository title and its description do not quite agree about scope, and the difference matters. The title says Adaptive and Focus Layer, the repository description says Adaptive and Focusing Neural Layers, and then a single line settles the question: the Focus Layer is in construction. Everything documented in the arguments section, every hyperparameter and every training command belongs to the Adaptive Layer. There is no flag for a focus mechanism, no coefficient for it and no comparison arm that varies it. If your interest in this work is the focusing idea rather than the sparsity idea, the repository will not get you there, and the honest reading is that half the announced contribution was never implemented rather than that it is hidden somewhere in the tree.

The flags are grouped by what they actually change

The argument list is organised into three groups, and reading them as groups rather than as a flat list makes the training command legible. Data-related flags select the voice sets with --men and --women, set the number of mixed speakers with --nb_speakers, and control the mixing strategy: --no_random_picking forces a regular alternating pattern of man and female voices instead of picking speakers at random, which removes one source of run-to-run variance. Adaptive Layer coefficients cover the sparsity beta, the weight regularisation, the overlap coefficient and the loss choice among l2, sdr and l2+sdr. Training hyperparameters are just the batch size and the learning rate. The pretraining invocation that carries them all is a single line:

python -m experiments.training.pretraining --men --women --loss sdr+l2 --separation mask --learning_rate 0.001 --nb_speakers 2 --batch_size 4 --filters 256 --max_pool 256 --beta 0.0 --regularization 0.0 --overlap_coef 1.0 --no_random_picking

One flag in that documented command is missing from the documentation entirely: --separation mask appears in the invocation but is not described in any of the three groups.

TensorFlow appears twice, at two different versions

The requirements file is the most immediate obstacle to running this, and the obstacle is a contradiction rather than an age problem on its own. It pins tensorflow_gpu at 1.4.0 and tensorflow at 1.5.0rc1, a release candidate, in the same eleven-line list. Two different TensorFlow versions cannot both be the intended one, so installing the file as written will either conflict or leave the resolution to whichever package manager is more forgiving. The rest of the pins are from the same period:

scipy==1.0.0
tqdm==4.19.4
SoundFile==0.9.0.post1
matplotlib==2.1.0
numpy==1.12.0
tensorflow_gpu==1.4.0
librosa==0.5.1
haikunator==2.1.0
h5py==2.7.0
scikit_learn==0.19.1
tensorflow==1.5.0rc1

With haikunator at 2.1.0 providing the model definitions that the Deep Clustering and L41 architectures come from.

Training is submitted to a cluster, not run locally

The repository's job files say where the compute lives. There are PBS job files, one for pretraining and one for front fine-tuning, alongside shell entry points including job.sh and two model-specific training scripts named for the Deep Clustering and L41 architectures with an stft suffix marking the spectrogram baseline. Two Python scripts sit beside them, one named qsub_pretaining.py and one for submitting front fine-tuning, which is how jobs reach the scheduler. So the intended workflow is: edit configuration through config.py, submit a pretraining job, then submit a fine-tuning job that depends on it. That structure has a practical consequence for anyone picking this up today, because the code assumes a batch scheduler with job dependencies rather than a script you run and wait for.

The last commit is dated 7 July 2018

The repository is not archived, but it has not been touched in a long time: the last commit is dated 7 July 2018, and there are no tagged releases at all. That date should govern how you treat every other file here. The requirements describe a TensorFlow 1.x stack from before version 2 changed the graph execution model, the comparisons were against architectures that were state of the art at the time, and the two TODO baselines were never filled in. There is no issue tracker guidance, no changelog and no release history to consult for what changed. What the repository is: a snapshot of a specific experiment, with a README that documents its arguments carefully enough to rerun, and code that has been sitting untouched for eight years.

Editorial conclusion

This repository suits someone reproducing or extending a 2018-era multi-speaker separation experiment, not someone looking for a working separation tool. Before you plan around it, check four things: whether a TensorFlow 1.x environment on Python 3 of that era is something you can still provide, whether the two TensorFlow pins in the requirements file conflict in your setup, whether the Focus Layer you need is the one never finished, and whether the baselines marked TODO matter for the comparison you want to draw.

Frequently asked questions

Official sources

1. Official README
2. Project repository
3. README
4. Releases