voice_activity_detection: a 1D-ResNet VAD trained on LibriSpeech test-clean

The problem it addresses: frame-level speech detection without a cloud service

Voice activity detection answers a narrow question: does this slice of audio contain speech or not. Many pipelines need that answer before anything expensive happens, whether that is an ASR decoder, a recording trigger, or a bandwidth-saving encoder. The project targets engineers who want that decision made by a locally trained neural network rather than by an energy threshold or a hosted API, and who are comfortable reading PyTorch code. The README states the goal directly: design and implement a real-time Voice Activity Detection algorithm based on Deep Learning. The intended user is someone who wants to see the whole path from raw audio to a speech logit, including feature extraction and model export, and who is willing to run training themselves. It is explicitly framed as a toy project, with the note that it splits the test-clean sub-dataset into train, val and test for quick iteration. That framing matters: this is a reference implementation and a starting point, not a packaged product with a stable API.

The feature tensor: 1024 samples become a 16x65 MFCC stack

The core design decision is what the model actually sees. Each audio window of SEQ_LEN = 1024 samples at 16 kHz is converted into a 16 x 65 feature tensor. The README lists exactly what is stacked: 5 MFCC coefficients, 5 first-order MFCC deltas, 5 second-order MFCC delta-deltas, and 1 RMS energy value. That is 16 rows of features across 65 time frames. Everything downstream depends on this shape, which is why the pipeline is not plug-and-play with arbitrary audio. If your sample rate differs from 16 kHz, the window duration changes and the frame count changes with it. The choice to include delta and delta-delta coefficients means the model sees local spectral change, not just static spectrum, which is a common way to give a small classifier more temporal context without a recurrent layer. RMS energy is appended as a single scalar row, so the network can fall back on loudness when the cepstral features are ambiguous. None of this is unusual, and that is the point: the feature stage is conventional and readable, which makes the repository useful as a teaching example even if you never train it.

The model: stacked residual blocks with a single speech logit

The classifier is `vad.model.Resnet1D`, described in the README as stacked residual blocks built from 3 x `Conv1d` plus `BatchNorm1d` with a 1x1 shortcut, followed by global average pooling and a fully connected head that produces one speech logit. The architecture is defined by a `ModelConfig` dataclass, and the README is explicit that the same configuration must be used at training, export and inference time. That constraint is the single most likely source of silent breakage in this project. The `--n-filters` and `--fc-units` flags change the architecture, and the README repeats the warning in the inference section: if you trained with custom values, pass the same values to `vad-inference` so the checkpoint loads into a matching architecture. A single logit output means the decision is binary by construction. There is no multi-class head, no speaker conditioning, no streaming state carried between windows. The README reports 99 percent train accuracy, 98 percent validation and 97 percent test accuracy for the 1D-ResNet, presented as a table rather than as a benchmark against other systems, so treat those numbers as the author's own measurement on the author's own split.

Getting it running: uv sync, then three CLI entry points

The project requires Python 3.11+, PyTorch 2.2+, and the uv package manager. Installation is a clone followed by `uv sync`, which installs all dependencies; the README also notes `uv sync` covers the development setup with ruff and pytest, and that `make lint`, `make test` and `make format` wrap the common tasks. Docker is available as an alternative: `make build` then `make local-nobuild` for the CPU image, and `make build-gpu` for GPU support, which the README says requires the NVIDIA Docker runtime. The workflow has three commands. First, `uv run vad-data --data-dir /path/to/LibriSpeech/` builds PyTorch datasets and writes `.pt` files to `/path/to/LibriSpeech/dataset/{train,val,test}/`; `--max-files N` limits how many files per split are processed. Second, `uv run vad-train --data-dir /path/to/LibriSpeech/dataset/ --model-dir /path/to/models/` trains and exports both a state dict and a TorchScript model to `<model-dir>/exported/`, unless `--no-export` is passed. Flags include `--epochs/-e`, `--batch-size/-b`, `--lr`, `--n-filters` and `--fc-units`; the training device is selected automatically across CUDA, Apple MPS and CPU, and TensorBoard logs land in `<model-dir>/logs/`. Third, `uv run vad-inference --data-dir /path/to/LibriSpeech/ --checkpoint /path/to/models/exported/model_state_dict.pt --smoothing --max-files 1` runs sliding-window inference with optional smoothing. Note the checkpoint path: inference loads `model_state_dict.pt` from the exported directory, not the TorchScript artifact.

The dataset dependency is the real cost of using this

You cannot run this project end to end without LibriSpeech. The README instructs you to download the ASR corpus from openslr.org/12 and extract it to a path of your choice, noting roughly 1000 hours of 16 kHz read English speech from audiobooks. More importantly, the labels come from outside the repository: the author states that the test-clean set was automatically annotated with a pretrained VAD model, and that the `labels/` folder and a pre-trained inference-only model are available from a Google Drive link. That means the ground truth is itself model-generated, not human-verified. For a project whose headline metric is test accuracy, this is the detail to weigh most heavily. The README also says the project splits test-clean into train, val and test for quick iteration and can be extended to a full large-scale dataset, so the reported numbers describe a small, read-speech, English, audiobook domain. Broadcast audio, telephone speech, overlapping speakers and noisy environments are all out of scope as described. If your audio does not resemble read English audiobooks, the accuracy table tells you very little about your case.

Where it breaks down, and what to use instead

Two limitations are visible from the documentation alone. The first is the annotation source: training against labels produced by another VAD means the model is partly learning to imitate that system, and any systematic error in those labels is inherited. The second is the gap between the stated goal of real-time detection and the delivered interface. `vad-inference` is a script that processes files with a sliding window and an optional `--smoothing` flag; the README does not document a streaming API, a ring buffer, or a callback that emits decisions as audio arrives. Anyone needing frame-level decisions inside a live audio callback will be writing that layer themselves. As an alternative, consider a conventional energy-and-zero-crossing VAD such as the one in the WebRTC audio processing stack. The difference in approach is fundamental: WebRTC-style VAD uses signal statistics and adaptive thresholds with no learned parameters, so it ships as a small fixed library, works on any sample rate after resampling, and needs no training data or checkpoint. This project trades that portability for a learned decision boundary that can separate speech from stationary noise where an energy threshold cannot, at the cost of a LibriSpeech download, a training run, and a checkpoint that must match the architecture flags. Neither is strictly better; they fail in different places.

Maintenance, packaging and the GPL-3.0 boundary

The repository is not archived, the default branch is master, and the last push recorded is 2026-07-22. There are no retrieved releases, so versioning is by commit rather than by tagged artifact. The CI workflow runs ruff lint, `ruff format --check` and the pytest suite on every push and pull request, and the CD workflow builds the Docker image and pushes it to Docker Hub on pushes to master or main, tagged with the commit SHA. That is a reasonable maintenance posture for a small project: lint, tests and a published image, with `make test` producing a coverage term report and an HTML report in `htmlcov/`. The tests directory is described as pytest integration tests covering training, export and inference, which means a broken pipeline should surface in CI rather than at your first inference run. The licence is GPL-3.0. That is a copyleft licence, and it governs the code in this repository; how it interacts with your own distribution depends on whether you link, modify or ship the code, which is a question for your own legal review rather than something this article can settle. If you plan to embed a VAD in a closed product, read the licence before you build on the checkpoint path.

Editorial conclusion

Adopt it if you want a readable, end-to-end PyTorch example of MFCC-based VAD with a working train, export and inference path, and if you are willing to supply your own audio and labels. Do not adopt it as a drop-in component for a streaming product: the README describes a sliding-window inference script, not a real-time streaming API, and the reported 97 percent test accuracy comes from the author's own annotation of LibriSpeech test-clean. Before relying on it, verify three things: that your audio is 16 kHz, that your inference command passes the same --n-filters and --fc-units values used during training, and that the GPL-3.0 licence is acceptable for how you plan to ship the code.

Official sources

1. filippogiruzzi/voice_activity_detection on GitHub
2. Issues
3. License: GPL-3.0
4. README