# nnAudio: spectrograms as trainable PyTorch layers

> A toolbox that replaces fixed audio transforms with one dimensional convolutions, so the Fourier kernels themselves can be learned alongside the network.

**KinWaiCheuk/nnAudio** — Audio processing by using pytorch 1D convolution network

- Repository: https://github.com/KinWaiCheuk/nnAudio
- Stars: 1,127 · Forks: 97
- Language: Python
- License: MIT
- Published: 2026-10-07 · Updated: 2026-10-07 · Language: en
- Canonical page: https://hysenlabs.com/projects/kinwaicheuk-nnaudio

## A spectrogram implemented as a convolution

The premise of nnAudio is that a spectrogram does not have to be a fixed transform. The library implements STFT, mel, MFCC, CQT, VQT, gammatone and CFP as PyTorch layers built on one dimensional convolution, so the basis functions become tensors that carry gradients instead of constants applied once during preprocessing. Two things follow from that choice. The transform runs wherever the model runs, on GPU without a separate audio pipeline, and its kernels can be updated by the optimizer alongside the network reading the output.

The name is a small clue to the implementation. It comes from `torch.nn`, the module the code leans on, and that is the basis for the README's portability claim. Comparing the project against other GPU audio processing tools, torchaudio and tf.signal, the README notes that they are not using a neural network approach, and hence the Fourier basis can not be trained. As of PyTorch 1.6.0, torchaudio was also still difficult to install under Windows because of `sox`, which is the specific friction nnAudio says it avoids by relying mostly on PyTorch convolution.

## Reading the comparison table instead of the feature list

Most of the README is a thirteen row comparison table with columns for nnAudio, torch.stft, kapre, torchaudio, tf.signal, torch-stft and librosa. Rows cover trainable, differentiable, linear frequency STFT, logarithmic frequency STFT, inverse STFT, Griffin-Lim, Mel, MFCC, CQT, VQT, Gammatone, CFP and GPU support. Reading it column by column tells you more than the surrounding prose, because the table is where the project states what it will not do.

nnAudio is the only column that is ticked across the board. `torch.stft` covers linear STFT, inverse and GPU but has no logarithmic frequency variant, no Griffin-Lim and none of the filterbank transforms. kapre, the closest Keras based alternative with a similar concept, is trainable and differentiable and handles linear and logarithmic STFT, inverse and mel, yet leaves MFCC, CQT, VQT, gammatone and CFP empty. torchaudio is differentiable and covers inverse, Griffin-Lim, mel and MFCC, but its CQT, VQT, gammatone and CFP cells are blank. librosa has the broadest transform coverage and is the only other column with CQT and VQT, at the cost of being neither trainable nor differentiable and having no GPU support.

## Two ways to install, one pinned version

The README offers a source install that points at a subdirectory of the repository rather than at the package root:

```bash
pip install git+https://github.com/KinWaiCheuk/nnAudio.git#subdirectory=Installation
```

That `#subdirectory=Installation` fragment lines up with the repository tree, where `Installation/` sits beside `Sphinx/`, `figures/`, `CHANGELOG.md`, `LICENSE` and `.gitignore`. The second form is the one most people will copy, because it pins a version:

```bash
pip install nnaudio==0.3.4
```

Full parameter documentation lives at the documentation site rather than in the repository, which is normal for a Sphinx project and worth knowing before you start reading the tree for API details. The dependency floor is stated plainly: Numpy 1.14.5, Scipy 1.2.0, PyTorch 1.6.0 with Griffin-Lim only available after that release, and Python 3.6. One dependency claim is more interesting than the rest. The README lists librosa 0.7.0, then explains that only a single `mel` function is needed from `librosa.filters`, so the author copied that chunk into the package and nnAudio runs without librosa installed.

## The module rename that is still in progress

The changelog records a naming migration that, as of the entries shown, has not completed. Version 0.3.0, dated 19 Nov 2021, changed module naming: `nnAudio.Spectrogram` will be replaced by `nnAudio.features` in future releases, and various spectrogram types are currently accessible via both methods. That dual access means either import path works today, which is convenient, but it also means nobody has yet deleted the older one.

The earlier release note shows why the refactor mattered. In v0.2.0 the code was restructured so that any spectrogram class supports `stft_layer.to(device)` the way other torch modules do, and the device argument was dropped as a result. For an nnAudio layer inside a PyTorch model, `model.to(device)` is then enough to move the transform onto the GPU, which is the behaviour people expect from a module. The same note contains a small inconsistency worth copying carefully: the older form is written `nnAudio.Spectrogram.STFT(device=device)` while the replacement in that line is spelled `Spectogram`, with one r. If you are grepping your own code for the old class name, both spellings can turn up.

A later change, version 0.3.1 on 24 Dec 2021, added the VQT feature and is the newest entry in the changelog file.

## Tests that want a real GPU

The contribution notes explain how to run the tests and what they will cost you. You change into the Installation folder and run `pytest`, and the README warns that you need at least 1931 MiB of GPU memory to pass all the unit tests. That number is a concrete signal about how the library is tested: the convolution kernels are verified against GPU execution, so a CPU only environment cannot clear the full suite even though the package itself imports and runs fine without one.

The open contribution list is short. The one pending feature named is an invertible Constant Q Transform, which matters because CQT is the transform listed as supported but, without the inverse, cannot be paired with a reconstruction step the way linear and logarithmic STFT can.

Two more details from that section. Contributions are invited from people familiar with digital signal processing and neural networks, and the alternative contribution is documentation work, specifically better demonstration code or a tutorial. The project also asks that you cite the paper if you use the library. The reference is Cheuk, Anderson, Agres and Herremans, published in IEEE Access volume 8 in 2020, pages 161981 to 162003, with the title describing the toolbox as an on-the-fly GPU audio to spectrogram conversion tool built on 1D convolutional neural networks.

## Version numbers and a successor repository

The three number sets in this repository do not agree, and knowing which one describes your install saves an afternoon. The install line pins `nnaudio==0.3.4`. The changelog's newest entry is 0.3.1 from December 2021. The published releases stop earlier still: v0.1.5, described as the first stable release for documentation using the sphinx-versioning extension, and v0.2.0, both from November 2020. There is no published release for any of the 0.3.x versions, so the changelog file is the better record of what changed and the install line is the better record of what you get.

The second thing to know is at the very top of the README. A notice announces that a new version of nnAudio has been released at AMAAI-Lab/nnAudio2, actively maintained by Dorien Herremans and her lab, who was the original author's PhD supervisor. A second notice asks for maintainers, stating that the project has grown beyond what one person can maintain and that the original author is no longer able to do regular code review, naming code review, issue triage and ongoing maintenance as the areas needed.

That context matters for a 1,127 star dependency. This repository is not archived and its last push was on 2026-05-21, so the code is still moving, but the maintenance burden has moved elsewhere. There are 21 open issues against it.

## Conclusion

nnAudio earns a place when the front end itself is part of what you want to learn, because it is the one audio toolbox in this comparison that makes the basis functions differentiable and trainable while still covering CQT, VQT, gammatone and CFP. If you only need a stable mel spectrogram on the GPU, torchaudio gets you there with less code and no GPU memory test suite to satisfy. Two things to check before committing: the pinned version on the install line does not match the changelog or the published releases, and the maintainer notice at the top of the README points to a successor repository. Start with the STFT and mel layers, which are the most heavily used, and read the documentation site for the parameter names before wiring up the rarer transforms.

## FAQ

### Is nnAudio still maintained, or should I use nnAudio2?

The README at the top directs readers to AMAAI-Lab/nnAudio2, maintained by Dorien Herremans and her lab, and a second notice asks for maintainers because the original author can no longer do regular code review. The nnAudio repository itself is not archived and its last push was on 2026-05-21, so both versions are live.

### How does nnAudio differ from torchaudio and librosa?

nnAudio builds its transforms from one dimensional convolutions, which makes the basis functions trainable and differentiable. torchaudio and tf.signal are not neural approaches, so their Fourier basis cannot be trained, and librosa is neither trainable nor differentiable and has no GPU support. torchaudio does cover inverse STFT, Griffin-Lim, mel and MFCC, which nnAudio includes as well.

### Which transforms does nnAudio support that the alternatives do not?

The comparison table gives nnAudio full coverage of linear and logarithmic frequency STFT, inverse STFT, Griffin-Lim, mel, MFCC, CQT, VQT, gammatone and CFP. CQT and VQT appear only in the nnAudio and librosa columns, and gammatone and CFP appear only in nnAudio. Invertible CQT is named as still pending, so the inverse is not there yet.

### Do I need a GPU to run nnAudio tests?

The README says you need at least 1931 MiB of GPU memory to pass all the unit tests, and the quick tip is to change into the Installation folder and run pytest. The package itself imports and runs without a GPU, but the full test suite verifies the convolution kernels on GPU execution, so a CPU only environment cannot clear it.

### Why is nnAudio easier to install on Windows than torchaudio?

The README attributes the difference to sox, which made torchaudio difficult to install in a Windows environment as of PyTorch 1.6.0. nnAudio relies mostly on PyTorch convolutional neural networks instead, which the project presents as the reason it is more compatible across operating systems. It also deliberately avoids a hard librosa install by copying the single mel filter function it needs.

## Sources

- [Issues](https://github.com/KinWaiCheuk/nnAudio/issues)
- [KinWaiCheuk/nnAudio on GitHub](https://github.com/KinWaiCheuk/nnAudio)
- [License: MIT](https://github.com/KinWaiCheuk/nnAudio/blob/master/LICENSE)
- [README](https://github.com/KinWaiCheuk/nnAudio/blob/master/README.md)
- [Releases](https://github.com/KinWaiCheuk/nnAudio/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/kinwaicheuk-nnaudio