noise2noise-pytorch: an unofficial PyTorch port with documented deviations from the paper

The problem Noise2Noise solves, and who this repository is for

Noise2Noise, from Lehtinen et al. (2018), trains an image denoiser without ever seeing a clean target. The training signal comes from pairs of independently corrupted observations of the same scene. The network learns to map one noisy image to another, and because the noise in the target is zero-mean, the expected optimum is the clean image. That is the claim the paper makes, and this repository is one person's attempt to reproduce it in PyTorch. The README opens by calling itself an unofficial implementation, which sets the expectation correctly. This is not a library you import. It is a pair of scripts, train.py and test.py, plus a src directory containing the model, and you drive it from the command line. The audience is narrow: someone who has read the paper, wants a PyTorch version rather than the authors' TensorFlow code, and is comfortable reading the source when the README is incomplete. If you need a denoising component inside a production pipeline, nothing here is packaged for that. There are no releases, no versioned artifacts, and no homepage. The repository has a single MIT licence and a master branch.

What the training loop actually does with the noise parameters

The mechanism is visible in the command-line arguments. You pick a noise type with --noise-type and a magnitude with --noise-param, and the training script corrupts images on the fly. For gaussian, the README states that the noise parameter is the maximum standard deviation sigma. For poisson, it is the Poisson parameter lambda. For text, it is the approximate probability p that a pixel is covered by text. The loss is selected separately with --loss, and the README's examples pair l2 with gaussian and poisson, and l1 with text. Crops are taken at --crop-size, which is 64 in the training example and 256 in the test example. The model is a U-Net, and the README's known-issues section says that the U-Net has shared weights, meaning the same enc_conv is used in the decoder part. The README links to the specific line, src/unet.py line 82, and describes the fix as trivial while stating the author does not have time to make it. The README also says the model seems to perform well even with this error. That is an honest disclosure, but it also means the architecture you get is not the architecture the paper describes.

Getting the data in place: the COCO split the README recommends

The README does not ship a dataset. It suggests ImageNet, as the paper's authors used, but notes that any dataset will do, and then offers COCO 2017 as a smaller alternative. The validation set is described as 1 GB and can be split into train and valid. The commands create a data directory with train, valid and test subdirectories, download val2017.zip, unzip it, and then move the first 4200 files into train and the next 800 into valid. A second set of commands downloads test2017.zip and val2017.zip for the full roughly 7 GB version that more or less matches the paper, unzipping into train and valid respectively. Test images go into data/test manually, and the README says only a handful will do to visually inspect the denoiser. The layout matters because train.py takes --train-dir and --valid-dir, and test.py takes --data. If you point those at a different structure, the path resolvers in the code are what will fail, and the README explicitly warns that training and testing fail on Windows out of the box due to differences in os.path. On macOS or Linux, the commands are copy-pasteable as written.

Training and testing commands, and the flags that change behaviour

The README's Gaussian example is the most complete. It passes --train-dir ../data/train --train-size 1000, --valid-dir ../data/valid --valid-size 200, --ckpt-save-path ../ckpts, --nb-epochs 10, --batch-size 4, --loss l2, --noise-type gaussian, --noise-param 50, --crop-size 64, --plot-stats and --cuda. Several of those flags carry meaning beyond their names. By default the model trains with noisy targets; --clean-targets switches to clean targets, which turns the run into ordinary supervised denoising and is presumably there for comparison. CUDA is off unless you pass --cuda. Checkpoints are saved after every epoch automatically. --plot-stats saves plots alongside checkpoints. The Poisson and text examples in the README are abbreviated and, in the text case, use --loss l1 rather than l2. The test command loads a checkpoint with --load-ckpt, points at a directory with --data, repeats the noise type and parameter, sets --crop-size 256, and uses --show-output 3 to display three noisy/denoised/clean montages. Removing --show-output disables the display. Both scripts accept --h for the full argument list, and the repository includes examples/train.sh and examples/test.sh.

Two documented deviations from the paper, and one unresolved case

The known-issues section is the most useful part of the README because it tells you where the implementation and the paper disagree. The first is the shared-weight U-Net described above. The second concerns activations: the README says activation functions should be LeakyReLUs everywhere except the last layer, which should be ReLU, and that the current implementation and pretrained models assume the opposite because the author originally misread the paper. The README adds that using the correct version was reported to yield unstable training without batch norm, and that the author has not tested it. That is a real limitation with a real consequence: if you fix the activations, you may need batch norm, and the repository does not tell you how to add it. The third issue is Poisson noise. The README states it is unclear how to deal with Poisson noise since it is data-dependent and thus nonadditive, and directs readers to the official TensorFlow implementation to adapt properly. So the Poisson path exists in the CLI, but the README does not claim it is correct. If your noise model is Poisson, this repository is the wrong starting point.

Reported results and what they do not tell you

The README reports that the Gaussian model was trained for 100 epochs on a 2000/400 train/valid split, and the text model on 1000/200 because corrupting with text is slower. Both were trained on an old NVIDIA GTX 780. The results table gives peak signal-to-noise ratios for a single image, monarch. For Gaussian noise at sigma 25, the noisy input is 20.34 dB, the denoised output is 32.68 dB, and training with clean targets gives 32.49 dB. For text overlay at p 0.25, the noisy input is 15.07 dB, denoised is 28.10 dB, and clean targets give 27.79 dB. The interesting number is the comparison between noisy-target and clean-target training, and in both cases the noisy-target model edges out the clean-target model. That is consistent with the paper's claim, but it is one image and one training run per configuration. The README does not report variance across images, a test set size, or a wall-clock time. Treat these numbers as a sanity check that the code runs, not as a benchmark.

Where this sits next to the official TensorFlow implementation

The README itself names the alternative: NVlabs/noise2noise, the authors' TensorFlow implementation, cited for Poisson adaptation. The difference in approach is not just framework. The official repository is maintained by the paper's authors, so its noise models and architecture choices are the reference. This repository is a reimplementation by a third party, and its README documents two places where it knowingly departs from the paper. It also credits Yusuke Uchida's Keras implementation as a source of help, which makes it a port of a port in places. If your goal is to reproduce the paper's numbers, the official code is the safer reference. If your goal is to read and modify PyTorch code, this repository is smaller and easier to step through, and the known-issues list tells you exactly which lines to look at. The trade-off is that you inherit the deviations and the author's stated lack of time to fix them.

Dependencies, maintenance and the MIT licence

The README pins PyTorch 0.4.1, Torchvision 0.2.0, NumPy 1.14.2, Matplotlib 2.2.3, Pillow 5.2.0 and OpenEXR 1.3.0, and offers pip3 install --user -r requirements.txt to install the latest versions instead. That gap between pinned and latest is the maintenance cost you are accepting. The code was tested on Python 3.6.5 on macOS High Sierra and Ubuntu 16.04, and the README says it will fail with Python 2.7.x due to 3.6-specific functions. OpenEXR appears in the dependency list and the Monte Carlo rendering noise instructions are in a separate MonteCarlo.md file, so that path pulls in an extra dependency you may not need for Gaussian or text work. There are no releases, so there is no version to pin against; you take the master branch. The licence is MIT, which is permissive and compatible with commercial use, but the repository is unofficial and the README does not address the licensing of the paper's method or of the pretrained weights it mentions. That is a question for your own legal review, not something the README settles.

Editorial conclusion

Adopt this repository if you want a small, readable PyTorch codebase for experimenting with Noise2Noise on Gaussian or text-overlay corruption, and you are willing to work around the shared-encoder bug and the inverted activation order noted in the README. Do not adopt it if you need correct Poisson handling, Windows support, or a maintained dependency set; the README itself points to the official TensorFlow implementation for Poisson. Before training, verify three things: that your environment is Python 3.6.x or later and not 2.7.x, that your paths resolve on a POSIX filesystem, and that you have read src/unet.py line 82 to decide whether the shared enc_conv weights matter for your use case.

Official sources

1. Issues
2. joeylitalien/noise2noise-pytorch on GitHub
3. License: MIT
4. README