All-In-One Deflicker repairs video flicker with a learned atlas

Blind means the method does not know the flicker pattern in advance

The paper is Blind Video Deflickering by Neural Filtering with a Flawed Atlas, accepted at CVPR 2023, by Chenyang Lei, Xuanchi Ren, Zhaoxiang Zhang and Qifeng Chen, with the first two marked as equal contributors.

The word blind in the title is the whole point of the problem setting. Flicker in video is a periodic brightness variation, and a method that knows the flicker signal can simply divide it out. Here the method does not know the signal in advance, so it has to infer both what the underlying scene brightness should be and what the flicker is doing to it, from the video alone.

The title's second half names the mechanism. The method builds an atlas, a learned representation of the video's appearance, and it is described as flawed, meaning the atlas is not a perfect reconstruction and the filtering is designed to work with that imperfection rather than against it.

The neural filtering part is what makes it applicable to real footage rather than only to synthetic cases. The atlas is estimated by a neural network over the course of the video, and each frame is then filtered to remove the flicker while keeping the scene.

The repository contains inference code, a test script, a source directory, an environment file and a data directory, plus a demo.gif showing a slow-moving subject. The project is Apache-2.0 licensed, the last push was on 2026-07-14, and there are no GitHub releases, so the weights come from the separate download step rather than a release feed.

Environment setup is pinned, and the weights come from a second repository

The provided environment is Python 3.10 with torch 1.12 and CUDA 11, and for a torch 1.6 with CUDA 10 setup there is an alternative environment file linked at a specific commit.

Creating it is two commands:

conda env create -f environment.yml 
conda activate deflicker

The pretrained checkpoint is fetched from a separate repository, and the sequence is a clone followed by a move and a cleanup:

git clone https://github.com/ChenyangLEI/cvpr2023_deflicker_public_folder
mv cvpr2023_deflicker_public_folder/pretrained_weights ./ && rm -r cvpr2023_deflicker_public_folder

That arrangement is a little unusual and worth understanding. The weights are not in this repository, and they are not attached to a release, so you clone a public folder repository, take the pretrained_weights directory out of it, and delete the rest. The repository exists to host the weights because binary files do not belong in the source tree.

The pinned versions also tell you something about the age of the project. Torch 1.12 with CUDA 11 is a 2022-era stack, and the alternative with torch 1.6 and CUDA 10 is older still. If you are on a current machine, expect friction: the environment file will want a CUDA 11 toolkit, and torch 1.12 predates a lot of the newer Python packaging work.

The install instructions also assume a working conda installation and a git clone over SSH-style HTTPS, so you need network access to GitHub for both steps.

Inference takes a video or a folder of frames

The input goes under `data/test`, and the two supported input shapes are a video file and a folder of images.

The commands need the repository root on the Python path:

export PYTHONPATH=$PWD
python test.py --video_name data/test/Winter_Scenes_in_Holland.mp4 # for video input
python test.py --video_frame_folder data/test/Winter_Scenes_in_Holland # for image folder input

The result lands under `results/$YOUR_DATA_NAME/final/output.mp4`, so the output is a single video file regardless of whether the input was a video or a folder of frames.

The resource note is the one constraint worth internalising. The inference code only takes about 3000M of GPU memory, which is stated as a fact rather than a warning, and it is a modest figure for a method of this kind. A 3 GB GPU footprint means this runs on a workstation card without a memory-saving mode.

The two input modes are worth comparing. A folder of frames skips the video decode and encode entirely, which is useful when you have already extracted frames, and when the source is a sequence that was never in a container. A video file goes through the whole pipeline.

The script name is `test.py` rather than something like `infer.py`, which reflects that this repository is the inference half of a research project. There is no training entry point described.

The list of flickering content is broader than synthetic video

The section listing all evaluated types of flickering videos is the most useful part of the README for anyone deciding whether this is the right tool.

The synthesised cases come from text-to-video algorithms, with Magic Video and Make-A-Video named and linked. This is the case that motivated much of the recent interest in deflickering, because generative video models produce temporal brightness artefacts that look like flicker.

But the evaluated set is much wider than that. Old movies. Old cartoons. Time-lapse videos. Slow-motion videos.

And then a category that is easy to miss: videos processed by image processing algorithms. Style transfer, with a UniversalStyleTransfer implementation linked. Image Enhancement, with the MIT CSAIL hdrnet project. Intrinsic Image Decomposition, with the OpenSurfaces publication linked. Image-to-Image Translation and Colorization, both pointing at the pytorch CycleGAN and pix2pix repository.

That last category is the interesting one, because it explains what the blind in the title has to cope with. A per-frame style transfer or colorization pipeline will apply a slightly different transform to each frame depending on the input, and that inconsistency reads as flicker. The same is true of anything frame-by-frame. The method was therefore designed against flicker that comes from a per-frame processing error rather than from a single periodic signal.

Old films and cartoons are the same story from the other direction, since age-related brightness variation in scanned footage is exactly a signal the method must separate from real scene changes.

Segmentation masks improve the atlas for videos with a subject

The advanced feature is segmentation mask support, and it exists for a specific reason: the atlas improves for videos featuring a salient object or human.

Two mask sources are supported. Carvekit, for background removal, and Mask-RCNN from detectron2. The Carvekit route is a clone plus the standard invocation:

git clone https://github.com/OPHoperHPO/image-background-remove-tool.git
export PYTHONPATH=$PWD
python test.py --video_name data/test/Winter_Scenes_in_Holland.mp4 --class_name portrait # portrait triggers Carvekit

The Mask-RCNN route installs detectron2 from git and passes a class name:

python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
export PYTHONPATH=$PWD
python test.py --video_name data/test/Winter_Scenes_in_Holland.mp4 --class_name anything # actually not work for this video 

The `--class_name` argument determines the COCO class name of the sought foreground object, and using `anything` chooses the first instance retrieved by Mask-RCNN rather than a named class. The comment in the example is telling, since the sample video does not work with that setting.

Two limits are stated plainly. The current implementation supports only one foreground object with a background, so it is a single-subject tool. And you are advised to check the generated masks under `data/test/{vid_name}_seg`, because if the images are all black you can only use the non-segmentation implementation.

That check is worth doing before you spend time on the rest of the pipeline, since a black mask set means the whole mask path produced nothing usable.

Two hyperparameters are marked important and the rest is a guide

The section on choosing hyperparameters is unusually candid for a research release, and it reads as a warning that the defaults will not fit your video.

The iteration number is marked important. It should be changed according to the total frame number of the video and the downsample rate of the image size. The example is 10000 iterations for a video with 80 frames and a downsample rate of 4. If the results are not as expected, you can try to increase `iters_num`, for example to 100000, and if you use the segmentation mask implementation, increasing it is suggested.

The optical flow loss weight is also marked important. `optical_flow_coeff` and `alpha_flow_factor` should be set according to the intensity of flicker, with `alpha_flow_factor` only used in the advanced features with segmentation masks. The sample video uses 500.0 and 4900.0, and a video with minor flickering can use 5.0 and 49.0.

The downsample rate is third on the list. Downsampling the resolution of the neural atlas by 4 times makes the convergence much faster and slightly influences the quality, and you can choose your own rate.

The fourth item is the maximum number of frames.

The pattern across all four is that the settings are relative to the input rather than absolute, and the two marked important are the two that change the character of the result rather than its speed. That is the honest description of a method with a learned atlas: the atlas quality is what determines whether the deflickering works, and atlas quality is a function of how long you optimise and how much you weight motion consistency.

A tenfold difference between minor and severe flicker in the recommended weights shows how much this matters.

Editorial conclusion

Adopt All-In-One Deflicker if you have videos with brightness flicker and need them repaired in one pass, since the method handles both synthesised flicker from text-to-video models and real footage from old films and cartoons. Skip it if you need to train the method yourself, because the repository ships inference code and pretrained weights rather than a training pipeline. Verify first that your environment matches Python 3.10 with torch 1.12 and CUDA 11, that you can free around 3000M of GPU memory, and that you are prepared to tune the iteration count and optical flow loss weight, since the README marks both as important and gives no automatic setting.

Frequently asked questions

Official sources

1. ChenyangLEI/All-In-One-Deflicker on GitHub
2. Issues
3. License: Apache-2.0
4. README