DCVC-UF: chunk-based neural video compression for ultra-fast encode and decode

Chunks, not frames, are the unit of coding

The core idea in the paper abstract is a change in what gets encoded. Rather than processing frames sequentially, the approach encodes a chunk of multiple frames into a single compact latent representation and decodes them simultaneously. Two mechanisms make that work. Cross-frame interaction modules handle joint spatial-temporal modelling, so the information a frame carries is used when representing the frames around it. Frame-specific decoders then reconstruct in parallel, which is where the throughput comes from, since a chunk can be decoded as parallel work rather than a chain of dependencies. The abstract also names a second speed lever: a streamlined entropy coding mechanism that consolidates bit-stream interactions into a single step, reducing decoding overhead. The stated motivation is that neural video codecs have demonstrated superior compression ratio while their computational complexity remains a barrier to real-world deployment, and the chunk framing is the answer to that barrier rather than to the compression ratio itself.

One model, 64 quantization levels, adjustable bitrate

The practical features listed in the README matter more for deployment than the architecture does. A single model with 64 QP levels enables continuous and fine-grained bitrate adjustments across a wide bitrate range, which removes the usual need to train a separate model per quality point. Rate control falls out of the same mechanism, since adjusting the quantization parameters lets the codec react to dynamic and varying network conditions during a real communication scenario. The third feature is unified YUV and RGB coding. The codec is primarily optimized for the widely adopted YUV420 format, and the README says it can adapt to RGB content coding. That combination, one model covering a wide range with a rate control knob and one path for both colour formats, is what separates a research artifact from something you could put in a live video call. The README also states that additional practical functionality is being explored and that further NVC solutions will be provided in the repository.

The install path is a conda environment plus a CUTLASS checkout

The prerequisites are Python 3.12 and conda, CUDA 13.0, and PyTorch, where the README notes that pytorch-2.9.1 has been tested and that other versions may work. The environment is set up like this:

    conda create -n $YOUR_PY_ENV_NAME python=3.12
    conda activate $YOUR_PY_ENV_NAME

    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130
    pip install -r requirements.txt

The requirements file pulls in the usual scientific stack, including numpy, scipy, pillow, matplotlib, tqdm, cmake, ninja and pybind11, plus bd-metric for quality evaluation and psutil for process measurement. Then the project has to be built, and this is the step that separates it from a pip install. Two C++/CUDA extensions must be compiled: MLCodec_extensions_cpp, which is the rANS entropy coder built on pybind11, and inference_extensions_cuda, which holds the fused inference kernels and is CUTLASS-based. The build pulls a pinned CUTLASS:

git clone https://github.com/NVIDIA/cutlass third_party/cutlass
cd third_party/cutlass
git checkout v4.4.1
cd ../../src/cpp/
bash install.sh
cd ../layers/extensions/inference/
bash install.sh

The pinned v4.4.1 tag matters, because the fused kernels are written against a specific CUTLASS version.

Entropy coding on the CPU can become the bottleneck

The README has a section on CPU performance scaling that reads like a troubleshooting note, which tells you it is a real problem in practice. The arithmetic coding runs on the CPU, and if the CPU is not running at maximum frequency while encoding or decoding actual bitstreams, the entropy coding may bottleneck throughput. The governor can also reset after each reboot, so a machine tuned once is not necessarily tuned later. You can check the current frequency with:

grep -E '^model name|^cpu MHz' /proc/cpuinfo

And switch the governor to performance mode with:

echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

To put it back:

echo ondemand | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

This is a Linux-specific path, using procfs and the cpufreq driver, and it requires root. On a managed instance or a container with a restricted sysfs, these commands may not be available, in which case the CPU side of the codec becomes an unpredictable component in your latency budget.

Pretrained models arrive as a OneDrive folder, not a registry

Model distribution is a place where this project is harder to use than it should be. The README points to a download link for the pretrained models, and asks you to put them into the ./checkpoints folder. There are four checkpoints: one for image coding and three for video coding, named HT-L, HT-S and LD variants. A backup link is given as well, described as a place where all the pretrained models could also be found. There is no mention of a Hugging Face repository, a torch hub entry or an index in requirements.txt, so the weights are distributed as files from a hosted drive. The repository does contain a checkpoints/ directory, which is where they are expected to land. The naming of the three video variants tells you the model family covers different latency and complexity targets, but the README does not document which to pick for a given deployment, so that choice is yours to benchmark.

The reported numbers are the paper's, measured on specific hardware

The performance claims in the README should be read as research results with a stated setup rather than as a product specification. Bitrate savings are given against VTM-17.0 on the UVG dataset, in an all-frames, single intra-frame setting with intra-period set to -1, in YUV420 colourspace, and the comparisons are presented as BD-Rate curves. Speed figures for 1080p encoding and decoding are reported on an NVIDIA 4090 GPU, and there is a separate complexity analysis across various resolutions and devices. The intra-frame codec is the one with concrete numbers in the text: on Kodak, DCVC-UF-Intra achieves a 10.6% bitrate reduction compared to VTM, and for 1080p content the intra codec reaches 81.5 FPS for encoding and 95.0 FPS for decoding on an NVIDIA A100 GPU. The comparison baseline is VTM, a traditional codec, and the neighbouring charts compare against learned image codecs. Whether any of this transfers to your content depends on the dataset, and the README does not include a quality target list for arbitrary footage.

Training code arrived after the inference code

The repository is the official implementation of DCVC-UF and the wider DCVC-family models, and the project page for the family lives under a separate DCVC-family directory with its own README. The main README carries a note that training code has been released, pointing to a training.md file, and the top-level entries confirm it with train_image.py and train_video.py sitting at the root. The paper abstract had said both training and testing codes would be released, so the training scripts fulfil part of that statement. The MIT licence is recorded in LICENSE.txt with a NOTICE.txt alongside it, and the copyright holders of any third-party material are recorded in the notice file. On maintenance, the repository is not archived and the last push was on 2026-08-25, and there are no GitHub releases, so there is no tagged artefact to pin. The tree also carries azure-pipelines.yml, a flake8 configuration, test_cfg for test configurations, and helper scripts such as compare_bd_rate.py and test_compress_time.py that are aimed at reproducing the reported evaluations.

Editorial conclusion

Adopt DCVC-UF if you are building a real-time video pipeline with NVIDIA GPUs, you can live with a CUDA and CUTLASS build step, and you want learned compression with a single model spanning a wide bitrate range. Do not adopt it on CPU-only hardware, in a container without a GPU, or if you need reproducible offline builds, because the arithmetic coding runs on the CPU and the README warns that a throttled clock will bottleneck entropy coding throughput. Before you commit, build both extensions with the CUTLASS checkout at v4.4.1, download the four checkpoints into ./checkpoints, and check the training.md guide, since the training code was released after the inference code.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. microsoft/DCVC on GitHub
4. README