Video DeltaNet makes video generation faster than playback by giving the backbone a second attention path

The headline number is a fourteen second clip in about nine

The claim is that this generates video faster than it plays. The specific measurement: a fourteen-and-a-half second clip end to end in roughly nine seconds on eight of a current data-center accelerator, of which about seven seconds is the denoising itself, across eight denoising steps.

Those numbers come with the context that makes or breaks them. They were measured with a specific serving stack, on eight accelerators, at eight steps. The same checkpoint is available through a second path that is more modest, and the documentation gives smaller GPU counts for that path rather than pretending one number covers both.

The step count has a subtlety worth catching, because it is the kind of thing that makes a benchmark mean something different from what you assume. The step parameter counts points on a noise schedule rather than model evaluations, so nine of those points is eight actual evaluations. A paper that reports a step count in one convention and a benchmark in the other will look faster or slower than it is, and this one is explicit about which it uses.

One linear branch added, one softmax branch kept

The architecture is described in one sentence and it is the whole idea. There is a frame-wise linear attention branch, which is cheap, and a softmax branch, which is what preserves the visual quality and frame-to-frame consistency of the original backbone. Both run. Neither replaces the other.

That is a meaningfully different design from the two obvious alternatives. Swapping softmax attention for linear attention everywhere gives you the speed and you lose consistency, and video is exactly the modality where temporal consistency is the thing users notice first. Replacing nothing gives you the original model's behaviour and none of the speedup. Keeping both, and paying for both, is a bet that the expensive branch can be kept small enough to afford.

The consistency argument is also why this is harder in video than in images. A still image that is wrong in one pixel is a still image. A video that flickers between frames is unusable, so any change to the attention pattern shows up immediately as temporal artifacts, and the softmax branch is what keeps that from happening.

The checkpoint is plug-and-play because nothing merges into the backbone

The checkpoint adds the linear attention branch plus two small adapters that can be merged into the backbone during inference, and the critical phrase is that this happens without touching the backbone weights. The base model is unmodified and the fast path is layered on top.

That is what makes it a checkpoint rather than a fork. If you can add the acceleration to a model you already have, the comparison against the unmodified model is one a reader can reproduce, and someone using a variant of the base model does not have to wait for someone to retrain it.

The project also states what is open. Not only the weights but the optimised inference stack and the training code that produced it, released together. That combination is rarer than it should be. A released weight with no training code is an inference-only curiosity; released training code with no serving integration is a paper reproduction. Both together is what lets someone check the claim rather than take it.

One checkpoint, four tasks, including reference images

The news log is the fastest way to understand the scope, because it shows what got added and when. The release came with a blog post, the training and inference code, and the weights. Two days later the same checkpoint added text-driven and first-last-frame conditioning. Three days after that, a reference-image task using the same weights as the first-last-frame path.

That last one is the interesting entry. Adding a new task to a video model usually means retraining, and here it meant routing reference images through weights trained for something else. The documentation frames it as a similar task rather than a new capability, which is the honest description and also the more useful one, since it tells you what the model considers equivalent.

The practical consequence is that you should not assume the tasks are equally supported. Text-driven and first-last-frame are the paths the serving stack provides natively, and the reference-image path is described as running through the other weights. If reference-to-video is what you need, check that path specifically rather than assuming it works because the other three do.

The single-card path is the number most people need

Set up is a clone and a pinned environment, and the pins are specific rather than approximate:

git clone https://github.com/OpenVDN/vdn-minimax-h3.git
cd vdn-minimax-h3

The runtime and the attention kernel version are named exactly, and the install command needs a flag to allow a pre-release dependency. Getting this wrong is the difference between a working install and a dependency resolver error.

Then the part that matters if you are not sitting on eight accelerators. On a card with twenty-four gigabytes, the transformer is streamed one block at a time instead of being loaded whole:

apply_group_offloading(pipe.transformer, onload_device="cuda", offload_type="block_level",
                       num_blocks_per_group=1, use_stream=True)

The documented result is that a long clip, three hundred and forty-five frames, peaks at twenty-two gigabytes, or twenty in reduced precision. That is the number to plan around. The eight-accelerator benchmark is the impressive figure, but the block-level streaming figure is the one that tells you whether you can run this at all, and it is the one the documentation spends a paragraph on.

The same model, two ways in, with the prompt advice given up front

There are two inference paths and they serve different purposes. The first goes through the model conversion library the checkpoints are published for, loaded as modular components with a named workflow, which is the shortest route to a first render and the one most people should take.

The second goes through a separate inference server that provides native serving for the text-driven and reference-image tasks, and is where the eight-accelerator configuration lives. The same checkpoint, so a change of path changes your throughput and your operational burden rather than your output.

Both paths take a prompt and optional first and last frame images, and the script form of the first path takes both keyframes as arguments, so a first-last-frame render needs no code.

python src/inference/infer_diffusers.py "a prompt" --out results/diffusers.mp4
python src/inference/infer_diffusers.py "a prompt" \
    --first prompts/image/first.png --last prompts/image/last.png

What stands out is a step that has nothing to do with code. The setup instructions tell you to read the upstream model's context documentation and its published prompt-writing guidance, and to rewrite your prompt with them before encoding. It claims this can greatly improve output quality. That is an admission that the prompt is doing real work in the result, which is true of every generative video model and is rarely stated as a setup step.

Licences are split, and the project asks you to read them first

The licence is not one file. The repository root carries both a licence file and a notice file, and there is a separate directory for third-party licences, which is the structure a project uses when its own code and its dependencies are under different terms.

The setup instructions put the licence read as step zero, before cloning and before downloading weights, with a note saying to read it before downloading or running the model. That ordering is unusual and deliberate. A model checkpoint carries a licence that may not be the one you would assume from the code licence, and discovering that after downloading thirty gigabytes is the wrong order.

The rest of the repository is the shape you would expect for a project like this: a configuration directory, a source directory, scripts, documentation, a directory of example prompts including first-frame images, assets, and a directory of patches applied to the model conversion library. There are no published releases, so versions are tracked by the model repository rather than by tags, and the weights are mirrored on a second model host as well as the primary one.

Editorial conclusion

Two claims to separate before you plan around this. The speed figure is measured on eight high-end accelerators with a serving stack the authors also maintain, so it does not transfer to a single consumer card, and the documentation is careful to say so by listing the hardware. The memory numbers do transfer, and they are the more useful claim: a fourteen second clip fits in twenty-two gigabytes, or twenty in reduced precision, on a card you can actually buy. The last thing to read is the licence, which the project asks you to read before downloading rather than after, and which separates its own code from third-party components.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. OpenVDN/vdn-minimax-h3 on GitHub
4. Project website
5. README