# Kandinsky 5: an open family of video and image diffusion models > Kandinsky 5 is a family of open weights diffusion models for text to video, image to video and image generation, built around a Diffusion Transformer with flow matching, a Qwen2.5-VL and CLIP text encoder and a HunyuanVideo 3D VAE. The repository ships the inference code, the model configurations, ComfyUI notes and a growing set of LoRA releases. **kandinskylab/kandinsky-5** — Kandinsky 5.0: A family of diffusion models for Video & Image generation - Repository: https://github.com/kandinskylab/kandinsky-5 - Website: https://kandinskylab.ai - Stars: 843 · Forks: 72 - Language: Python - License: MIT - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/kandinskylab-kandinsky-5 ## One family, several sizes and lengths The repository provides a family of diffusion models that generate a video or an image given a textual prompt, an image, or both, and the family is structured along two axes: how big the model is and how long the output is. At the top sits Kandinsky 5.0 Video Pro, a line-up of 19B models that generate high quality HD video from English and Russian prompts with controllable camera motion. It comes in several variants: an SFT model, which the README says delivers the highest generation quality, and a Pretrain model, designed for fine-tuning by researchers and enthusiasts. Every variant exists in a 5 second and a 10 second version, and there is additionally an image to video model that takes an input image and a text prompt. Below that is Video Lite, a smaller text to video model at 2B parameters, and Image Lite, the image generation line. The Lite checkpoints are the interesting ones for people with ordinary hardware, because the README records that after further VAE tiling optimisation and the addition of several attention backends, generation should now work on GPUs with 12 GB of memory. The checkpoint naming shows how the variants are assembled. There are pretrain, SFT and no-cfg checkpoints at 5 and 10 seconds, plus a distilled 16 step variant whose weights combine pretraining, supervised fine-tuning, configuration guidance distillation and diffusion distillation into sixteen steps. The 10 second checkpoints are described as faster models trained with the NABLA algorithm, which is where the extra length does not cost proportionally more time. Image editing, listed as its own section in the table of contents, is the fourth line of the family. ## The pipeline: DiT, flow matching and a 3D VAE The architecture section is short and specific, which makes it easy to see what the model actually is. The pipeline is a latent diffusion pipeline with flow matching. The main generative backbone is a Diffusion Transformer, a DiT, which conditions on text with cross-attention. Text embeddings come from Qwen2.5-VL and CLIP. Video is encoded and decoded into a latent space by the HunyuanVideo 3D VAE. So the chain is: a vision language model and CLIP turn the prompt into embeddings, the DiT generates in latent space under flow matching with the text as a cross-attention condition, and the 3D VAE decodes the latent sequence into video frames. Two choices in there are worth flagging for anyone comparing models. Using Qwen2.5-VL for text conditioning is a different choice from the CLIP-only text encoder used in many earlier text to video models, and it pairs with the report that an NF4 quantised version of Qwen2.5-VL from BitsandBytes is supported. The HunyuanVideo 3D VAE is likewise borrowed rather than built here. The model zoo table is where the practical numbers live. Each row gives a configuration file path, the video duration, the number of function evaluations, a link to the checkpoint on the Hugging Face hub and a latency figure. For example the T2V Pro SFT 5s HD entry points at configs/k5_pro_t2v_5s_sft_hd.yaml with 100 function evaluations and a listed latency of 1241, while the 10 second HD entry has its own configuration file and no latency recorded. The unit of that latency figure is not stated in the README, so treat it as a relative indicator rather than a specification. ## Memory, attention backends and the ComfyUI path The engineering effort in the project has gone mostly into making the models run on modest hardware, and the update log records that work step by step. The 2025-10-07 entry adds SDPA support so the code can run without Flash attention, and adds Magcache support for no-configuration checkpoints, extending it to SFT and no-cfg checkpoints. It also reports that memory consumption in the VAE has been reduced, with the entire pipeline now running at 24 GB with offloading. The 2025-10-19 entry goes further: further VAE tiling optimisation, support for the NF4 version of Qwen2.5-VL from BitsandBytes, and a choice of Flash Attention 2, Sage Attention or SDPA for five second generation through the attention engine option. The claim attached to it is that generation should now work on GPUs with 12 GB of memory. Those two entries tell the story of the project better than any benchmark would. The first wave of open video models needed serious hardware; this one has been walked down to a 12 GB card. For users who do not want to run the Python code, there is a ComfyUI directory with its own README, which was updated in the same period, and the models were accepted into the Hugging Face diffusers library, which means the pipelines are also reachable through the standard diffusers API rather than only through this repository. The requirements file pins the stack: torch 2.8.0 exactly, transformers at 4.56.2 or later, diffusers, accelerate, einops, imageio and imageio-ffmpeg, opencv-python, bitsandbytes, peft for the LoRA work, omegaconf for the configuration files, and av for video handling. ## LoRA: camera control and simplified training The most interesting release for people who want to control generation is the LoRA work, and it comes in two parts. On 2025-11-24 the project open sourced LoRAs for camera control, in a Lite set and a Pro set, with inference code in two example notebooks, one for image to video and one for text to video. Camera motion is one of the things closed video services expose as a control, and having it as a LoRA means it can be trained and shipped independently of the base weights. On 2025-11-26 a simplified LoRA training setup was published in a separate repository, kandinsky-5-lora-train, which suggests the authors expect fine-tuning to be a common activity rather than something the base repository has to carry. The peer review landscape has already produced third party tooling. The README lists community work, starting with SimpleTuner, which supports Kandinsky 5.0 LoRA and full rank training, and CacheDiT, which offers full cache acceleration with DBCache, TaylorSeer and Cache CFG, with a runnable example for the text to video pipeline. The repository also includes LoRA test scripts and example notebooks for every mode: text to image, image to image, text to video, image to video, and the two LoRA variants, plus an examples shell script that presumably chains them. ## How the code is laid out The repository is a research project rather than an installed package, and the layout reflects that. There is a kandinsky directory holding the model and pipeline code, a configs directory with one YAML file per checkpoint variant, an examples directory with the notebooks, a benchmark directory, and a comfyui directory with the integration notes. At the top level sit download_models.py for fetching the weights, test.py and test_lora.py, a requirements.txt, an assets directory and a paper.pdf holding the technical report. The configuration driven design is what makes the model zoo manageable. Every checkpoint in the table is a YAML file naming the parameters for that variant, and the same code runs all of them, so adding a new configuration does not mean new code. The technical report is linked from the header of the README with an arxiv identifier, and the project page and a Habr article are linked alongside it. The model weights themselves are hosted on the Hugging Face hub rather than in the repository, with separate collections for Video Lite, Video Pro and Image Lite, and separate collections for the LoRA sets. One licence note. The repository is MIT licensed, which covers the code in it. The weights are distributed through the Hugging Face hub under the terms set there, and those terms are what govern the models themselves, so read them before shipping output. ## Where Kandinsky 5 sits and what to check first The comparison that matters is against other open video models, and the README makes a claim in the update log that is worth repeating precisely rather than paraphrasing: on 2025-12-12 Kandinsky 5.0 Video Pro was ranked top one open source text to video model on the LMArena leaderboard. That is a leaderboard ranking of a hosted sampling service rather than a reproducible benchmark, but it is the clearest signal in the repository about how the model compares, and it is dated, so treat it as a snapshot. The practical difference between this family and a closed service is control and hosting. An open checkpoint can be fine tuned, quantised, cached and served on your own hardware, which is what the LoRA releases, the attention backend options and the 12 GB memory figure are all about. The cost is that you supply the GPU and the sampler time. Four things to verify before you start. Which checkpoint variant matches your card and your duration requirement, since the 5 second and 10 second models are separate weights. What the number of function evaluations in the model zoo table means for your latency budget, since the distilled 16 step checkpoints exist to cut that. Whether the ComfyUI path or the diffusers path fits your pipeline better than the raw repository code. And what the licence on the weights says about the use you have in mind. The last entry in the update log is 2025-12-12, the last push was on 2026-08-07, and the repository has no GitHub releases, so the version of the code you get is whatever commit you check out. ## Conclusion Kandinsky 5 is worth evaluating if you need an open weights video model you can fine tune and self host, particularly because the Lite checkpoints are small enough to run on 12 GB of memory and the camera control LoRAs target a control that closed models do not expose. It is the wrong choice if you need the highest quality generation and can use a hosted service, or if you cannot supply the GPU memory and the patience for a diffusion sampler. Verify first which checkpoint variant matches your hardware and your length requirement, because the 5 second and 10 second models are separate weights, and check the model zoo table for the number of function evaluations and the reported latency before you commit to a resolution. ## FAQ ### What models does the Kandinsky 5 repository provide? A family covering text to video and image to video with the 19B Kandinsky 5.0 Video Pro line, a 2B Video Lite text to video model, Image Lite for image generation, and a separate image editing line. ### What architecture do the Kandinsky 5 models use? A latent diffusion pipeline with flow matching, a Diffusion Transformer as the generative backbone with cross-attention to text embeddings, Qwen2.5-VL and CLIP for text embeddings, and the HunyuanVideo 3D VAE to encode and decode video into latent space. ### How much GPU memory does Kandinsky 5 need? The README records that after VAE tiling optimisation and support for Flash Attention 2, Sage Attention or SDPA, generation should work on GPUs with 12 GB of memory, and that with offloading the entire pipeline runs at 24 GB. ### Does Kandinsky 5 support LoRA fine-tuning and camera control? Yes. LoRAs for camera control were open sourced for both the Lite and Pro models with inference examples, and a simplified LoRA training setup was published in a separate kandinsky-5-lora-train repository. ## Sources - [Issues](https://github.com/kandinskylab/kandinsky-5/issues) - [kandinskylab/kandinsky-5 on GitHub](https://github.com/kandinskylab/kandinsky-5) - [License: MIT](https://github.com/kandinskylab/kandinsky-5/blob/main/LICENSE) - [Project website](https://kandinskylab.ai) - [README](https://github.com/kandinskylab/kandinsky-5/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/kandinskylab-kandinsky-5