# ConsisID: keeping a person's identity stable in generated video > ConsisID is a tuning-free, DiT-based controllable image-plus-text-to-video model from Peking University's Yuan Group, accepted as a CVPR 2025 highlight. Its approach decomposes the reference image and the generated frames in the frequency domain so that identity survives the diffusion process, and the repository ships training and inference code, evaluation tooling and a set of preview data on Hugging Face. **PKU-YuanGroup/ConsisID** — [CVPR 2025 Highlight🔥] Identity-Preserving Text-to-Video Generation by Frequency Decomposition - Repository: https://github.com/PKU-YuanGroup/ConsisID - Website: https://pku-yuangroup.github.io/ConsisID/ - Stars: 863 · Forks: 48 - Language: Python - License: Apache-2.0 - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/pku-yuangroup-consisid ## What ConsisID is and what the paper claims ConsisID is the official implementation of a tuning-free, DiT-based controllable image-plus-text-to-video model whose purpose is to keep human identity consistent in the generated video. The README states plainly that the approach draws inspiration from previous studies on frequency analysis of vision and diffusion transformers. The problem it addresses is the one every image-to-video system has. A text-to-video model given a prompt produces a plausible person, but that person drifts: the face changes between frames, the hairstyle shifts, the age moves. It produces a sequence of similar-looking strangers rather than one person. For anything involving a recognisable individual, that is a failure rather than a stylistic choice. The paper title is the identity-preserving text-to-video generation by frequency decomposition, and the frequency decomposition is the technical core. The intuition is that identity information in an image lives disproportionately in the high-frequency components, the fine structure of a face, while the low-frequency components carry the broad shape and the motion. Separating those bands and treating them differently through the diffusion process lets the high-frequency identity signal survive the generation where a plain model would wash it out. The work was accepted at CVPR 2025 as a highlight, and the technical report on arXiv is 2411.17440, with a third version uploaded on 2025-03-27. The project page is at pku-yuangroup.github.io/ConsisID, and the group is the Yuan Group at Peking University. A tuning-free model matters for adoption in a specific way. If the method needed per-subject fine-tuning to hold an identity, every new face would mean a training run, and the method would be unusable at scale. Tuning-free means the model is used as published. ## The repository layout: training, inference, evaluation and data prep The tree shows a complete research pipeline rather than a single demo. At the top level are train.py for training and infer.py for inference, app.py which is the Gradio application, and train_single_rank.sh and train_multi_rank.sh for the distributed launch scripts. Then there are directories for the model definitions in models/, the evaluation code in eval/, the data preprocessing in data_preprocess/, helper utilities in util/, tools/ for scripts and an asserts/ directory holding the project logo. That layout is the honest description of what this repository is. It is a training codebase with an inference path attached, not a turnkey application. Someone who wants to generate a video with a specific person will spend their time in infer.py and in getting the weights, and someone who wants to adapt the method will be in train.py and the data pipeline. The data preprocessing directory is worth calling out. Identity-preserving video generation needs a reference image per subject and a set of videos of that subject, and the preprocessing is what pairs them and resizes and encodes them into the format the model consumes. The dependency list shows what that involves: decord for video decoding, opencv-python, insightface at 0.7.3 for face detection and recognition, and facexlib for face restoration. Those two face libraries are the sign that the identity handling runs through an explicit face pipeline rather than through the diffusion model alone. There is a preview dataset on Hugging Face, linked from the README, and a Hugging Face Space for trying the model, plus a Colab notebook hosted by a third party. Those are the entry points for someone who wants to see the behaviour before installing anything. ## The dependency stack and what it pins The requirements file is long and pinned exactly, which is the norm for a research codebase that was made to work on a specific set of versions. The deep learning stack is torch 2.5.1 with matching torchaudio 2.5.1 and torchvision 0.20.1, xformers at 0.0.28.post3 for the memory-efficient attention paths that video diffusion needs, and diffusers at 0.33.1 with transformers 4.46.3 and tokenizers 0.20.1. Deepspeed 0.15.2 and accelerate 1.1.1 are the distributed training and mixed-precision tools, and peft at 0.12.0 is there for parameter-efficient fine-tuning, which fits a model that is meant to work tuning-free but still needs the capability available. The video and image side has decord 0.6.0 for decoding, opencv-python 4.10.0.84, scikit-video at 1.1.11 and moviepy at 2.1.1 for writing output, pillow 10.4.0, and imageio-ffmpeg for the encoder. The identity side is the interesting group. insightface 0.7.3 and facexlib 0.3.0 handle face detection, recognition and restoration, pyfacer at 0.0.4 is a further face utility, and consisid_eva_clip is listed as a local package without a version pin, which suggests it is vendored or installed from a source distribution rather than from an index. The serving and tooling layer is gradio 5.6.0 with spaces 0.30.4 for the app and the Space, tensorboard 2.17.1 for logging, wandb for experiment tracking, and a set of smaller utilities: numba 0.60.0, numpy 1.26.4, timm 1.0.9 for vision backbone weights, einops for tensor reshaping, sentencepiece, ftfy for text cleanup, onnx and onnxruntime-gpu for an ONNX export path, nvitop for GPU monitoring during long runs, spandrel for loading upscaling models, and func_timeout for bounding calls that could hang. The pin set is the thing to check before anything else. xformers in particular is tightly coupled to the torch version, and diffusers 0.33.1 is specific enough that a newer release will not simply drop in. ## The group around the project: datasets, benchmarks and other models The README spends a long section on the surrounding work, and reading it tells you how the dataset and evaluation ecosystem around ConsisID is organised. There is OpenS2V-Nexus, described as a detailed benchmark and million-scale dataset for subject-to-video generation. It consists of two parts: OpenS2V-Eval, a fine-grained benchmark hosted as a Hugging Face Space, and OpenS2V-5M, a million-scale dataset on Hugging Face. That release is dated 2025-05-27 in the news list, and it directly follows the ConsisID acceptance. There is also ChronoMagic-Bench, a benchmark for metamorphic evaluation of text-to-time-lapse video generation, and MagicTime, time-lapse video generation models presented as metamorphic simulators. The most directly useful entry for someone using ConsisID is the DanceGRPO collaboration, dated 2025-08-30, which transferred ConsisID data for image-to-video reinforcement learning training. The README credits that work and points at its own documentation for details, and it notes that OpenS2V-5M can be tried as an alternative for RL training. That is a specific, actionable piece of information: if you want to train a model that keeps identity under reinforcement learning, there is a documented path from the ConsisID data. There is also Helios, introduced in March 2026, described as a video generation model achieving minute-scale, high-quality video synthesis at 19.5 FPS on a single H100 GPU, without relying on conventional long video anti-drifting strategies or standard video acceleration techniques. And Open-Sora Plan, a separate large video generation model. This is a group publishing several related projects rather than a single model, and the pattern is coherent. ConsisID is the identity-preserving model, OpenS2V-Nexus is the data and benchmark for it, ChronoMagic-Bench is the evaluation harness, and Helios is the separate answer to the length problem. ## Limitations: a research codebase, a stale branch, and no release history The limits here are more substantial than in most of these projects, and they are visible in the repository metadata. The first is the maintenance state. The last push to the repository was on 2026-04-14, which is more than six months before the present date, and the repository has no GitHub releases at all. The news list ends with the Helios announcement in March 2026, and the entries before that run from February 2025 to August 2025. There is no tag to pin and no release to compare against, so the version you get is whatever is on the branch when you clone it. The second is that this is a research codebase with a long pinned dependency list. The versions in requirements.txt are exact, down to the patch, and several of them are old enough that they will not install cleanly against a current CUDA or a current Python. xformers in particular is compiled against a specific torch build, and building it yourself is a known irritation. There is no install section in the visible README, no container and no environment file, which for a project of this size means the setup burden is on the reader. The third is the hardware. A video diffusion model with a DiT backbone is not a laptop workload, and the presence of deepspeed, onnxruntime-gpu and nvitop in the requirements is a reasonable proxy for the size of the machines involved. The README's own preview Space and Colab notebook exist precisely because running the thing locally is not trivial. The fourth is the nature of the claim. A tuning-free identity-preserving model is evaluated on whether the face looks like the reference across frames, which is a perceptual judgement, and the preview dataset and the Space let you judge it by eye. That is the honest test, but it is not a benchmark with a number, and the benchmark in the group's portfolio, OpenS2V-Eval, is a separate artefact rather than something this repository runs. The fifth is scope. ConsisID keeps identity consistent. It does not add new capability, control camera movement, or produce long-form video; the minute-scale generation the group demonstrates is Helios, a different model in a different repository. ## ConsisID against a general image-to-video model and against per-subject fine-tuning Two comparisons make the choice clearer. The first is against a general image-to-video model. Given a reference image and a prompt, a general model produces a video that is thematically right and facially inconsistent, because nothing in its training objective asks for the identity to be preserved. The difference in approach is that ConsisID decomposes the reference and the generated frames in the frequency domain and uses that separation to keep the identity signal alive through the diffusion process. That is a change to what the model optimises, not a wrapper around an existing one. The cost is that ConsisID is a single-purpose model. It does one thing, identity, and it does it at the expense of everything else the general model was also good at. If you need scene generation, camera control and long duration as well as a consistent face, you are choosing between them. The second comparison is against the alternative approach to identity preservation, which is per-subject fine-tuning. LoRA-style adaptation of a video model on a particular person is a well-trodden technique and it produces very strong identity adherence for that subject. The difference in approach is entirely about scale and about what you are committing to. Fine-tuning needs training data for the subject, compute, and an artefact per subject. If you need one video of one person, a fine-tune gives a better result than a tuning-free method, because the model has actually seen that person. ConsisID's tuning-free claim is that a single model, applied to a reference image at inference time, holds identity without any per-subject work, which is what makes it usable when the subjects are unknown in advance, when you need many identities, or when you cannot collect training footage of the person you want to depict. That is the decision. One known subject with data available, fine-tune. Many subjects, no data, or a product where the reference image is all you have, use the tuning-free method. ## Conclusion Adopt ConsisID if you need generated video in which a specific person's appearance stays consistent across frames and shots, and you are willing to work with a research codebase pinned to a particular torch and diffusers stack. Do not expect a packaged tool: the repository is a training and inference codebase with a long pinned dependency list. Verify first that you can install the exact versions in requirements.txt, that your GPU has enough memory for the model you intend to run, and that the weights and preview data you need are available on Hugging Face before you commit to a training run. ## FAQ ### What is ConsisID and how does it preserve identity in generated video? ConsisID is a tuning-free, DiT-based controllable image-plus-text-to-video model that keeps human identity consistent in the generated video. The README says the approach draws inspiration from previous studies on frequency analysis of vision and diffusion transformers, and the paper title describes the mechanism as frequency decomposition. ### Which Python and machine learning versions does ConsisID require? The requirements file pins torch 2.5.1 with matching torchaudio and torchvision 0.20.1, xformers 0.0.28.post3, diffusers 0.33.1, transformers 4.46.3, deepspeed 0.15.2 and accelerate 1.1.1. The face stack is insightface 0.7.3 and facexlib 0.3.0, and the app layer is gradio 5.6.0. ### Where can I try ConsisID without installing it? The README links a Hugging Face Space preview for ConsisID, a Colab notebook hosted by a third party, and a preview dataset on Hugging Face. A paper page is also linked on Hugging Face for the arXiv entry 2411.17440. ## Sources - [Issues](https://github.com/PKU-YuanGroup/ConsisID/issues) - [License: Apache-2.0](https://github.com/PKU-YuanGroup/ConsisID/blob/main/LICENSE) - [PKU-YuanGroup/ConsisID on GitHub](https://github.com/PKU-YuanGroup/ConsisID) - [Project website](https://pku-yuangroup.github.io/ConsisID/) - [README](https://github.com/PKU-YuanGroup/ConsisID/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/pku-yuangroup-consisid