SoulX-FlashHead: streaming talking heads at 96 frames per second

What the framework is for

SoulX-FlashHead is described as a unified 1.3B-parameter framework designed for high-fidelity, infinite-length, and real-time streaming portrait video generation. The title of the technical report is more specific still, oracle-guided generation of infinite real-time streaming talking heads.

Three words in that description carry the weight. Infinite-length means the output is not a fixed clip that has to be cut, which is the usual failure mode of a generative video model used for anything but a short loop. Real-time streaming means it has to run at frame rate while a session is happening, not offline overnight. And high-fidelity is the part everybody asks for and few systems deliver at the same time.

The 1.3B parameter count is the other interesting number. A small model that can be run at interactive rates on consumer hardware is a different proposition from a research-scale system that needs a cluster, and it is what makes the two-tier release below sensible.

The project page, the technical report on arXiv, the model weights and a dataset are all published, and the dataset has its own name, VividHead. The authorship is a long list with equal contribution and corresponding author markers, which is the usual shape for a paper with this many contributors.

Two released models, two different speed envelopes

The highlights section is where the engineering trade-off is stated, and it is stated in frames per second on named hardware.

Model_Lite, which is released, gets 96 FPS, or three-concurrent real-time streaming at 25 or more FPS on a single RTX 4090. Model_Pro, also released, generates high-quality videos at 10.8 FPS on a single RTX 4090, or real-time at 25 or more FPS on two RTX 5090 cards. Model_Pretrained is coming soon, and is described as providing high-performance weights and experimental foundations for community research.

The pattern is a two-speed product. The lite model is the one for a service with several concurrent users, since three sessions at real time on a single card is a plausible deployment. The pro model is the one for a single high-quality stream, and it buys that quality by needing a second card to reach real time, or by accepting a non-real-time rate on one card.

The number to keep in mind is that the pro model at 10.8 FPS is not real time on a single card. That is an honest constraint of a 1.3B model doing high-fidelity video, and the fact that the README states it in frames per second rather than in adjectives makes the comparison with the lite model straightforward.

The two-model structure also implies a sensible deployment pattern: a fleet of lite sessions for the common case, with the pro model reserved for the one stream that matters, and the multi-GPU inference script in the repository for the pro path.

The release calendar, which is short and dense

The news section covers about five weeks, and it is worth reading as a release plan rather than as a changelog.

On 2026-02-07 the dataset and the technical report went out, and two days later the project page, the model weights and the inference code. The same day, an online demo became available through the Soul App. A ComfyUI node arrived on 2026-03-02, contributed by HM-RunningHub, and a Gradio app on 2026-03-04 with both common and streaming mode, followed by a streaming online demo on Hugging Face on 2026-03-09.

The order matters. The paper and the data came first, then the weights and the code, then the community integrations, and the community integrations came from outside the project, which is a good sign for a codebase that other people can build on.

The todo list at the end of the README is short and mostly ticked. The technical report, the project page, the inference code, the streaming online demo and the distilled checkpoint of the pro and lite releases are all done. The only outstanding item is the pretrained checkpoint release.

So the missing piece is the one thing a researcher would most want, the full pretrained weights, which the README says will provide high-performance weights and experimental foundations for community research.

The environment is pinned to specific versions

The quickstart is a conda environment, a specific PyTorch build, the requirements file, and two attention libraries, and the pins are exact, which tells you how much of this is tested against one configuration.

conda create -n flashhead python=3.10
conda activate flashhead
pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt

Then FlashAttention, which is the step people hit, because it compiles:

pip install ninja
pip install flash_attn==2.8.0.post2 --no-build-isolation

and the README's advice for when that takes too long is to download the prebuilt wheel from the flash-attention releases page instead. SageAttention is optional and installed the same way, with the same no-build-isolation flag. FFmpeg comes from the system package manager, or from conda-forge if you would rather not need root.

The requirements file is where the shape of the stack becomes clear. There is diffusers and transformers at 4.57.3, accelerate, xformers at 0.0.31, xfuser for distributed inference, decord and imageio for video, opencv in both variants, librosa and mediapipe for audio and face landmarks, pyloudnorm for loudness, scikit-image, loguru for logging, gradio at 5.50.0, flask, and nvidia-nccl-cu12 for the multi-GPU path.

The absence of a Windows path is implicit in all of that, and the presence of NCCL and the two attention libraries is a clear statement that this is a Linux and NVIDIA box.

Two apps, one node, and a dataset

The repository is small and the entry points are obvious from the file names. There is generate_video.py, gradio_app.py and gradio_app_streaming.py, and three inference scripts: inference_script_single_gpu_lite.sh, inference_script_single_gpu_pro.sh and inference_script_multi_gpu_pro.sh. The flash_head/ directory holds the model code, examples/ holds a still image and a 16 kHz audio sample, and assets/ holds the report and project material.

The two Gradio applications map onto the two modes the README describes, a common mode and a streaming mode, and the separate shell scripts map onto the three deployment shapes, lite on one card, pro on one card, and pro across several. That is a tidy structure, and it means a user picks a script rather than a configuration.

The examples are minimal and instructive. A portrait image and a podcast audio clip named for a Sichuan dialect at 16 kHz, which is a common sample rate for speech models, are enough to reproduce the pipeline. That the sample is a specific dialect is also a hint about the training data, and the dataset, VividHead, is the other half of that picture.

The dataset being published alongside the model is a meaningful choice. For a talking-head model, the data is the thing that determines the range of appearances, expressions and voices the model covers, and publishing it means the community can study and extend it rather than only consume the weights.

Where SoulX-FlashHead is the wrong choice

The first wrong-tool case is a workload that is not a talking head. This framework is built for portrait video driven by audio, with infinite length and real-time streaming as the two design constraints. Generation of scenes, objects, or multi-character video is a different research problem, and nothing in the README suggests those cases are covered.

The second is a machine without a recent NVIDIA GPU. The whole environment is pinned to a CUDA 12.8 PyTorch build, with FlashAttention, SageAttention and NCCL in the dependency list, and the performance numbers are quoted on a 4090 and a 5090. A team that has to run on AMD hardware or on a CPU is not in the supported set at all, and the README does not describe a fallback.

The third is a product that needs a licence it can point at. The repository is Apache-2.0, but the repository licence and the licence on the weights are different documents, and the README here does not restate the terms of the model release. For a commercial avatar product, that distinction is the one to settle before anything else, and it is a question for the model card on Hugging Face rather than for this page.

The fourth is a team that needs a stable dependency. There are no GitHub releases, the environment is pinned hard enough that a change in any of those versions is a support incident, and the pretrained checkpoint that a researcher would want is still unticked in the todo list.

Editorial conclusion

Adopt SoulX-FlashHead if you are building a real-time avatar or talking-head product and you can afford the CUDA stack it pins, starting with the lite model on a single card and only moving to the pro model when quality matters more than concurrency. Do not adopt it if you need a guaranteed licence for commercial use of generated likenesses, since the model licence is separate from the repository licence and the README here does not restate its terms. Before you deploy, read the technical report for the method, check the licence on the model weights rather than on this repository, and expect to build the environment from scratch, because the pins for PyTorch, transformers, xformers, FlashAttention and NCCL are exact.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. Soul-AILab/SoulX-FlashHead on GitHub