LTX-2

LTX-2 pairs synchronized audio and video generation in one DiT-based model, and this repository is the official Python inference and LoRA trainer package for it.

The README makes a first-of-its-kind claim: LTX-2 is the first DiT-based audio-video foundation model that folds modern video generation into one model. That includes synchronized audio and video, high fidelity, multiple performance modes, production-ready output, API access, and open access. The repo is the official Python inference and LoRA trainer package, written in Python, with the homepage at ltx.io. The license field in the metadata is listed as NOASSERTION.

Setup starts with dependencies, and the details get heavy quickly. The natten extra is the fastest backend for the diffusion video VAE, but it is Linux and CUDA only, so Windows and macOS skip it and fall back to a Triton or eager implementation. The download is roughly 66 GiB, and the CLI keeps the repository folder layout under a local directory, which is why paths in the docs include folders like diffusion models and vae. If Hugging Face returns a 401 or 403, the fix is to accept the model terms and log in with a read token, with fine-grained tokens needing the read gated repos scope. Under GPU memory pressure, the README suggests FP8 quantization and offloading to CPU or disk.

Multiple pipelines exist for different jobs. DistilledPipeline is the fastest, with eight predefined sigmas. DFRPipeline adds detail-fidelity rendering with keyframes, a spatial detailing pass, and optional temporal 2x or 4x refinement. TI2VidTwoStagesPipeline does production-quality text or image to video with 2x upsampling, and an HQ variant swaps in a second-order sampler. Then there are single-stage prototyping, image and video to video via LoRA, keyframe interpolation, audio to video, a retake pipeline for specific time regions, HDR video to video, and a DubIt pipeline that rephrases while matching speaker identity and lip movement.

Optimization advice centers on the distilled pipeline, FP8 quantization, and FlashAttention on Blackwell and Hopper hardware, with PyTorch SDPA as the automatic fallback elsewhere. Gradient estimation can cut inference steps from 40 down to 20 or 30. For prompts, the README wants a single flowing paragraph, within 200 words, that starts with the main action and stacks precise details about movement, appearance, camera, lighting, and changes. Think of it as writing a shot list, the docs suggest.

The emphasis here is practical: big downloads, explicit hardware paths, and a menu of pipelines for different output types, all documented from the perspective of someone who will actually run it.

• Source 1 · Project repository
• Source 2 · README