JoyAI-Echo: two separate audio-visual generation projects in one repository

The two projects inside JoyAI-Echo, and why they are split

JoyAI-Echo is not one tool. The README states plainly that the repository holds two independent projects, each with its own environment, checkpoints and entrypoint, and tells you to pick the one you need. Echo-LongVideo targets long-horizon, multi-shot audio-visual generation, with the README claiming support for 10+ minutes of generation and a paired audio-video memory bank that carries continuity across shots. Echo-WM is an omnimodal world model for generative media that responds to continuous navigation while video, environmental sound, music and speech evolve together.

The repository layout reflects that split. echo_longvideo/ contains inference.py, configs/, prompts/ and an ltx-* directory. echo_wm/ contains inference_wm.py, a Gradio demo and its own bundled ltx-* copy. The README is explicit that the two do not share a Python environment or a checkpoint directory, and that echo_wm/ bundles its own copy of ltx-core and ltx-pipelines so that installing one project never affects the other. That is a deliberate isolation choice, and it costs disk space and setup time. It also means a bug report or an issue about one subproject has nothing to do with the other.

The intended audience is research groups working on generative video and audio, not application teams looking for a drop-in media API. The README closes with a restriction to academic research and non-commercial use only, which frames everything else in the repository.

How Echo-LongVideo carries continuity across shots

The mechanism named in the README is a paired audio-video memory bank. Long-horizon generation in this project is multi-shot, and the memory bank is what carries continuity from one shot to the next for both modalities. The README does not describe the internal structure of that bank, how entries are evicted, or how memory size scales with the number of shots, so the exact data flow beyond the name is undocumented at the top level. The subproject README at echo_longvideo/README.md is where the README points for the environment, checkpoints and entrypoint details.

What the top-level README does confirm is that Echo-LongVideo is the 1.5 generation. The news list dates JoyAI-Echo 1.5 / Echo-LongVideo to 2026-08-28 and describes it as including long-horizon generation, consumer-GPU inference profiles and a Director Agent. The phrase consumer-GPU inference profiles is worth noting: it implies the project ships profiles tuned for hardware below datacenter class, but the README does not list which GPUs, how much VRAM, or what quality trade-off each profile makes. Anyone planning capacity should read the subproject README rather than assume.

Version 1.0 is preserved on the echo1.0 archive branch, dated 2026-06-22. That archive branch matters if you need to reproduce older results, because the main branch has moved to 1.5.

Echo-WM, the world model with a causal rollout path

Echo-WM is the second project, released 2026-08-26 according to the news list. The README describes it as an omnimodal world model that responds to continuous navigation, with video, environmental sound, music and speech evolving together. That is a different problem from long-video generation: instead of producing a fixed multi-shot sequence, the model is meant to be entered and steered.

The roadmap section is the most informative part of the README for anyone judging maturity. Echo-WM is on LTX-2.3 today. Two backbone variants are checked off: LTX-2.3 Base, a bidirectional audio-visual DiT used by Echo-WM Base at roughly 10 seconds, and LTX-2.3 Flash Preview / Causal, described as the current public preview with chunk-causal attention, KV-cache rollout and 4-step inference. The causal path has its own document, echo_wm/README_CAUSAL.md.

Everything else on the roadmap is unchecked. LTX-2.5 Base and LTX-2.5 Causal are planned, with the causal variant described as block-causal masks, a sink plus FIFO cache and a few-step student. The acceleration list is entirely unchecked: sparse attention with SageAttention and similar kernels, FlashAttention / FlashInfer, a paged KV-cache with variable length so rollouts stay bounded and RoPE and UCPE rebased when tokens evict, and FP8 or TensorRT compilation of the DiT forward. Read that as a statement of where the cost sits today: long rollouts are the expensive part, and the mitigations are not shipped. The 2026-09-04 news item adds a UE simulation pipeline for the Echo-WM world data engine, covering physics-based trajectory generation, Movie Render Queue rendering and distributed scheduling.

Installing JoyAI-Echo and running your first generation

The README gives a Quickstart for each subproject rather than a single install. Both use conda. Start with the long video path if that is what you need. The README's command creates the environment from the subproject's environment.yml and activates it as echo-long.

cd echo_longvideo
conda env create -f environment.yml && conda activate echo-long

For the world model, the README uses a plain conda environment named echo-wm on Python 3.11 and then installs requirements.txt with pip. Note that this is a different environment name and a different Python version from the long-video path, which is consistent with the claim that the two projects do not share an environment.

cd echo_wm
conda create -n echo-wm python=3.11 -y && conda activate echo-wm
pip install -r requirements.txt

After either environment exists, checkpoints are the next step, and the README is clear that they are downloaded separately in both cases. It does not list the files or paths at the top level; it says to see each README for the exact files and paths. So the top-level Quickstart gets you an environment, not a runnable model. Expect to open echo_longvideo/README.md or echo_wm/README.md before you can generate anything, and expect the download to be the largest single step in setup.

One more entrypoint the README mentions: echo_wm/ ships a Gradio demo alongside inference_wm.py. If you want to see the world model respond to navigation before writing your own driver, that demo is the shortest path. The README does not document its launch command, so read the subproject README for that. There is also a separate ComfyUI integration at github.com/zhuang2002/ComfyUI_JoyAI_Echo, listed in the header links rather than in the Quickstart.

Where JoyAI-Echo is the wrong tool

The licence is the first hard boundary. The README states the project is based on LTX-2 by Lightricks Ltd, that portions of the original LTX-2 codebase have been modified by JD.com for academic and research purposes only, and that the project is not intended for commercial use. For commercial use of LTX-2 or its derivatives, the README says to contact Lightricks Ltd. If your product plan involves shipping generated video or audio, this repository is not the starting point.

The second boundary is the split itself. If you want one environment that can do both long-video generation and interactive world modelling, this is not that. The README says the two do not share a Python environment or a checkpoint directory, and echo_wm/ carries its own copy of ltx-core and ltx-pipelines. You will pay for that isolation twice in disk and setup time.

The third boundary is the state of the rollout stack. The acceleration work that would make long causal rollouts cheaper, sparse attention, FlashAttention / FlashInfer, paged KV-cache, FP8 and TensorRT, is listed as unchecked on the roadmap. The README gives no throughput numbers and no VRAM figures for the consumer-GPU inference profiles it mentions. If your decision depends on a cost per minute of generated video, this repository does not give you one. The base model path is also roughly 10 seconds per the roadmap note, so anything longer depends on the causal preview, which the README itself labels a preview.

Finally, the top-level README is a router, not a manual. It defers environment, checkpoint and entrypoint detail to the two subproject READMEs. Anyone who reads only the front page will not have enough to run either project.

Alternatives and how the approach differs

The most direct comparison is LTX-2 itself, by Lightricks Ltd. JoyAI-Echo is built on it: the README says the project is based on LTX-2, that portions of the original codebase were modified by JD.com, and that all original copyright, licence, patent, trademark and attribution notices are retained, with the project remaining subject to the LTX-2 Community License Agreement. The difference in approach is what sits on top of the backbone. JoyAI-Echo adds a paired audio-video memory bank for multi-shot continuity in Echo-LongVideo, and a chunk-causal, KV-cache rollout path with 4-step inference in Echo-WM, plus its own checkpoints and a Director Agent. If you need the unmodified upstream model, its own licence terms and its own support channel, going to LTX-2 directly avoids the derivative layer.

For the world model side, the roadmap names the intended trajectory rather than a competitor: Base and Causal are to move from LTX-2.3 to LTX-2.5, loading official LTX-2.5 weights with a Gemma 4 text encoder and 2.5 VAE / DiT into the existing bidirectional path. That is a plan, not a shipped capability, and it tells you the project intends to track upstream releases rather than diverge.

A third path is the ComfyUI integration at github.com/zhuang2002/ComfyUI_JoyAI_Echo, which the README lists in its header. That is a different way to consume the model, node-based rather than script-based, and it is a separate repository with its own maintenance. The README does not describe what it supports or which subproject it wraps.

Maintenance, upgrade cost and licence implications

The last push to the default branch was on 2026-09-04, which is recent, and the repository is not archived. The news list shows a steady cadence through mid-2026: 1.0 on 2026-06-22, Echo-WM on 2026-08-26, 1.5 with Echo-LongVideo on 2026-08-28, and the UE simulation pipeline on 2026-09-04. No releases were retrieved, so versioning is by branch and by dated news entries rather than by tagged releases.

Upgrade cost is shaped by the roadmap. Moving Echo-WM from LTX-2.3 to LTX-2.5 is described as loading official LTX-2.5 weights, with a Gemma 4 text encoder and 2.5 VAE / DiT, into the existing bidirectional path, and then repeating the Flash recipe on 2.5 with block-causal masks, a sink plus FIFO cache and a few-step student. Both are unchecked. If you build on the causal preview today, the README's own roadmap suggests the attention and cache design under you is expected to change, and the planned paged KV-cache explicitly involves rebasing RoPE and UCPE when tokens evict, which is the kind of change that invalidates tuned inference settings. The archive branch echo1.0 exists for 1.0, so older work is at least retrievable.

On licensing, the repository's LICENSE file is present but the metadata carries NOASSERTION, and the README states the project remains subject to the LTX-2 Community License Agreement while restricting itself to academic and research use. The README also points commercial users of LTX-2 or its derivatives to Lightricks Ltd. Treat the README's wording as the operative statement and read LICENSE and THIRD_PARTY_NOTICES.md yourself; this is not legal advice.

Editorial conclusion

Adopt JoyAI-Echo if you are doing academic or non-commercial research on long audio-visual generation or interactive world models and you can afford two isolated environments plus separately downloaded checkpoints. Do not adopt it if you need a commercial licence, a single shared environment, or a packaged release; the repository explicitly restricts use to academic and research purposes. Before committing, verify which subproject you need, read that subproject's own README for the exact checkpoint files and paths, and confirm the LTX-2 Community License Agreement terms that the repository says still apply.

Frequently asked questions

Official sources

1. Issues
2. jd-opensource/JoyAI-Echo on GitHub
3. Project website
4. README