OmniShow: ByteDance's All-in-One HOIVG Model, and What Its Repository Actually Ships

The Gap OmniShow Targets: One Model for Four Conditioning Combinations

Most open video generation models pick a lane. A reference-to-video model takes an identity image. An audio-driven model takes a voice track. A pose-driven model takes a skeleton sequence. When a task needs two or three of these at once, the usual answer is to chain models or to retrain a new one per combination, and both approaches leak errors across stages. OmniShow's stated goal is to collapse that into a single model. The README describes it as the first all-in-one model for Human-Object Interaction Video Generation with text, reference image, audio and pose conditioning, and lists four supported settings in one framework: R2V, RA2V, RP2V and RAP2V. The naming is literal. R is the reference image, A is audio, P is pose, and V is the generated video. The paper claims OmniShow is the only model in its comparison set that supports the full RAP2V setting, which is the specific gap the project exists to fill. The audience is narrow and identifiable: researchers working on controllable human video, and product teams building avatar or product-demo pipelines where a person must interact with a specific object that is supplied as an image rather than described in words. If your problem is text-to-video with no reference identity to preserve, this project is heavier than what you need.

Three Mechanisms: Channel-Wise Injection, Gated Audio Attention, Decoupled Training

The README names three components, and each one answers a different failure of naive conditioning. Unified Channel-wise Conditioning handles the reference image and pose. Instead of adding these as separate cross-attention streams, it augments the noisy video tokens with pseudo-frames through channel concatenation, and supervises those pseudo-frames with a reference reconstruction loss. The reconstruction loss is the interesting part: it forces the pseudo-frame channels to keep semantic detail from the reference rather than drifting into a generic latent, which is the usual reason reference identity degrades over a long clip. Gated Local-Context Attention handles audio. Audio features are packed with surrounding context, injected through masked attention so that each video frame attends only to its corresponding audio segment, then passed through an adaptive gate. The README states the gate exists to stabilize early training, which is an admission that the masked attention alone is unstable at the start. Decoupled-Then-Joint Training is the data strategy. Specialized R2V and A2V models are trained first on separate sub-task datasets, then fused by weight interpolation, then jointly fine-tuned. This is a pragmatic answer to the fact that datasets with all four modalities aligned are rare, while datasets with two are common. It also means the final model inherits the initialization of its two parents, so the quality of the sub-task runs bounds the fused result.

Getting It Running: Wan-Based Code, No Published Install Steps

This is where the repository is thinnest. The Updates section records that training and inference code for Wan-based models was released in 2026.05, and the Todo list marks Training Code (Wan-Based) and Inference Code (Wan-Base) as done, but the README as supplied contains no installation block, no requirements file listing, no checkpoint download command, no dataset preparation script and no example invocation. The only concrete pointers are the HuggingFace dataset page for HOIVG-Bench at huggingface.co/datasets/donghao-zhou/HOIVG-Bench and the arXiv technical report at arxiv.org/pdf/2604.11804. The repository is Python, the default branch is main, and the licence is Apache-2.0. Anyone planning to run this should expect to read the training and inference scripts directly to recover the entry points and the expected directory layout, because the README does not document them. Treat that as the first task, not an afterthought: before allocating GPU time, confirm what the inference script expects for each conditioning input and whether checkpoints are hosted anywhere the README names. The material supplied here does not answer either question.

HOIVG-Bench Is the Part You Can Use Without Training Anything

The most immediately usable artifact is the benchmark, not the model. HOIVG-Bench is described as 135 curated samples, each containing a detailed text caption, a human reference image, an object reference image, semantically aligned audio and a coherent pose sequence. The evaluation code was released in 2026.07, after the training code. The design choice worth noting is that every sample carries all modalities, which is what makes the four task settings comparable on the same data: the same clip can be evaluated as R2V, RA2V, RP2V or RAP2V by withholding conditions. That is a cleaner setup than assembling four separate test sets, and it is the reason the benchmark tables can put a model that supports only R2V next to one that supports all four. The metrics are task-specific rather than a single score, spanning TA, FaceSim, NexusScore, Sync-C, Sync-D, AKD, PCK, AES, IQA, VQ and MQ. The README does not define these abbreviations, so you will need the technical report before you can interpret your own numbers. If you are evaluating a competing model and want a fixed target, this benchmark is the deliverable to take.

Where the Reported Results Are Weak, and Where the Comparison Is Uneven

The README's own tables are more candid than the surrounding prose. In R2V, OmniShow does not lead on TA or FaceSim: Phantom-14B scores 8.609 TA against OmniShow's 7.746, and Phantom-14B's FaceSim of 0.876 edges OmniShow's 0.874. OmniShow leads on NexusScore, AES, VQ and MQ. In RP2V the gap is wider in the other direction. VACE scores 7.690 TA against OmniShow's 6.526 and 0.600 FaceSim against 0.474, while OmniShow leads on NexusScore, AKD, PCK, IQA and VQ. The pattern is consistent: OmniShow's advantage concentrates in perceptual and alignment metrics, while prompt adherence and identity similarity in the pose-conditioned setting favour a specialist. That is a real trade-off, not a rounding error, and it follows from the architecture. A model that splits capacity across four conditioning modes will not match a model tuned for one. The comparison is also uneven in a way the tables do not flag. OmniShow is the only entry in the full RAP2V setting, so it wins that column by default rather than by measurement. The RA2V table is the fairest fight, since every listed method supports that setting, and there OmniShow leads on FaceSim, NexusScore, Sync-C, Sync-D, AES, IQA, VQ and MQ while trailing HuMo-17B on TA by 0.053. Read the tables with that structure in mind.

The Alternative: VACE and the Specialist-Per-Task Approach

The natural alternative is VACE, which appears in both the R2V and RP2V tables. The difference in approach is the conditioning mechanism. VACE-style pipelines treat control signals as separate inputs routed into a pretrained video backbone, which lets you add or drop a control without retraining the base model, and lets a team pick the best specialist per task. OmniShow instead fuses the conditions into one model through channel concatenation, masked attention and weight interpolation between sub-task parents. The practical consequence is that VACE beats OmniShow on TA and FaceSim in the pose-conditioned setting, while OmniShow covers RAP2V at all. If your pipeline can afford to route each request to a different checkpoint, the specialist route is currently stronger on those two metrics. If you need one endpoint that accepts any subset of the four conditions and you cannot maintain four checkpoints, the fused design is the reason to choose OmniShow. For audio-driven work specifically, the HuMo family is the closer comparison: HuMo-17B leads OmniShow on TA in RA2V and is competitive on FaceSim and Sync-C, so if audio plus reference is your only requirement, the larger HuMo variant is a legitimate starting point rather than a fallback.

Maintenance, Licence and What the Todo List Still Owes

The licence is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant, subject to the usual notice and attribution conditions. That is a permissive choice for a ByteDance-affiliated release and it removes the licensing question that blocks many research models. It does not settle the question of the base checkpoint. OmniShow is built on Wan, and the README does not state which Wan weights or licence terms apply to the derived model, so verify that chain before shipping anything. On maintenance, the Updates timeline shows steady progress through 2026: technical report in April, benchmark data in April, training and inference code in May, benchmark evaluation code in July. The last push recorded is 2026-07-29. No releases were retrieved, which means there is no tagged version to pin and no changelog to diff against, so an upgrade means pulling main and reading the diff yourself. The Todo list is the honest signal here: the final entry, Inference Code (Wan-Base), appears as an unchecked checkbox in the supplied README even though the Updates section announces that the code was released. That inconsistency is unresolved in the material available, and it is the first thing to check in the repository itself.

Editorial conclusion

Adopt OmniShow if you are a research group or a video-generation team with the GPU budget to fine-tune a Wan-scale diffusion transformer, and if your task is exactly human-object interaction video where a reference image, an audio track and a pose sequence all matter. Do not adopt it if you need a maintained library, a pip-installable package, a hosted endpoint, or a model whose conditioning inputs are optional rather than structural. Before committing, clone the repository and confirm two things yourself: whether the inference script exposes a single-reference path, since the README's Todo list leaves the 'Wan-Base' entry unchecked, and whether the HOIVG-Bench evaluation code accepts your own generated videos rather than only the released dataset. The benchmark table is the strongest evidence in the repository, and it is also the only evidence, because no release artifacts were retrieved.

Official sources

1. Correr-Zhou/OmniShow on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README