SCAIL-2 replaces pose intermediates with masks, and a wrong mask quietly drops you into replacement mode

The default branch is the inference one, and training is on another

The repository's default branch is not named main. It is named after the video backbone the project builds on, which is the first signal that this is a fork or a rebase of something upstream rather than a fresh tree.

The page is explicit about scope. It says the code is for inference of the model, an open-source model supporting end-to-end character animation. So the default branch is inference only.

Training arrived later and somewhere else. A news entry dated 2026-08-06 announces the training code, and it points at a branch with a different name, one that suggests a training stack rather than the video backbone. The same entry offers a low memory version of the training code from a third-party repository, which is a polite way of saying the released training path wants more video memory than some people have.

Two more facts to carry. There are no published releases at all, so there is nothing to pin: you clone a branch and run whatever is at its head. And the last commit to the default branch is dated 2026-08-24, so the tree has not moved in the six weeks since.

That combination is fine for a research release and awkward for a dependency. If you need a stable SCAIL-2 later, the thing you want to record today is the commit you tested.

Three mask colours, and a wrong one changes the mode instead of failing

This is the most useful section on the page, and it is short.

The mask is described as a critical input even in animation mode, where you might expect the reference image and the driving motion to be enough. Three colours have three meanings. Black tells the model the background at that location should not be visible. White tells it the background at that location should be visible. Colour, meaning a non black and non white value, encodes the correspondence between character regions and the driving motion.

So the mask is not a binary foreground selection. It carries three different pieces of information in one image, and only one of them is about visibility.

Then the sentence that makes this worth an article. Without a correct mask, animation mode collapses into replacement-mode behaviour on certain inputs. No error, no warning, just different output.

That is the failure mode to know about. If your results look like the character is being swapped rather than animated, the first thing to check is the mask and not the prompt, the reference, or the resolution. The page shows a reference mask and a driving mask side by side for three cases, end-to-end animation, pose-driven animation, and replacement mode, which is effectively a diagram of how the same two images differ per mode.

The masks also do the work of multi-reference. Extra visual inputs supply what a single reference cannot, such as a back view, a close-up, or an occluded background, and the colour assignment logic is what decides which mask each of them gets.

A to-do list with one item, already finished

The project has a to-do section, and it contains exactly one line: open-source the training code on that other branch. It is ticked.

That is worth noticing for what it says about the project's own scope. Everything else on the page is delivered work: the model, the inference code, multi-reference inference, a relighting adapter for replacement mode, two integrations into a node-based interface tool, and a preference-model adapter. The only thing the authors considered unfinished was the one thing that has since landed.

The integration timeline explains why the integrations are treated as news rather than as a support policy. Two entries share a date in June 2026: the integration itself, and the model plus inference code being open sourced. A second integration, adding multi-reference support, landed about a week later. So the node-based tool was an early target rather than an afterthought, and each addition to the model is a separate pull request there.

Two of those news entries are dated in the format with slashes while the others use dots. Small, and the kind of inconsistency that makes a timeline harder to scan than it needs to be.

The relighting adapter is also worth noting as a shape: it ships as a single file on the model host, is enabled in this repository, and is enabled in the node-based implementation. One artefact, three consumers.

Sixty thousand motion pairs synthesised from three other models

The argument is that intermediate pose representations are the problem. The previous version identified how to represent a pose and how to inject one as the bottlenecks, and concluded that depending on an intermediate still limits complex motion and identity generalisation. Skeleton maps are ambiguous in complex scenes, and they restrict the driving source to human movement seen from outside, which is why they cannot drive a character with an animal. For replacement and multi-character work, the field's answer has been inpainting masks, and the argument here is that a mask is just another intermediate.

The training data is the interesting part. Sixty thousand motion pairs were synthesised using three existing models: this group's own preview model, a video animation model, and a character animation model from another lab. So the data pipeline is three third-party systems generating pairs that the new model then learns from.

The interface that made it possible is described as a unified motion transfer interface containing two types of masking channels plus a dedicated rotary position encoding design. Two mask channels is what lets the mask carry visibility and correspondence at once, and the rotary design is what lets one model consume data produced by systems that disagree about layout.

Then the part that makes a synthetic pipeline worth the risk: reserve driving, so the model is trained to exceed its teachers. The claimed result is emergent capability rather than imitation, namely cross-identity replacement, driving from animals, and handling a richer control intermediate, specifically the mesh rendering from a body estimation model, in a zero-shot way. The last of those is the strongest claim on the page and the one with no numbers attached.

A preference-model step aimed at the bias of pose-driven generators

There is a second method described almost in passing, and it is the one with an artefact you can download.

The page says the bias of pose-driven generators is modelled as a preference, and introduces a mechanism called Bias-Aware direct preference optimisation to improve detail. The sentence describing it contains a typo in the word mechanism, which is worth noting only because it tells you the paragraph has not been edited in a while.

What the mechanism does is the interesting part. A pose-driven generator has a systematic bias, and rather than collecting more data it is given a preference signal about its own outputs. That is a cheap intervention relative to regenerating a dataset, and it targets a specific failure mode, which is detail quality rather than motion.

The artefact is released as a low-rank adapter on the model host and can be enabled in this repository and in the node-based implementation. So the same file serves three consumers, which is the pattern the whole release follows: base weights, then small adapters, then integrations.

The community section follows the same shape. It thanks named individuals on three platforms and notes that audio comes from the reference videos, which tells you the expected input is a clip with sound rather than an image sequence.

Two modes prefer two different resolutions, and both need dimensions divisible by 32

The checkpoint table has a notes column that is worth more attention than the download links.

The model was trained with mixed resolutions and mixed frame rates. End-to-end driving supports both of two resolutions, while pose-driven performs better under the larger one. And for any other resolution, both dimensions have to be divisible by 32, with a resolution given as a worked example.

That is three constraints where most model pages give one, and the third is the one that bites. An arbitrary resolution that does not divide by 32 is not rejected with a clear error; it is either rejected somewhere deep in the attention shapes or silently padded. Either way the fix is on the page and easy to miss.

The two modes preferring different resolutions is the other useful fact. A user who tested with pose driving at the smaller resolution and then switched to end-to-end would conclude the model got worse, when the table says it was the wrong setting.

Weights are hosted in two places, a model hub and a regional model repository, which is a distribution choice rather than a technical one, and the download command notes that the video autoencoder and the text encoder have both been folded into the checkpoint. So there is one file to fetch rather than a base model plus two adapters, and also no obvious way to swap the text encoder for a different one.

An attention package with no version, and a vendor SDK in the requirements

The requirements list has sixteen entries and two of them deserve comment.

The attention package is declared with no version at all, not even a lower bound. It is the one dependency in this ecosystem that has to be compiled against the exact combination of framework and graphics toolkit you have, so an unpinned entry in a requirements file is an invitation to spend an evening. Everything else is at least bounded: the framework at 2.4 or newer, the transformers library at 4.49 or newer, the diffusion library at 0.31, an image library, an accelerator, the numeric library capped below its next major, and a web interface library at 5 or newer.

The second is a software development kit for one commercial model provider, sitting in the same list as the numeric library. Its presence means the inference path can call that provider, whether or not you intend to.

The tree tells you where the code actually lives. There is a directory named after the video backbone, which is a substantial amount of code in a project whose stated contribution is a model and an interface, and a submodule manifest besides. There is a second top level directory for the pose-driven path. And there are six loose scripts at the root: the inference entry point, two converters for weights and for adapters, one that caches and reuses a zero pose latent, one that caches a webdataset, and one that rewrites prompts.

Those last two are the ones that tell you about the intended workflow. Caching zero pose latents means you are expected to run the same empty-pose prefix repeatedly, and caching a webdataset means the training or evaluation data is prepared once into a format the pipeline can stream.

Editorial conclusion

Read the mask section before you run anything, because the failure mode is silent. If your mask is wrong on certain inputs, animation mode does not error, it produces replacement-mode output instead, and you will spend an hour tuning a prompt before looking at the pixels. Two things to settle first. Which mode you actually want, since end-to-end and pose-driven are separate example sets with different resolution advice, and the mask semantics differ between them and replacement mode. And whether you have the training side at all, because the default branch is inference only and the training code is on a differently named branch, with a third-party fork offered for low memory machines. Beyond that, check the resolution rule rather than the model size: dimensions have to divide by 32, the two driving modes prefer different resolutions, and a checkpoint trained at mixed resolutions and frame rates does not license arbitrary ones.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. zai-org/SCAIL-2 on GitHub