SCAIL: character animation by full-context pose conditioning

Channel concat loses the rotation under 1.3B

SCAIL names two bottlenecks standing between character animation and production use: poor generalisation to characters outside the training set, and incoherent motion in hard scenarios, with flipping and turning called out as failures that show up in basic motions. The paper's second contribution is not the pose itself but the channel through which the pose reaches the network. Channel concat, pose-guider and residual layers are named as the norm, and the objection against all three is the same: they add features rather than showing the full context, which is enough under controllable generation but runs into the ceiling of the pretrained backbone. The concrete claim concerns rotation. Channel concat under a 1.3B model fails to maintain correct body rotation, because that backbone lacks capacity for complex human motion. SCAIL's answer is to show the model the entire turning context, which lets it learn how the motion works instead of being told the pose one frame at a time.

The pose side lives outside the model

The first contribution is a pose representation the authors call 3D-Consistent, described as identity agnostic, aware of depth, and rich enough to preserve motion information a plain keypoint list loses. That representation is the conditioning input, not the animation. Pose tooling sits outside the model code in this repository: scail_pose sits at the top level, POSE_INSTRUCTION.md documents the pose condition separately from the README, and 3D pose preprocessing was folded whole into WanGP by deepbeepmeep. Since 2026.3.1 a Blender 3D rig can drive scail-pose, credited to toyxyz in issue #30, which is what opens up body shapes and poses outside the human template. ComfyUI-SCAIL-Pose added two more paths on the same side, saving the NLF mesh as a 3D glb animation and previewing the SCAIL-Pose skeleton in 3D.

The SAT entry points, and two unexplained directories

What ships is inference scaffolding around SAT, the SwissArmyTransformer framework, and its shape is visible in the root entries. sample_video.py is the sampler entry point, data_video.py handles video data, diffusion_video.py and dit_video_crossattn_sc_xc.py carry the diffusion process and the transformer, and arguments.py holds the parameters, which is where a reader has to go because the README prints no flags at all. configs/ and eval/ hold run configuration and evaluation, examples/001/ and examples/002/ hold sample inputs next to examples/input.txt, and gen_prompts_gemini.py is the script behind the prompt optimisation item on the TODO list. Two entries in that listing resist explanation. sat/ sits beside .gitmodules, and the README does not describe what the SAT copy inside this tree contains or how it differs from SwissArmyTransformer itself. notuse/ is present with no note anywhere saying what it holds or why it stayed in a tree whose stated scope is inference.

An inference-only tree that pins the training stack

Here is the contradiction worth knowing before starting. The opening paragraph scopes the repository to inference of the preview model, yet requirements.txt asks for deepspeed, pytorch_lightning, wandb, tensorboardX, webdataset and datasets, which are distributed training and logging tools that an inference pass never calls. The README also promises on 2025.12.19 to update the training code of SCAIL on SAT for reproducibility, and no box on the TODO list is marked for it. The dependency set is therefore shaped for training, the training code is not in the tree, and the checklist records three finished boxes: preview weights and inference config at 512p and 5s, prompt optimisation snippets, and the Wan implementation, none of them the training recipe. A second tension sits beside it: the shipped preview is a 14B DiT, while the experiment carrying the core argument runs on a 1.3B model.

An unusually high dependency floor across the stack

The pins tell you which era of the ecosystem this code was cut for. Almost all of them are floors rather than exact versions, so a fresh environment resolves to whatever is newest: torch starts at 2.9.1, gradio at 6.0.2, transformers at 4.57.3, datasets at 4.4.1 and openai at 2.9.0. A machine sitting on an older PyTorch cannot satisfy the file.

torch>=2.9.1
diffusers>=0.35.2
transformers>=4.57.3
numpy==1.26.4

Only one line in that file is an exact pin, numpy==1.26.4, while every other entry is a lower bound, so it is the line that blocks an environment which already moved to a newer NumPy, and the first place to look when an import fails. Video handling pulls in five separate libraries under one comment.

# Video IO dependencies
moviepy>=2.2.1
av>=16.0.1
imageio>=2.34.2
imageio-ffmpeg>=0.6.0
decord>=0.6.0

All of them have to resolve together for video to build, and the spread inside that block is wide: av is asked for at 16.0.1 while decord is held at 0.6.0.

Weights first, because no install command is documented

There is no install line to copy. The README does not document a pip invocation, a conda environment, a checkpoint download script or a single inference flag, so the first real step is assembling the environment by hand from requirements.txt and the model pages. The weights live at zai-org/SCAIL-Preview on HuggingFace and at ZhipuAI/SCAIL-Preview on ModelScope, with a quantised build at SCAIL-Preview-GGUF. The repository is Apache-2.0 and has no GitHub releases, so there is no tag to pin a working configuration to, and the last push was on 2026-05-06. For a first run, read arguments.py and configs/, because that pair holds the resolution, the frame count and the conditioning switches. Marked done on the TODO list, the preview weights are 512p and 5s, so any input has to match that shape. On the host side the project reversed its first choice: on 2025.12.19 the maintainers offered the Wan official framework of SCAIL instead of SAT for more convenient inference, and that code sits on the wan branch, so a plain clone of the default branch gives you the SAT layout described above.

ComfyUI and WanGP as the lighter host

If running a 14B DiT through a research framework is more than you need, two community paths change the shape of the work. SCAIL went native in ComfyUI on 2026.3.1, which turns the whole thing into a node graph, and it is also available inside ComfyUI-WanVideoWrapper, where KJ did the adaptation while pose extraction and rendering were partly ported to ComfyUI-SCAIL-Pose. The second path is WanGP, where a low VRAM route for the preview model was contributed by deepbeepmeep with 3D pose preprocessing integrated and speed work done; its claim of compatibility with any PyTorch version describes that port, not the SAT layout in this repository. The third is the quantised SCAIL-Preview-GGUF build, which the README records as available without attaching any quality or size figures to it. The difference in approach is real: the SAT route is a Python script you launch with arguments, while ComfyUI hands the same conditioning to a graph editor and a pose node that can export the mesh.

Community clips, and no numbers to weigh them

What the evidence offers is a gallery rather than a table. The community section lists Chibi Gotham Battle, Homer Bullet Time with Uni3c, Anime Art Animation, Street Fighter 6 Motion Mimic, Doodle Art Animation, Dual Dance and Group Dance, plus Quadrupeds Animation with ViTPose, and the quadruped result is the striking entry because the model had no animal training data at all. Those are submissions shared with their authors' consent, put together by people who chose their own inputs, so they demonstrate range and say nothing about a failure rate. There is no benchmark table, no baseline score and no ablation anywhere in the README, and the only comparison offered is visual, against other methods on the project page. Two claims therefore rest on trust: that a 14B preview generalises to characters it never saw, and that the pose representation it was trained on is what the scail_pose tools emit. Nothing in the repository checks either one.

Editorial conclusion

SCAIL suits a researcher who already runs large diffusion models and wants the pose conditioning idea rather than a product. Teams shipping character work now have lighter routes through ComfyUI or a GGUF build, and neither asks for the deepspeed and pytorch_lightning stack sitting in requirements.txt. Before committing, read arguments.py and configs/ to see what the 512p, 5s preview accepts, confirm your environment tolerates the numpy==1.26.4 pin, and check whether the SAT training code promised on 2025.12.19 has landed.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. zai-org/SCAIL on GitHub