nv-tlabs/ardy: NVIDIA's Interactive Motion Model and What It Actually Ships

What ARDY generates, and the constraint that shapes it

Most text-to-motion systems generate a clip. You give them a prompt, they return a fixed-length sequence, and the interaction ends there. ARDY is built for the opposite workflow: an operator types or changes a prompt while motion is already being produced, and drops spatial constraints into the stream as it runs. The README describes the model as "designed for interactive motion generation, supporting online text prompting and flexible long-horizon kinematic constraints (root paths/waypoints, full-body keyframes, and sparse joint positions/rotations) with real-time responsiveness."

The intended user is a research engineer or technical artist working on character control, not someone who wants a batch animation tool. The demo is a viser web viewport where you drive a humanoid with mouse and keyboard locomotion controls while streaming text prompts at it. That framing explains the checkpoint table: ARDY-Core-RP-20FPS-Horizon40 and ARDY-Core-RP-20FPS-Horizon8 target a Core skeleton, while ARDY-G1-RP-25FPS-Horizon52 and ARDY-G1-RP-25FPS-Horizon8 target the Unitree G1. Two frame rates, two horizons, two skeletons. The horizon number is the prediction length the model was trained for, and the README treats it as a per-checkpoint property rather than a runtime flag, which is a real constraint if your application needs a horizon none of the four checkpoints provides.

The second constraint is licensing, and it is easy to miss. The repository is Apache-2.0, but every checkpoint links to the NVIDIA Open Model agreement, and the text encoder is the gated meta-llama/Meta-Llama-3-8B-Instruct model. Three separate terms apply to three different pieces of the stack.

Autoregressive diffusion with a hybrid representation

The name is the architecture summary. ARDY is autoregressive, so it produces motion step by step rather than denoising an entire sequence at once; that is what makes online prompting possible, because new text can enter the conditioning between steps. It is a diffusion model, so each step is a denoising operation rather than a direct regression. The hybrid representation is the part the paper title emphasizes and the README does not unpack beyond the constraint list: root paths and waypoints, full-body keyframes, and sparse joint positions or rotations are all accepted as inputs, which implies the representation carries both a global trajectory component and a local pose component rather than a single flattened vector.

The repository layout supports the split. There is an ardy/ package for the model code, a MotionCorrection/ directory alongside it, and scripts/ for entry points. MotionCorrection is not a Python module: setup.py defines a CMakeExtension and a CMakeBuild class that shells out to cmake and builds a native extension at install time, and pyproject.toml lists cmake>=3.15 in build-system requires. So a pip install of this project compiles C++. The README calls it "the bundled motion-correction C++ extension" and notes it needs CMake >= 3.15 and a C++17 compiler. That correction stage is what keeps generated motion consistent with the constraints you inject mid-stream, and it runs outside the Python model path.

One detail in pyproject.toml is worth reading closely. The torch floor is written as torch>=2.4.0a0, not torch>=2.4. The comment explains why: NGC containers ship torch as a prerelease such as 2.4.0a0+...nv24.07, which sorts below 2.4.0, so a plain floor would let pip replace the container's matched torch, torchvision and torch-tensorrt builds with PyPI wheels. The same reasoning caps vector-quantize-pytorch at <1.26. This is a project that has been run inside NVIDIA containers and adjusted its dependency bounds to survive there.

Installing ARDY and getting the demo on screen

The README's tested configuration is Ubuntu Linux 22.04, an RTX 4090, nvidia-driver-575 and Python 3.11. Start with a fresh environment, then install PyTorch before ARDY. The ordering is deliberate: the README states that installing PyTorch yourself first lets you pick the CUDA build matching your GPU and driver, and the ARDY install then keeps that build rather than pulling a default wheel.

conda create -n ardy python=3.11 -y
conda activate ardy
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
pip install -e ".[all]"

The cu126 index URL in that block is the README's own example, not a universal answer; substitute the index for your CUDA version as described at the PyTorch install page the README links. Partial installs are supported if you do not want everything. `pip install -e .` gives core model inference only, `.[demo]` adds the interactive demo with viser and gradio, and `.[trt]` adds TensorRT acceleration. The TensorRT extra has its own requirements: an NVIDIA driver >= 525 and network access to pypi.nvidia.com during install. The README's fallback for setups that do not meet those is explicit: install `.[demo]` instead and pick a non-TensorRT acceleration mode in the demo.

Before the demo will run, the text encoder needs a Hugging Face token. Access to meta-llama/Meta-Llama-3-8B-Instruct must be granted on your account first, then either log in or place the token in the cache file.

hf auth login

The alternative the README gives is pasting the token into `~/.cache/huggingface/token`. If the `hf` command is missing, `pip install --upgrade huggingface_hub` provides it. Then launch the demo:

python scripts/run_demo.py

Open `http://localhost:2333` in a browser. Left-drag rotates, right-drag pans, scroll zooms. In the UI, pick a checkpoint from the Model Directory. The README notes that checkpoints download automatically when used in the demo, so there is no manual download step. If you relaunch often, run `python scripts/run_text_encoder_server.py` in a separate terminal so the demo does not reinstantiate the text encoder each time. The kinematically constrained demos additionally need the Bones SEED dataset, downloaded from Hugging Face and placed under `datasets/bones-seed/` with `g1/csv/` and `metadata/seed_metadata_v004.csv` inside it. That step is optional unless you want to sample constraints from real motion sequences.

Where ARDY is the wrong tool

The gated text encoder is the first wall. meta-llama/Meta-Llama-3-8B-Instruct requires granted access on your Hugging Face account and a token at runtime. There is no bundled alternative encoder and the README does not describe an offline or non-Llama path. If your organization cannot accept the Llama 3 licence terms, the model as shipped does not run, regardless of the Apache-2.0 code licence.

The second wall is hardware. The tested configuration names an RTX 4090 with nvidia-driver-575, and the TensorRT path needs driver >= 525. The README does not document a CPU path, a reduced-precision fallback, or memory requirements, so anyone planning to run this on a laptop or a CPU-only CI machine has no documented route. The install also compiles a C++ extension through CMake, which means a plain `pip install` on a machine without a C++17 toolchain fails rather than degrading to pure Python.

The third issue is checkpoint coverage. Four checkpoints exist, covering Core and Unitree G1 skeletons at fixed FPS and horizon combinations. If your character rig is not one of those two skeletons, or your application needs a horizon that is neither 8 nor 40 (Core) nor 8 nor 52 (G1), the README offers no fine-tuning recipe and no way to change the horizon at inference. The SOMA body model version is listed as coming soon, which is a statement about future work, not a current capability. Treat the skeleton list as closed for now.

Finally, the README is thin on failure behaviour. It does not document what happens when a constraint is infeasible, how the model recovers from a contradictory prompt, or what rollback looks like if a generation goes wrong. For an interactive system, that is the documentation gap that matters most.

ARDY against MotionBricks and Kimodo

People searching around this project also search for MotionBricks and Kimodo, so the comparison is worth stating precisely. The difference that matters is where the model sits in the interaction loop. ARDY is autoregressive with online prompting: text arrives while motion is being generated, and the constraint types listed in the README (root paths, waypoints, full-body keyframes, sparse joint positions and rotations) can be injected into a running stream. A clip-based motion generator, which is the common shape for this class of tool, takes a prompt and returns a finished sequence, so changing the prompt means starting a new generation and changing a constraint means re-running with a different input.

That distinction has consequences beyond latency. With a clip-based pipeline you can afford a long denoising schedule and heavy post-processing per clip, because nothing is waiting on the output. ARDY's real-time claim forces a different budget, and the C++ motion-correction extension is the visible sign of it: constraint consistency is enforced by compiled code rather than by a slower Python post-pass. The trade is that the horizon is baked into each checkpoint, so you cannot buy a longer prediction window by waiting longer.

ARDY also ships a browser demo rather than a library-first interface. The viser viewport with mouse and keyboard locomotion controls is where the streaming behaviour is actually observable. If you evaluate ARDY by reading the model code alone, you will miss the part that makes the design legible.

Maintenance, versions and what the licences cover

The last push to the repository was on 2026-07-10, and the four checkpoints carry the same release date. The repository is not archived. There are no releases retrieved, so version tracking happens through pyproject.toml, which currently declares version 0.2.0. There is no changelog in the top-level entries, so upgrade cost between versions is not documented anywhere in the repository.

Upgrade cost has two sources here, and both are visible in the dependency file. The first is the transformers pin: pyproject.toml requires transformers==5.8.1 exactly, with a comment noting that the vendored ardy/model/llm2vec has only been tested with that version. An exact pin means you cannot move transformers forward without testing the vendored code yourself. The second is the torch floor written as 2.4.0a0 and the vector-quantize-pytorch cap at <1.26, both of which exist to keep container-provided builds intact. The comment on the cap says to revisit it when moving to a newer container, so these bounds are expected to change.

On licensing, three layers apply and they are not the same. The repository code is Apache-2.0, stated in pyproject.toml and in the LICENSE file. The checkpoints are released under the NVIDIA Open Model agreement, linked from every row of the checkpoint table. The text encoder is the gated Llama 3 model under its own terms. Redistributing the code, the weights and the encoder together means satisfying all three, and the README does not attempt to reconcile them. This is a description of what the repository states, not legal advice; check the terms yourself before shipping anything.

Editorial conclusion

Adopt ARDY if you need streaming text-to-motion with root paths, waypoints or full-body keyframes and you already have an RTX-class GPU plus approved access to meta-llama/Meta-Llama-3-8B-Instruct. Do not adopt it if you cannot get that gated model access, if you need CPU inference, or if you want a permissively licensed text encoder bundled with the weights. Before committing, verify three things: that your driver and CUDA build satisfy the torch>=2.4 requirement, that CMake >= 3.15 and a C++17 compiler are present because the motion-correction extension builds at install time, and that the checkpoint skeleton (Core or Unitree G1) matches your rig. The Apache-2.0 licence covers the repository code only; the checkpoints carry the NVIDIA Open Model agreement and the text encoder carries the Llama 3 terms.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. nv-tlabs/ardy on GitHub
4. README