ABC: Amazon's FAR lab opens its scalable robot behavior cloning stack

Behavior cloning, opened by a corporate lab

ABC, from Amazon's Frontier AI and Robotics lab, is an open-source take on scalable behavior cloning: training robot policies from demonstrations, with the full loop published, data, training, simulation and deployment. The README's release list is broad: ABC-DiT and ABC-VLA training, pretrained and task-finetuned checkpoints, a self-contained simulator, evaluation tooling, real-robot deployment utilities and data conversion scripts, with the ABC-130k dataset on Hugging Face as the raw substrate.

For robotics researchers, the significance is the packaging. Behavior cloning papers usually ship weights and a promise; this ships the training entry points, a simulator with a task catalogue and randomization, deployment code for real hardware, printable GELLO teleop hardware with a bill of materials, and conversion utilities, under Apache-2.0. The last push was on 2026-09-17, so the release is current rather than archival.

Setup: uv, CUDA 12.8, and honest hardware numbers

The environment targets Linux with an NVIDIA GPU on CUDA 12.8 and Python 3.12, managed through uv:

curl -LsSf https://astral.sh/uv/install.sh | sh

then clone, pin Python and sync from pyproject.toml. ffmpeg is the other system dependency. The README states hardware expectations without softening them: the reference DiT training run uses eight H100 or H200 GPUs with 80 GB of VRAM each, smaller machines reduce process count and batch size, and the minimum evaluation VRAM has not been established, which is a refreshingly frank unknown.

Data and checkpoints arrive through a prepare script with scoped options: a preview around 130 megabytes, the full bottles-in-bin training set around 35 gigabytes, the pretrained 75-thousand-step policy at 8.1 gigabytes, or single simulation tasks browsable by list, all landing in a cache directory you can relocate with an environment variable.

The smoke test that proves the pipeline

After setup, the documented smoke test downloads the bottles checkpoint and its simulation assets, then runs a short rollout on one GPU:

uv run prepare.py --checkpoint
uv run eval_policy.py \
    --checkpoint "${ABC_CACHE:-cache}/bottles_75k.pt" \
    --num-worlds 1 --num-chunks 2 --no-fast-inference \
    --save-video --video-every-n-actions 15

Success produces a summary JSON and a video under a named output directory, and the README says exactly what that proves: checkpoint loading, rendering and policy inference all work. It is equally clear about what it does not prove: two action chunks are too short to measure task success, so real evaluation needs the full instructions. The first launch compiles MJWarp CUDA kernels for about a minute, and the checkpoint bundles its vision backbone and normalization statistics so no separate DINO download is needed for this specific test.

Training ABC-DiT, with defaults published

Training runs through torchrun, with a documented resume flag that carries optimizer, scheduler and data-stream position. The README publishes the production reference configuration rather than hiding it in code: learning rate 1e-4 with a thousand-step linear warmup, AdamW at 0.9 and 0.95, weight decay 0.01, gradient clipping at 10, ten percent state masking, batch 90 per GPU, 75 thousand steps, and an hours-weighted two-component data mixture. Expected throughput is 2.6 to 3 iterations per second on H100-class hardware, converging to a loss near 0.048.

The result claim is tied to a video in the assets directory: with the full dataset pulled, the trained checkpoint should handle the bottles-in-bin task in simulation and on the real robot similarly to the referenced recording. A CLIP vision backbone is available as an alternative to DINOv3 through a model flag. Publishing defaults, throughput and expected loss together turns the training run from a leap of faith into a checklist.

ABC-VLA: a diffusion port of Gemma 3

The second policy family ports Gemma 3's vision-language architecture into a diffusion policy: the 4B model with SigLIP at 224 by 224, one selectable feature layer, learned-query pooling with QK normalization, a state token and a small AdaLN DiT head, with FAST tokens and alternative conditioners explicitly omitted. Training supports FSDP with documented restrictions and selects the policy by flag.

One licensing detail deserves attention: training from the Gemma base requires Google's own pyTorch checkpoint from Kaggle, under the Gemma Terms of Use, because the Hugging Face format is incompatible, so accepting Google's terms is part of the VLA path. The subtask and operator prompt options apply to the VLA as to the DiT, but the README notes neither has been validated for it yet, an honest boundary on the newer policy.

Real robots, teleop hardware, and the limits

The deploy package covers local and remote inference, real-time communication, GELLO teleoperation, DAgger-style data collection and recording, and the GELLO hardware itself is documented down to printable parts and a bill of materials, which makes the physical side of the stack reproducible for the price of a 3D printer. Simulation runs on MuJoCo through a Gym-style API with evaluators, and data conversion utilities bridge recordings into training format.

The limits are the honest ones of an open research release. Compute is the first wall: eight 80-GB GPUs for the reference run, with unknown minimums for evaluation. Licensing is the second: DINO weights come from Meta under their own terms and are not downloaded by the prepare script, and the Gemma checkpoint carries Google's terms. There are no releases or tags yet, only the moving main branch, so reproduction means pinning a commit. None of this hides the value: for a lab that wants the current behavior-cloning recipe with data, simulator and robot deployment in one Apache-2.0 tree, the pieces are all here.

Editorial conclusion

ABC fits robotics researchers and advanced hobbyists who want to train and evaluate behavior-cloning policies on published data with a real deployment path, and who can fund the GPU footprint the reference configuration states plainly. Skip it if you lack CUDA 12.8-class hardware, need a tagged stable release, or expect turnkey warehouse-style autonomy, which this research stack does not claim. Verify the pipeline end to end on your hardware: run the uv sync, the checkpoint smoke test with video output, and check the summary JSON, because a completed two-chunk rollout proves the stack works before you commit to the 35-gigabyte full training set.

Frequently asked questions

Official sources

1. amazon-far/abc on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README