Halo wraps parallelism around a Hugging Face model instead of replacing it

The model stays a Hugging Face model

Halo is an open-source framework built by White Circle for training large language and multimodal models, and the first design decision it makes is about what it refuses to change. It trains Hugging Face models directly. Trainers subclass Hugging Face or TRL trainers, checkpoints remain standard SafeTensors, and `from_pretrained` continues to work. The README claims that supporting a new model family typically takes under 140 lines of integration code, and that is the practical promise: the cost of adopting a new architecture is measured in a small patch rather than a reimplementation. The scope of the framework is broad, covering pre-training, SFT, preference optimization, distillation, fully asynchronous multi-turn RL, and the same codebase runs on one GPU or across multiple nodes.

Parallelism is added around the model

The second design decision is how the distributed work is organised. The README states it directly: parallelism is added to the existing model. Expert Parallelism wraps MoE blocks, context parallelism wraps attention, and tensor parallelism and expert tensor parallelism shard weights in place, with no separate distributed implementation of the model. The stack underneath is PyTorch, with FSDP2, DTensor and DeviceMesh providing the primitives, and DeepEP handling the all-to-all communication for expert parallelism. The configuration is constrained on purpose. EP, CP, TP and ETP each have their own setting, EP combines with CP, TP or ETP across GPUs and nodes, and other pairings are rejected at config time. That is a useful restriction. A framework that accepts every combination of every parallelism strategy is a framework where the interactions are discovered at run time, when the job is already expensive.

One infrastructure for every training method

The third point in the README's list of reasons is that training methods share the same infrastructure. Pre-training, SFT, preference optimization, distillation and RL use the same parallelism and checkpointing code, so a new method does not need its own distributed implementation. The examples directory shows the range, with `examples/classification/`, `examples/distillation/`, `examples/embedding/`, `examples/grpo/`, `examples/preference/`, `examples/reward/` and `examples/sft/`. The launcher confirms the same thing in practice, since `halo launch --list` covers sft, smpo, dpo, rlvr and environmental-grpo among others. Performance work is folded into the same layer rather than bolted on, with DeepEP V2, FlashAttention 4, Liger, Grouped GEMM, padding-free packing and an AdamWBF16 optimizer, and implementations plus fallbacks for Blackwell, Hopper and older GPUs.

Asynchronous RL with a clean split

The most distinctive piece is how reinforcement learning is separated from training. Multi-turn environment rollouts run as Ray actors against vLLM or SGLang servers, and with two or more servers a prefetch queue overlaps the next round's generation with training. The trainer pushes updated weights to the servers over native NCCL. The README is explicit about what is not there: there is no Megatron backend and no veRL dependency, and the training side stays plain Transformers with the same parallelism and checkpointing stack as everything else. The split is what makes the async design workable. The inference server is a well understood, separately versioned component, and the weight synchronisation is a point-to-point operation over NCCL rather than something entangled with the training loop.

Images per architecture, and no latest tag

Installation is by container, and the images are architecture specific. PyTorch, Flash Attention and DeepEP V2 are already built in, and the images can be pulled anonymously with no account required:

docker pull public.ecr.aws/whitecircle/halo:blackwell
docker pull public.ecr.aws/whitecircle/halo:hopper

Versioned tags such as `:blackwell-1.1.0` and `:hopper-1.1.0` pin the release, and there is deliberately no `latest` tag, since the images are architecture specific. The RL inference images are published alongside them, with vLLM and SGLang tags pinned the same way, as `:vllm-0.26.0-1.1.0` and `:sglang-0.5.17-1.1.0`. To build locally instead, the Makefile targets are:

make build-blackwell
make build-hopper

No host Python environment is required. The base image is NVIDIA's PyTorch container, and the Dockerfile comment explains that vLLM uses a separate image because its torch and transformers stack is incompatible with the training one.

One command per training run

`halo launch` takes a training method and a YAML config, and the examples in the repository show the shapes you need. A single GPU SFT run with LoRA, an expert parallel SFT run across eight GPUs, and an FSDP-2 run driven by Accelerate:

# SFT + LoRA (fits into a single GPU)
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yaml

# SFT + Expert Parallelism (MoE, 8 GPUs → torchrun)
halo launch sft examples/sft/qwen3_5/qwen3.5-35b-a3b-ultrachat-ep.yaml -n 8

The launcher decides between `python`, `torchrun` and `accelerate launch` based on the arguments, which is the piece of friction it removes. Options after the config that are not the launcher's own are passed to the trainer and override the YAML, so a learning rate or a maximum length can be changed on the command line without editing the file. Two utilities make the workflow safer: `--dry-run` prints the command without running it, and `--list` shows the available methods.

Tooling for checkpoints, quantization and environments

The other utilities go through `halo run`, and they cover the tasks that usually break after a distributed run finishes:

halo run merge-ep-shards --input_dir <ep-checkpoint-dir> --output_dir <merged-dir>
halo run quantize-to-lowp --input_dir <bf16-model-dir> --output_dir <nvfp4-out-dir> --format nvfp4

Merging expert parallel shards back into a loadable checkpoint and quantizing a bf16 model to a lower precision format are exactly the steps where distributed training leaves you with artefacts you cannot load with `from_pretrained`, and having them as first class commands is the difference between a usable framework and a pile of scripts. The tool list also covers `rm-scoring` and `run-env`, and the underlying scripts can be run directly, with `python scripts/training/sft.py` for a single process or `torchrun --nproc_per_node=8` for the distributed case, which matters when you need to step inside the launcher.

A custom licence, a young release line, and claims to weigh

The project metadata says `requires-python = ">=3.12,<3.13"` and a licence identifier of `LicenseRef-Halo` with licence files for both `LICENSE` and `APACHE-2.0.txt`, so the terms are not a standard identifier you can assume, and the file is what governs use. The release history is short and recent. Halo 1.0.0 was the first public release on 2026-08-20, covering EP, CP, TP and ETP on native Hugging Face models for both LLMs and VLMs, 15 MoE families, pre-training through multi-turn RL with vLLM or SGLang rollouts, and the bf16 AdamW optimizer. Halo 1.1.0 followed on 2026-10-11 with faster MoE training, exact resume and more reliable async RL, and the last push was on 2026-10-15. The performance claims are the project's own, with 1.67 times over Halo 1.0.0 for a Gemma 4 26B-A4B model on two GPUs, up to 27 percent faster expert permute for gpt-oss at EP8, and up to 3.5 times the throughput of stock TRL on 8 B300 GPUs. The repository also ships repo specific agent skills inside the training images for launching jobs, choosing parallelism settings and debugging, with the CLI baked in through a build argument.

Editorial conclusion

Adopt Halo when your team wants expert parallelism, context parallelism and asynchronous multi-turn RL while keeping training on native Hugging Face models and standard SafeTensors checkpoints, because the parallelism is added around the model instead of a second implementation of it. Do not adopt it for a single GPU experiment or without NVIDIA hardware, since the published images are architecture specific for Blackwell and Hopper and the framework assumes multi node scale. Verify first which parallelism combination your model and cluster support, since EP combines with CP, TP or ETP and other pairings are rejected at config time, and read the licence file, because the project uses a `LicenseRef-Halo` identifier alongside an Apache-2.0 file.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Releases
5. whitecircle/halo on GitHub