Miles: An Enterprise RL Framework for Trillion-Parameter Post-Training

What Miles Solves and Who It Serves

Miles addresses the problem of running reinforcement learning on models that are too large for a single node, where rollout generation and training must run at scale without stalling. The framework pairs SGLang for high-throughput rollout with Megatron-LM for scalable training, and it adds enterprise features like fault tolerance and low-precision training. The intended user is a team at a lab or company that post-trains frontier-scale LLMs and VLMs, with access to hundreds of GPUs. The README lists support for models like DeepSeek-V4, Kimi-K3, and Nemotron, and hardware from NVIDIA GB300 down to A100, plus AMD MI300X and MI355X. That is a narrow slice of the market, but a significant one. If you work with smaller models or a single GPU, Miles is overkill, and the Megatron dependency will be a burden.

Architecture: SGLang for Rollout, Megatron for Training

The core design is a decoupled pipeline. Rollout generation runs on SGLang engines, which sit behind a router that spreads requests across the fleet, preserves per-request metadata, and health-checks each engine. Training runs on Megatron-LM, with a PyTorch FSDP2 backend available as an alternative. The two sides are fully async, meaning rollout and training workers do not wait on each other. The README describes configurable on- and off-policy schedules and customizable async rollout and eval modes. The decoupling is what allows the framework to keep GPUs busy, but it also adds moving parts. The router, the engines, and the trainer must all stay in sync, and the documentation points to a "Fully Async RL" page for details, which the README does not reproduce. The key data flow is that rollouts produce tokens, which go to the trainer, and updated weights go back to the engines. The README highlights a token-in-token-out (TITO) feature that avoids detokenize and retokenize round-trips, which is a real efficiency gain for agentic workloads.

Getting It Running: Installation and Quick Start

The README links to an installation guide and a quick start, but it does not include the actual commands. What it does specify is a hardware requirement: NVIDIA GB300, GB200, B300, B200, H200, H100, or A100, plus AMD MI300X, MI325, MI350, or MI355X via ROCm. The documentation page has per-GPU status and a container image for each. That suggests installation is container-based, which is typical for this scale. The quick start is at miles.radixark.com/docs/getting-started/quick-start, and the launch script walkthrough is at miles.radixark.com/docs/user-guide/launch-script. Those pages will have the real commands, but from the README alone, you cannot run Miles. The lack of a pip install or a simple curl command in the README is a signal that this is not a tool you install on a laptop. Expect to build or pull a container, configure a cluster, and then write a launch script that specifies the model, the training backend, and the rollout settings.

Performance Features: P2P Weight Transfer and Low-Precision Training

Two features stand out for large-scale runs. First, P2P RDMA weight transfer moves new weights from the trainer to the engines in seconds, even for a trillion-parameter model like Kimi-K2.6. The README claims "updating 1 T parameters in seconds" and calls P2P the fast path for disaggregated setups. That is a concrete mechanism, using RDMA to bypass the CPU and network stack. Second, low-precision training with MXFP8 and NVFP4 is supported, with a numerically stable RL recipe that reduces precision-induced divergence. The README also lists FP8, INT4 QAT, BF16, and FP16. These features matter because precision loss is a common failure in RL, and the claim of a stable recipe is a differentiator. However, the README does not give benchmark numbers or a comparison to BF16 baselines, so you cannot verify the stability or the speedup from these claims alone.

Correctness and Resilience: R3 and Fault Tolerance

Miles addresses two failure modes that plague large RL runs. The first is MoE routing mismatch. In a mixture-of-experts model, the router can assign tokens to different experts during rollout and training, which destabilizes gradients. Miles implements Rollout Routing Replay (R3), which records the expert routing during rollout and replays it in the trainer's forward pass. The README says this removes the mismatch and overlaps compute and communication to keep the cost down. The second is engine failure. When an SGLang engine dies, Miles recovers it and resumes the run in place, with no restart and no pause. That is a strong claim, and the README does not describe the recovery mechanism or any limitations. You would need to test this in your own environment. The fault tolerance feature is a genuine advantage over frameworks that crash the whole run on a single node failure, but it is also a place where the documentation is thin.

Limitations and When It Is the Wrong Tool

Miles is not for small-scale experimentation. The Megatron-LM backend is designed for multi-node training, and even the FSDP2 backend is aimed at runs that want to train the HuggingFace implementation as-is, not at single-GPU fine-tuning. The README states that "the recipes, the parallelism, and the largest models all live on Megatron-LM," which means the FSDP2 path is a secondary citizen. If you have a single GPU or a small cluster, you will spend more time managing the cluster than training. Another limitation is the reliance on SGLang for rollout. If your model is not supported by SGLang, or if your hardware is not in the list, Miles will not work. The README claims day-0 support for several frontier models, but that is a moving target, and the supported models page is the only way to check. Finally, the framework is new. The only release is v0.1.0 from August 2026, so expect breaking changes and incomplete documentation.

Alternative: The Upstream slime Project

Miles was forked from slime, a project from THUDM. The README says Miles is "co-evolving with slime," which means the two projects share a lineage but have diverged. The key difference is that Miles adds enterprise features like P2P weight transfer, MXFP8 and NVFP4 training, and fault tolerance. Slime, presumably, has a simpler architecture and may be easier to run, but it likely lacks those features. If you need the enterprise capabilities, Miles is the choice. If you want a more community-driven project with less operational overhead, slime is a viable alternative. However, the README does not describe slime's feature set, so you would need to evaluate it separately. Another alternative is a framework like TRL, which is lighter and works with HuggingFace models, but it does not have the same scaling story for trillion-parameter models.

Maintenance and License Implications

Miles is licensed under Apache-2.0, which is permissive and allows commercial use, modification, and redistribution. That is a low-license-risk choice for enterprises. The project is actively maintained, with the last push on August 18, 2026, the same day as the v0.1.0 release. That is a single release, so there is no history to judge the maintenance cadence. The README links to a contribution guide, which suggests the project is open to external contributions, but it is still early. The upgrade cost is unknown. With a v0.1.0 release, you should expect frequent version changes, and the documentation may lag behind the code. The dependency on SGLang and Megatron-LM means you will also need to track their release cycles. Before adopting, check the documentation for the supported models and hardware, and plan for a pilot run to test the fault-tolerance and R3 features in your own environment.

Editorial conclusion

Miles is for organizations that train LLMs or VLMs at hundreds of billions or trillions of parameters and need the throughput of SGLang rollout plus Megatron training, with features like P2P RDMA weight transfer and MXFP8 low-precision training. It is not for small teams or quick experiments; the Megatron dependency and the operational complexity of a disaggregated fleet will overwhelm a single-GPU setup. Before adopting, verify that your specific model and hardware combination appears in the supported models list and the installation matrix, since day-0 support is claimed for several frontier models but the documentation is the only source of truth. Also check the fault-tolerance and R3 features against your own MoE routing needs, because the README does not describe failure modes or edge cases. If you need a simpler, more community-tested path, consider the upstream slime project or a lighter framework like TRL, but expect to lose the enterprise features. Miles v0.1 is a fresh release, so expect rapid changes and plan for upgrade churn.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes