LoongForge: a multi-backend training framework for LLMs, VLMs, diffusion and embodied models
A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs.
At a glance
- What is it?
- Baidu's Baige team has open sourced a training framework that splits into a patched-Megatron stack for language, vision-language and diffusion models and a torch-native stack for embodied models, with ready-to-run configs for more than 40 models and support for both NVIDIA GPUs and Kunlun XPUs. The interesting decision is the split itself: two backends rather than one, because the parallelism that suits a 70B language model is not the parallelism that suits a vision-language-action policy.
- Who is it for?
- Adopt LoongForge if you are training LLMs, VLMs, diffusion or embodied models on NVIDIA GPUs or Kunlun XPUs and you want a runnable config rather than a blank repository: the configs/models directory and the examples tree cover pre-training, continued pre-training, SFT and LoRA, and the embodied subsystem ships its own DDP, ZeRO-1, FSDP and HSDP paths.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem LoongForge is aimed at, and who it is for
Training a model family is not one problem. A dense LLM, a mixture-of-experts LLM, a vision-language model, a diffusion model and a vision-language-action policy each want a different split of parameters, activations and data across devices, and each hits a different bottleneck first. A single training framework that tries to serve all of them usually ends up serving one of them well. LoongForge's answer, visible in the architecture section of the README, is to stop pretending: it ships two stacks and routes model families to the one that fits.
The audience is therefore narrower than the tagline suggests. This is for teams that already know which model family they are training and want a working starting point, not for someone exploring what a training framework does. The README points at configs/models for ready-to-run configurations and examples for launch scripts, covering pre-training, continued pre-training, SFT and LoRA. The embodied side is explicitly a standalone subsystem under loongforge/embodied rather than a mode of the main stack.
The project is developed by the Baidu AI Cloud Baige team and is described as open sourced from AIAK-Training-LLM, a training acceleration suite. The README states that production runs have reached 5,000+ XPUs. That is a claim about the originating deployment, not a guarantee about yours, and it is the kind of number you should treat as context rather than evidence.
Two stacks, two sets of assumptions
The Megatron stack covers LLMs, VLMs and diffusion models. It is built on a patched Megatron-LM, published separately as Loong-Megatron, and the README lists the extensions: MoE parallelism, per-component heterogeneous parallelism, and long-sequence optimizations. The phrase per-component heterogeneous parallelism is the one worth pausing on. It means different parts of a model can be parallelised differently, which is what you need when a vision tower and a language decoder have wildly different shapes, and it is also what makes the configuration surface larger than a single-strategy framework.
The torch-native stack covers embodied models, described as VLA (vision-language-action) and WAM. It is a standalone subsystem supporting DDP, ZeRO-1, FSDP and HSDP, with optimizations described across I/O, communication strategy and kernel efficiency. That is a conventional menu of data and sharding parallelism, which makes sense for models that are typically far smaller than a frontier LLM and are more often limited by data loading and input pipeline throughput than by the all-reduce of a trillion-parameter model.
The split is a real design decision with a real cost. Two stacks mean two sets of launch scripts, two sets of environment expectations and two places to look when something breaks. The README does not describe a shared abstraction layer between them, and the architecture diagram is described as showing them alongside each other rather than unified. If you work across both families, expect to learn both.
Getting a run started: configs, examples and Docker
The material gives three entry points. The first is configs/models, which the README describes as ready-to-run configurations for supported models. The second is examples, which holds launch examples. The third is prebuilt Docker images, published under the loongforge organisation on Docker Hub, which the release notes say were unified so that all model families (LLM, VLM, VLA, Diffusion) share a single image. That unification is a practical improvement: before it, you would have needed to know which image matched your model family.
For the embodied stack, the README points at a concrete script rather than a description: examples/embodied/dreamzero/run_dreamzero_wan22_5b_full_fsdp_finetune.sh, added for a DreamZero Wan2.2-5B FSDP recipe. The recipe is described as including cache-aware data loading, compiled attention blocks, frozen-module handling and Delta-FP8 AllGather. Those four items tell you what the maintainers found worth optimizing for this model, and they are the parts of the script worth reading before you adapt it.
What the supplied material does not give is a full quickstart command sequence, the contents of any config file, or the names of individual config keys. The README references a Quick Start section and links to Read the Docs, but the actual commands are not reproduced in the text available here. Treat the repository's configs/models and examples directories as the source of truth for exact invocation, and read the script you intend to run rather than assuming a uniform interface across models.
Where the framework will not help you
Hardware is the first boundary. The badge and description name NVIDIA GPUs and Kunlun XPUs. If your cluster is neither, nothing in the material suggests a path. That is a narrower hardware envelope than frameworks that target a generic accelerator abstraction, and it is a deliberate consequence of optimizing kernels per platform.
Model coverage is the second boundary. The supported list is long (the README advertises 40+ ready-to-run model examples) but it is a list, not a general mechanism. Recent additions include Kimi-K3 with BF16 training for both LLMs and VLMs, GLM-5.2 plus a GLM-5.2 with MoonViT custom composition for adding multimodal capability, MiniCPM-V-4.6, Qwen3.8-27B, and Wall-OSS-0.5 for VLA. If your model is not on the list, you are porting, and the README does not describe a documented porting procedure.
There is also a correctness constraint hiding in the performance claims. The README states that optimizations keep training loss curves aligned with the baseline, and the DreamZero example is captioned as reaching 4.38x baseline throughput while the loss curves stay aligned. Alignment is the claim, not improvement. If your reason for switching is that you believe the framework will produce a better model, that is not what is being offered. What is offered is the same trajectory, faster. Verifying that alignment on your own data is the first thing to do after a successful launch, and it is more work than verifying that the job did not crash.
The MoE load-balancing work, and what it implies about the roadmap
The most specific technical item in the release notes is the TAOT paper, described as topology-aware dynamic expert replica placement for expert-parallel load imbalance in MoE training. The stated results are up to 74% overhead reduction over industry solutions and a 1.43x speedup measured on a real training case. Those numbers come from the paper and the project's blog post, not from an independent run, so treat them as the authors' measurements under their conditions.
The design idea is worth noting regardless of the numbers. Expert parallelism suffers when some experts receive far more tokens than others, and the usual remedies either drop tokens or add a balancing loss that changes what the model learns. Dynamic replica placement instead moves copies of hot experts onto the devices that need them, which requires knowing the network topology because a replica on the wrong link costs more than it saves. That is why the method is described as topology-aware.
This tells you something about where the project's effort goes. The MoE and expert-parallel path is being actively developed, which is good if you train MoE models and less relevant if you do not. It also means the parallelism configuration space is growing, and the gap between a default config and a tuned one is widening. A config that runs is not the same as a config that runs at the throughput the release notes describe.
How this differs from a single-backend framework
The obvious comparison is Megatron-LM itself, since LoongForge's LLM, VLM and diffusion path is a patched fork of it published as Loong-Megatron. The difference in approach is that upstream Megatron-LM is a general training library you configure and extend, while LoongForge is a curated set of model-specific recipes layered on a fork. Choosing LoongForge means accepting the fork's divergence from upstream in exchange for the recipes and the Kunlun XPU path. Choosing upstream means tracking the mainline but writing your own parallelism configuration for each model family and having no XPU story.
For the embodied side the comparison is different, because the torch-native stack is not a Megatron derivative at all. It is closer in spirit to a plain PyTorch distributed training setup with FSDP or HSDP, plus model-specific work on I/O and kernels. If your embodied model already trains under a hand-written FSDP script, LoongForge's value is the accumulated optimization and the evaluation module rather than the parallelism primitives, which are the ones PyTorch already provides.
That evaluation module is worth flagging as an under-described piece. The release notes say it is a unified evaluation module for the embodied stack, currently covering Pi0.5, xVLA and GR00T. Coverage of three models is a start, and the note that more are on the way is a statement of intent rather than a capability. If evaluation is central to your workflow, check what the module actually reports before assuming it replaces whatever you use now.
Maintenance, versioning and the licence
The release cadence visible in the material is roughly quarterly for tagged versions: v0.1.0 in May 2026, v0.1.1 later that month, v0.2.0 in August 2026, with the last push to the default branch in September 2026. The news entries are denser than the tags, which suggests model support lands on master between releases. If you depend on a specific newly supported model, you may need to track master rather than a tag, and that changes your upgrade risk profile.
The upgrade cost has a specific shape here. Because the LLM, VLM and diffusion path depends on a patched Megatron fork, a LoongForge upgrade can require a matching Loong-Megatron revision, and a change in either can move your throughput or your memory footprint. Pin both. The unified Docker image reduces one class of drift by putting all model families in one image, but it also means an image bump touches every family at once, including the ones you are not training.
The licence is Apache-2.0, which permits commercial use and modification and includes a patent grant, with the usual requirements around retaining notices and stating changes. That is a permissive licence and it is compatible with most corporate policies, but the framework depends on Megatron-LM and on model weights and code from the model publishers, each of which carries its own terms. The LoongForge licence does not settle those. Check the licence of the specific model you plan to train, and check the terms attached to any prebuilt image you pull, before you build a pipeline on top of it. This is a description of the licence text, not legal advice.
Editorial conclusion
Adopt LoongForge if you are training LLMs, VLMs, diffusion or embodied models on NVIDIA GPUs or Kunlun XPUs and you want a runnable config rather than a blank repository: the configs/models directory and the examples tree cover pre-training, continued pre-training, SFT and LoRA, and the embodied subsystem ships its own DDP, ZeRO-1, FSDP and HSDP paths. Do not adopt it if you need a framework you can read end to end in an afternoon, if your hardware is neither NVIDIA nor Kunlun, or if you depend on a model family outside the supported list and cannot absorb a port. Before committing, verify three things against your own target: that the specific model you intend to train has a config under configs/models or an example under examples, that the parallelism strategy documented for that model matches your cluster topology, and that the training loss curve reproduces the baseline you are migrating from, since the project's own claim is alignment rather than improvement. The Megatron stack is a patched fork, so pin the Loong-Megatron revision alongside the LoongForge release.
Community notes