SpecForge: Training EAGLE3, DFlash and Domino Drafters for SGLang Serving
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
At a glance
- What is it?
- SpecForge is the SGLang team's training framework for speculative decoding draft models, with one typed CLI entry point and a config tree that encodes method, topology and service ownership. The design bet is that porting effort, not training code, is what keeps speculative decoding out of production.
- Who is it for?
- Adopt SpecForge if you already serve on SGLang and want draft checkpoints that load without a separate conversion step, and if you can accept the framework's opinionated config tree. Do not adopt it if you need a method or topology outside the documented matrix, or if your serving stack is vLLM or TensorRT-LLM, since the stated benefit is SGLang compatibility.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The porting gap SpecForge is built to close
Speculative decoding needs a small draft model that proposes tokens and a large target model that verifies them. Training that draft model is the part most teams underestimate. The README states the motivation directly: many open-source speculative decoding projects are "not well-maintained or not directly compatible with SGLang." That second clause is the actual product thesis. A drafter that trains fine but needs a custom loader, a different tensor layout or a hand-written serving patch is not deployable on a schedule.
The intended user is an engineer who already runs SGLang for inference and wants to add a draft model without writing a porting layer. The framework is an ecosystem project from the SGLang team, licensed MIT, written in Python, and it targets PyTorch training with FSDP listed among its topics. It is not a general speculative decoding library and it does not try to be one. The scope is training plus handoff, and the handoff target is SGLang.
That narrowness is a feature for the intended audience and a hard boundary for everyone else. If your serving path is vLLM or TensorRT-LLM, the compatibility argument in the README does not apply to you, and you would be adopting a training framework whose main selling point is irrelevant to your stack.
One entry point, and a config path that encodes the topology
The README is explicit that there are "no method-specific Python training entry points." Every supported method goes through the same typed command:
specforge train --config examples/configs/online/disaggregated/external/qwen3-8b-eagle3-disaggregated.yaml
The interesting design decision is that the config path is not arbitrary. According to the README, "the path under examples/configs identifies feature mode, topology, and online service ownership." So examples/configs/online/disaggregated/external/ is not a folder someone happened to create; it encodes that this is an online run, disaggregated, with external service ownership. The same method can appear under offline/colocated/, offline/disaggregated/, online/disaggregated/external/ and online/disaggregated/managed-local/ with different operational meaning.
Service ownership is the part worth understanding before you pick a recipe. In an external recipe, SpecForge supervises the producer and consumer on one trainer node while "the user or scheduler owns Mooncake and SGLang." In a managed-local recipe, SpecForge also starts those services on the local host. That is a real operational difference: managed-local is convenient for a single machine and awkward if you already have a scheduler that expects to own service lifecycle.
Parallelism is split along a similar seam. The README states that online target parallelism belongs to SGLang, while deployment.trainer owns trainer DP and offline EAGLE3 USP process groups. Two systems, two configuration surfaces, and the boundary between them is drawn at the serving/training line rather than at the model.
The supported method matrix and its optimization pairings
The method table covers EAGLE3, P-EAGLE, EAGLE3.1, DFlash, DFlash2, Domino and DSpark. They are not variations on one implementation. EAGLE3 is described as feature-based autoregressive drafting; P-EAGLE is parallel EAGLE; EAGLE3.1 adds attention drift to the EAGLE3 approach; DFlash is block-parallel drafting; DFlash2 adds grouped dynamic convolution and top-k path selection; Domino is DFlash with GRU logit correction; DSpark is confidence-scheduled semi-autoregressive generation.
Each method carries its own example configs, and the availability is uneven. EAGLE3 has online external, offline colocated and offline disaggregated recipes. DFlash has online external, offline colocated and an online managed-local recipe. Domino has online external, offline colocated and a managed-local multiserver recipe. P-EAGLE, EAGLE3.1 and DFlash2 list only online external or online managed-local examples. If you want offline colocated P-EAGLE, the README does not show a config for it.
Optimization is also method-specific. EAGLE3 pairs with LK loss, and both DFlash and DFlash2 pair with D-PACE. P-EAGLE, EAGLE3.1, Domino and DSpark list no optimization in the table. That column is a useful signal about where the team has invested: the losses are not a global switch you can apply to any drafter.
One notable claim in the README is about failure behaviour rather than features: "Unsupported combinations are rejected during config validation or run assembly instead of falling back to an older trainer." That is the right default for a framework with this many method and topology combinations. Silent fallback would produce a checkpoint trained by the wrong recipe, and you would only discover it at acceptance-rate evaluation.
Getting a run started: commands and config surfaces
The documented path starts with the typed entry point and a recipe from the examples tree. The README's own command is the online disaggregated external EAGLE3 recipe for Qwen3-8B:
specforge train --config examples/configs/online/disaggregated/external/qwen3-8b-eagle3-disaggregated.yaml
If you want a simpler first run, the offline colocated EAGLE3 recipe is examples/configs/offline/colocated/qwen3-8b-eagle3-offline.yaml, and the offline disaggregated variant is examples/configs/offline/disaggregated/qwen3-8b-eagle3-offline-disaggregated.yaml. For a managed-local DFlash run, the README points at examples/configs/online/disaggregated/managed-local/qwen3-8b-dflash-1server-dp7-disaggregated.yaml, and for DFlash2 at examples/configs/online/disaggregated/managed-local/qwen3.6-27b-dflash2-disaggregated.yaml.
The config keys named in the material are few but load-bearing. deployment.trainer owns trainer DP and offline EAGLE3 USP process groups. Online target parallelism is configured on the SGLang side, not in the trainer block. Beyond those, the README does not enumerate the full schema, and the documentation links are where that detail lives: docs/sections/basic_usage/training.md for the supported method/topology matrix, and docs/sections/basic_usage/disaggregated_training.md for the online and offline launch workflows.
Two external dependencies appear in the online disaggregated path: Mooncake and SGLang. In the external recipes, you or your scheduler own both. The README does not describe Mooncake's role in the data flow beyond that ownership statement, so treat the disaggregated guide as required reading before you attempt an online run rather than assuming the config file is self-explanatory.
Where the documentation is thin and the design forces a choice
The README is a good index and a poor manual. It names methods, configs and guides, but it does not describe the training data format, the expected dataset preparation steps, the checkpoint output layout, or how acceptance rate is measured during training. Everything in the previous sections comes from the README's own text; anything about loss curves, throughput or convergence would be invented, and I am not going to invent it.
The config-path-as-specification design is the choice I would flag. Encoding feature mode, topology and service ownership into a directory structure makes the supported combinations discoverable and makes it obvious when you are off the matrix. It also means moving a recipe between modes is a path change plus whatever the YAML contains, and the README does not say how much of the config is shared across modes versus overridden per directory. If your infrastructure does not map cleanly onto external or managed-local ownership, you are choosing the closer of two options rather than the right one.
The method matrix is the second constraint. Seven methods sounds broad until you cross it with three topologies and two service-ownership models. The table shows that the intersections are sparse. A team that needs offline colocated training with DFlash2, or disaggregated training with P-EAGLE, has no listed recipe. That is not a bug, but it is a real limit on which combinations you can adopt today.
How SpecForge differs from a general speculative decoding trainer
The obvious alternative is a standalone speculative decoding training project, of which the README acknowledges there are many. The difference in approach is not the training algorithm, since EAGLE3, DFlash and the rest are published methods that any implementation can target. The difference is where the project draws its boundary.
A general trainer optimizes for method coverage and lets you handle serving integration yourself. SpecForge optimizes for a single serving target and treats integration as part of the deliverable. The README frames the goal as a framework where "no additional porting effort is required" for SGLang. That is a narrower promise than a general trainer makes, and it is a more useful one if SGLang is what you run.
The SpecBundle collection is the other half of that bet. It is described as a set of production-grade draft models released by the SpecForge team and industry partners, with the README claiming higher acceptance rate than existing open-source checkpoints across a range of domains and up to 4x speedup together with SGLang. Those are the project's claims, not measurements I performed, and the dashboard link in the README is where the underlying numbers would live. The practical implication is that SpecForge is not only a training tool: for some target models you may be able to take a released drafter instead of training one, and the framework is the path you would use if you later need a custom drafter for a model SpecBundle does not cover.
Maintenance, licence and what to verify before adopting
SpecForge is MIT licensed, and the README carries an MIT badge pointing at the LICENSE file. MIT is permissive, so the licence itself is unlikely to be the constraint on adoption. What MIT does not settle is the licence of the base models you train drafters for, or the terms attached to checkpoints you pull from SpecBundle. Those are separate questions and this is not legal advice; check them against your own distribution plans.
The repository is active, not archived. The README states the project is "regularly maintained by the SpecForge team," and the stated goal is that the code is "runnable out-of-the-box." No releases were retrieved for this review, which means versioned artifacts are not the distribution mechanism you should plan around. Pin a commit if you need reproducibility.
The maintenance cost that matters here is the method matrix. Seven methods across three topologies means the surface area the team maintains is large relative to the number of recipes actually listed in the table. The README's note that unsupported combinations are rejected at config validation or run assembly is a mitigation, not a substitute for checking whether your combination is supported. Before you build a pipeline around SpecForge, open docs/sections/basic_usage/training.md and confirm your method and topology appear in the matrix, then confirm the recipe you intend to use exists under examples/configs. If your combination is not there, the framework will tell you so rather than quietly training something else, which is the behaviour you want from a tool that produces checkpoints you intend to serve.
Editorial conclusion
Adopt SpecForge if you already serve on SGLang and want draft checkpoints that load without a separate conversion step, and if you can accept the framework's opinionated config tree. Do not adopt it if you need a method or topology outside the documented matrix, or if your serving stack is vLLM or TensorRT-LLM, since the stated benefit is SGLang compatibility. Before committing, read docs/sections/basic_usage/training.md for the supported method/topology matrix, then run the offline colocated EAGLE3 recipe at examples/configs/offline/colocated/qwen3-8b-eagle3-offline.yaml to confirm your hardware and data pipeline are accepted by config validation.
Community notes