Qwen3.8-27B on SGLang for DGX Spark: three serving modes, measured on the box
Qwen3.8 27B on SGLang for DGX Spark
At a glance
- What is it?
- MiaAI-Lab's repository packages Qwen3.8-27B serving for the NVIDIA DGX Spark into shell scripts and a .env file. The value is in the pinned flags and the three swap-in speculative modes, not in the model itself.
- Who is it for?
- Adopt it if you have a DGX Spark or GB10 box and want Qwen3.8-27B served without re-deriving the SGLang flags yourself; the .env.sample and the three start scripts are the whole interface. Skip it if you are on x86, on a multi-GPU server, or if you need YaRN context above 262144 together with DSpark or DFlash2, since the README states both draft configurations leak and cannot use it.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 6 days ago.
- What is it written in?
- Mainly Shell, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 16, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
Who the Qwen3.8-27B SGLang DGX Spark setup is for
The target machine is narrow: an NVIDIA DGX Spark or GB10 system, aarch64, SM121, with 128 GB of unified memory. The README lists Docker with the NVIDIA Container Toolkit working, plus docker, curl and an HF_TOKEN defined in ~/.bashrc. If that is not your hardware, the repository has nothing for you. There is no x86 path documented and no multi-GPU path documented.
What the project actually sells is a set of opinionated launch flags. The README says the flags start from the SGLang cookbook's DGX Spark cell (NVFP4 plus DSpark) and then pin choices measured on the author's box: GDN in bf16 rather than the cookbook's float32, extra_buffer_lazy, mem 0.90, chunk 8192, DSpark block 7 with 8 draft tokens, torch.compile with decode graphs, and a cpuset of 5-9,15-19. Each of those is a decision a first-time DGX Spark owner would otherwise have to make blind. The repository's contribution is the decisions plus the scripts that apply them, not a new inference engine.
What the start scripts actually do
Three shell scripts select a serving mode. start.sh runs MTP, start-dspark.sh runs DSpark, and start-dflash.sh runs DFlash2. All three read the same .env, and the README describes the precedence order explicitly: shell exports win, then .env, then start.sh's inline defaults. That layering means you can override a single value for one launch without editing the file, and it also means an exported variable you forgot about will silently beat your .env.
The scripts are idempotent. If the container is already running they report that and exit; if a stopped container exists they remove it. Configuration changes only apply on the next launch, so the documented cycle is ./stop.sh followed by the start script for the mode you want.
DFlash2 is the odd one out. start-dflash.sh does not use the model-specific image. It pulls lmsysorg/sglang:dev-qwen38-27b-dflash2 from Docker Hub on first run, about 14 GB compressed, pinned by its index digest. The README is candid about why: no released tag ships DFlash2, so the script depends on a CI build of an upstream dev branch. The digest pin freezes the content, but the tag itself is a dev tag. That is a real supply-chain trade-off, stated plainly rather than hidden.
Installing and serving a first request
There is no download step. The container pulls the checkpoint into ./.cache/huggingface on first start, roughly 24 GB for the default NVFP4 BF16-head repository. Copy the sample config once, then start a mode.
cp .env.sample .env
./start-dspark.shDSpark is the default recommendation for code in the README's own numbers. A fresh clone serves 262K context with YaRN off and 10 concurrent requests, because .env.sample ships YARN=0, CONTEXT_LENGTH=262144 and MAX_CONCURRENT_REQUESTS=10. Confirm the server is up by listing models.
curl http://127.0.0.1:8888/v1/modelsThe port is 8888. If the model list comes back, the engine is answering. To switch modes, stop and start again, since .env changes only take effect on the next launch.
./stop.sh
./start.shThat second pair runs MTP instead. The README's own summary numbers put DSpark around 51.5 on code and MTP around 34.5, with the long-essay probe favouring MTP at about 24 against DSpark's 18. Those figures come from the project's Measured on this box section; they are the author's on-device results, not a benchmark you can assume transfers to your workload.
Long context and concurrency up to 1M
The default is native 262K with YaRN off. Above that, the .env.sample requires YARN=1, and the scaling factor is derived rather than set: factor = round(CONTEXT_LENGTH / 262144). So 524288 gives 2.0, 786432 gives 3.0, and 1000000 gives 4.0. The sample notes that 2.0 and 4.0 are the model card's validated values and that in-between factors work but sit outside the card's tested points. It also warns that YaRN turns on implicitly at exactly 1000000 even if YARN=0.
The constraint worth reading twice is the DSpark one. The sample states that YaRN is not compatible with the DSpark speculative draft on this SGLang build, and the README repeats that DSpark cannot use YaRN or context above 262144, adding that DFlash2 has the same draft-config leak. So the two modes the README rates fastest on code are exactly the two you cannot combine with extended context. Choosing long context means giving up DSpark and DFlash2, and choosing them means staying at 262K. There is no documented workaround.
Concurrency is a separate dial. MAX_CONCURRENT_REQUESTS feeds the GDN state pool at four state slots per concurrent request, and the README notes the spec verify window is a separate engine-side buffer, verified in the build's kv_cache_configurator. Raising concurrency therefore costs memory in a way the sample does not quantify.
Where this repository is the wrong tool
The hardware requirement is the first filter and it is absolute. Nothing here helps on an x86 workstation, a multi-GPU node, or a Mac. The cpuset flag alone, --cpuset-cpus 5-9,15-19, assumes GB10's ten 3.9 GHz Cortex-X5 cores and its 2.8 GHz A725 efficiency cores. On any other CPU topology that pinning is at best meaningless and at worst counterproductive.
The DFlash2 path carries a second risk. It runs a dev-tag image from Docker Hub. The digest pin means the bytes will not change under you, but it also means you are on a build that upstream has not released, and the README itself frames it as the tag the cookbook maps to DGX Spark rather than a stable release. If your environment requires released, vendored images, start.sh and start-dspark.sh are the two you can use.
Finally, the tuning is specific to one box. The README calls the numbers measured on-device, and the GDN bf16 choice is justified as a 3% improvement over the cookbook's float32 on that machine. A 3% delta measured on one GB10 is not a guarantee on yours. The scripts give you a defensible starting point, not a validated optimum.
How it compares to running the SGLang cookbook cell directly
The obvious alternative is the SGLang cookbook's DGX Spark cell itself, which the README names as the origin of these flags. The difference is what each gives you. The cookbook gives a recipe; this repository gives three runnable scripts, a .env with documented precedence, and a set of deviations from the cookbook that the author measured. If you follow the cookbook directly you get float32 GDN, which the README reports as 3% slower here, and you make the memory, chunk and cpuset decisions yourself.
The second alternative is a different inference stack on the same hardware. The README does not compare against one, so there is nothing here to say about how SGLang's DSpark path stacks up against another engine's speculative decoding on GB10. The repository's comparison is internal: MTP versus DSpark versus DFlash2, all on the same box, with the caveat that the chat numbers for DFlash2 are described as streamed and that the README points readers to its counting notes rather than restating them.
A third option is simply not using speculative decoding. MTP is the conservative mode, and the README's own essay probe favours it. If your traffic is long-form text rather than code, the fastest-on-code modes are not obviously the right pick.
Licence, maintenance and upgrade cost
The repository is MIT-licensed. That covers the scripts and the .env.sample. It does not cover the model weights, the SGLang images, or the Docker Hub dev tag that start-dflash.sh pulls, each of which carries its own terms. The README does not discuss those terms, so check them at their sources before shipping anything.
On maintenance, the last push was on 2026-09-12. The repository is not archived. The README records that the earlier self-built DFlash2 image and its patch/ builder were retired on 2026-09-05, last carried by commit 751e29e, with an already-built local image still runnable via IMAGE=lmsysorg/sglang:qwen38-27b-dflash2. That is a concrete upgrade cost: if you built the old image, the supported path is now the pulled dev tag, and your local image survives only as a manual override.
The other upgrade cost is the dev-tag dependency. When upstream releases a tag containing DFlash2, the digest pin in start-dflash.sh becomes the thing to revisit. Until then, updating means changing that pin deliberately rather than pulling a moving tag. There are no releases in the repository, so upgrades arrive as commits to main and you should read the diff before pulling.
Editorial conclusion
Adopt it if you have a DGX Spark or GB10 box and want Qwen3.8-27B served without re-deriving the SGLang flags yourself; the .env.sample and the three start scripts are the whole interface. Skip it if you are on x86, on a multi-GPU server, or if you need YaRN context above 262144 together with DSpark or DFlash2, since the README states both draft configurations leak and cannot use it. Verify first that your Docker GPU passthrough works with docker run --gpus all, that HF_TOKEN is exported in ~/.bashrc, and that the ~24 GB checkpoint pull into ./.cache/huggingface fits your disk before you start the container.
Frequently asked questions
What is a DGX Spark used for in this project?
It is the only documented target. The README requires an NVIDIA DGX Spark or GB10 system, aarch64, SM121, with 128 GB of unified memory, and the cpuset pinning assumes GB10's ten Cortex-X5 cores.
What models can you run on a DGX Spark with this repository?
Qwen3.8-27B, in four checkpoint variants selected by QUANT: nvfp4 (the default, dense BF16 lm_head), nvfp4-fp4, fp8 and bf16. The container pulls the chosen checkpoint into ./.cache/huggingface on first start.
Which DGX Spark is best for serving Qwen3.8-27B?
The README does not compare DGX Spark models. It states one hardware requirement, GB10 with 128 GB unified memory, and pins the launch flags to that box.
What can you do with a DGX Spark running this setup?
Serve Qwen3.8-27B over an HTTP API on port 8888 with thinking mode enabled through the qwen3 reasoning parser and tool calling through the qwen3_coder parser. The default configuration is 262K context, YaRN off, 10 concurrent requests.
Community notes