Qwen3.8-Flash-Next on One DGX Spark: What the Single-Spark Recipe Actually Constrains
Qwen3.8-Flash-Next on ONE DGX Spark (TP=1)
At a glance
- What is it?
- A self-contained vLLM recipe for serving the 99 GiB Mia-AiLab Qwen3.8-Flash-Next-NVFP4 checkpoint from one DGX Spark's 121 GiB unified memory, with a host-side memory cap that quietly overrides the KV cache setting you think you are choosing.
- Who is it for?
- Adopt this if you already own a DGX Spark and want the Qwen3.8-Flash-Next-NVFP4 checkpoint served locally with vision and video input, and you accept that the KV pool is decided by HOST_RESERVE_GIB rather than by KV_TARGET_GIB. Do not adopt it if you need multi-node throughput, a documented rollback path, or a licence you can ship inside a closed product.
- Can I use it commercially?
- Yes, with strict conditions. AGPL-3.0 is a network copyleft licence: if people use a modified version over a network, for example as a hosted service, you must offer them its source code under the same licence.
- Is it still maintained?
- Yes. The repository last received commits 8 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 16, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The 99 GiB checkpoint on a 121 GiB box
The problem is arithmetic. The Mia-AiLab/Qwen3.8-Flash-Next-NVFP4 checkpoint is 99 GiB, and a single DGX Spark has 121 GiB of unified memory shared between host and GPU. Loading the weights the ordinary way leaves too little for a KV cache large enough to be useful, and too little for the host processes that keep the machine alive. The repository is a recipe for that squeeze, not a framework: shell scripts, an .env.sample, a bench directory, and a vLLM container image named vllm/vllm-openai:qwen38-flash-next.
It is aimed at people who already have the hardware. The README states the recipe is self-contained and that nothing in it depends on the 2-node files it was derived from, which matters because the obvious alternative for a model this size is to spread it across two Sparks. The author's own measurements are included, dated 2026-09-04 through 2026-09-06, and the README is unusually candid about which row came from which configuration. That candour is the most useful part of the project: the numbers move a lot between settings, and a reader who skims the tables will misread them.
The model itself is a vision-language model. Text, images and video all work out of the box according to the README, which is a claim about the served endpoint rather than about any script in this repository.
PLE offload, memory-mapped tables, and where the memory actually goes
The mechanism that makes a 99 GiB checkpoint fit is the PLE table. The README describes it as offloaded and memory-mapped, and start.sh budgets roughly 27 GiB for a packed PLE table built on first launch. So the disk requirement is about 130 GiB free: 99 GiB for the checkpoint and 27 GiB for that table. The table is not shipped; it is derived locally the first time you launch.
The KV cache is the other half of the budget, and this is where the design gets interesting. KV_TARGET_GIB is described in .env.sample as "A wish, not a grant". The GPU budget is capped at MemTotal minus HOST_RESERVE_GIB, and start.sh prints "KV target X reduced to Y" when the cap binds. On the author's host it binds every time: a wish of 20 GiB was measured over ten launches at 16.67 GiB, with 18.16 GiB when MTP is disabled. The README is explicit that 20 and 22 GiB of actual KV are no longer reachable through KV_TARGET_GIB alone, and that a pinned GPU_MEMORY_UTILIZATION still gets you there, with a warning.
That inversion is the project's central trade-off. The shipped default is 262,144 context with YARN off, MTP 3, HOST_RESERVE_GIB=26, KV_TARGET_GIB=20, KV_CACHE_DTYPE=fp8, MAMBA_SSM_CACHE_DTYPE=bfloat16, MAX_NUM_SEQS=4 and MAX_NUM_BATCHED_TOKENS=2048. The host reserve wins over the KV wish, deliberately, because the README records that earlier values of 22 lost servers: three on 2026-09-04, on a host left with 6.9 to 8.8 GiB of MemAvailable against a 6 GiB watchdog floor and a GPU driver that refuses allocations before that. The cap is a response to a real failure, not a conservative default.
Installing the recipe and getting to a first request
The README gives a four-command flow. Copy the sample environment file, download the checkpoint, start the server, and stop it when done.
cp .env.sample .env # edit IMAGE / HF_TOKEN if needed
./download.sh # fetch the ~99 GiB checkpoint (resumable)
./start.sh # ~10-12 min to /health; serves on :8888
./stop.sh # container + watchdog, gracefulTwo details in that flow are worth reading twice. download.sh is resumable, which you want for a 99 GiB transfer. And start.sh never downloads anything: it resolves the checkpoint from the local Hugging Face cache and fails fast if it is absent. Expect roughly 10 to 12 minutes to /health on the author's hardware; one measured run reached it in 10 minutes 51 seconds with the checkpoint read from NVMe.
Before you commit the disk, you can inspect what the script would do without running it. The README documents a dry-run flag that prints the derived memory budget and the docker command.
./start.sh --no-launchIf you want the abliterated checkpoint instead of the stock Mia NVFP4 one, the environment file exposes a switch. The README states that the gated repository requires accepting its terms on Hugging Face and setting HF_TOKEN first, and that the full snapshot is the same size as the stock one.
ABLIT=1 ./download.shThe server listens on port 8888, and the served model name is set by SERVED_MODEL_NAME in .env.sample. Overrides are environment variables, so a shorter context for a quick test is a prefix on the start command rather than an edit: the file's own comment shows MAX_MODEL_LEN=65536 ./start.sh. Note that MAX_MODEL_LEN cannot exceed the native 262144 when YARN=0.
Stopping cleanly, and the shared memory leak the README warns about
The container runs with --ipc host, which means POSIX shared memory segments vLLM leaves behind land on the host's /dev/shm and stay there until reboot. stop.sh sends SIGTERM and waits up to STOP_TIMEOUT, default 30 seconds, so vLLM can unlink those segments. ./stop.sh --force skips the wait.
This is a genuine failure mode, and it is the kind that bites people who treat a stop script as a formality. If you kill the container with docker directly, or if the wait is too short for your hardware, you leak shared memory into a host that has none to spare. Given that the whole recipe exists to survive on 121 GiB of unified memory, a leak that persists until reboot is not a cosmetic issue.
The README does not document rollback of a running deployment, and it does not describe what happens if stop.sh times out and you start again on the same host. Treat the graceful path as the only supported one and check /dev/shm yourself if a launch behaves oddly after a forced stop.
Where the measurements stop and the shipped configuration begins
The README flags its own gap, and it is worth taking at face value. KV_TARGET_GIB=20 was measured end to end over ten launches, including a 1/2/4/8-stream decode sweep, needles at 15/15 for 32k, and a 45-minute soak on 2026-09-06. Every one of those ran at 512k YaRN, not at the shipped 262k native rope. The 262k row in the profile table is at BF16 and predates the MADV_RANDOM mmap change. Short-context prefill at 32k is measured for both dtypes.
So the exact combination you get by copying .env.sample and running start.sh is not the combination with the deepest measurement record. That is not a reason to avoid the project, but it is a reason to run your own soak before trusting it with anything long-running. The README also notes that the decode numbers in the profile table predate a 2026-09-05 optimisation pass and are superseded by the sparkDash sweep, which reports 48.7 tok/s single-stream prose and 162.9 aggregate at 8 streams. Those superseded rows are still in the table, labelled as such. If you quote a number from this repository, quote the label with it.
One more caution on the tables: the two prefill columns in the sparkDash section are explicitly not a clean A/B, because they differ in rope config and KV target as well as chunk width. The README says so directly. Reading them as a like-for-like comparison would be a mistake the author already warned against.
What you give up compared with a multi-node deployment
The honest alternative for a model of this size is the 2-node setup this recipe was derived from. The difference is approach, not just scale: two Sparks give you tensor parallelism across nodes, so the weights and the KV cache are split rather than compressed into one box, and you are not fighting a 121 GiB ceiling or a host reserve that clips your KV wish. The README states that nothing here depends on the 2-node files, which tells you the author treated single-node as the constraint to solve rather than as a stepping stone.
The cost of that choice shows up in the numbers. A 512k YaRN request at KV_TARGET_GIB=20 and BF16 measured 1,537 tok/s prefill with a TTFT of 260.3 seconds. That is a long wait before the first token on a very large prompt, and it is a property of fitting the model on one device rather than of the scripts. If your workload is many concurrent long-context requests, the single-Spark profile is the wrong tool, and MAX_NUM_SEQS=4 in the shipped configuration says as much.
The other alternative is not to run this model at all. The search questions around this hardware ask whether DeepSeek V4 Flash can run on a DGX Spark, which is a question this repository does not answer. Nothing in the README describes support for any checkpoint other than Mia-AiLab/Qwen3.8-Flash-Next-NVFP4 and the gated abliterated variant, so do not read the scripts as a general-purpose serving harness.
Licence, upgrade cost, and the maintenance picture
The repository is AGPL-3.0. That is a copyleft licence with a network clause, and it is worth understanding before you build a hosted service on top of these scripts. If you modify the recipe and expose it to users over a network, the AGPL's source-availability obligation is the question to put to your own counsel. The model checkpoint is a separate artefact with its own terms on Hugging Face, and the abliterated variant sits behind a gated repository whose terms you must accept. This is not legal advice; it is a list of the three places where terms apply.
Upgrade cost is low in the ordinary sense and awkward in one specific way. The interface is a handful of environment variables, so changing a value is cheap. But the values are coupled: KV_TARGET_GIB interacts with HOST_RESERVE_GIB, MTP_NUM_SPECULATIVE_TOKENS interacts with the memory budget, and MAX_MODEL_LEN interacts with YARN. An upgrade that raises one number can silently push the cap into binding harder than before. The README's own history is the evidence: the shipped KV_TARGET_GIB went from 22 to 20, and both values lost servers.
On activity, the last push was on 2026-09-10, and the repository is not archived. There are no retrieved releases, so there is no versioned artefact to pin; you are tracking the main branch. A CHANGELOG.md exists in the repository layout, and the README carries dated measurements, which is a more useful signal here than a version number would be.
Editorial conclusion
Adopt this if you already own a DGX Spark and want the Qwen3.8-Flash-Next-NVFP4 checkpoint served locally with vision and video input, and you accept that the KV pool is decided by HOST_RESERVE_GIB rather than by KV_TARGET_GIB. Do not adopt it if you need multi-node throughput, a documented rollback path, or a licence you can ship inside a closed product. Before committing, run ./start.sh --no-launch to see the derived budget and the docker command, and confirm you have 130 GiB free for the checkpoint plus the packed PLE table.
Frequently asked questions
How much disk space does Qwen3.8-Flash-Next on one DGX Spark need?
The README says to budget about 130 GiB of free disk: 99 GiB for the checkpoint plus roughly 27 GiB for the packed PLE table built on first launch.
Does Qwen3.8-Flash-Next on one DGX Spark support images and video?
Yes. The README describes the checkpoint as a vision-language model and states that text, images and video all work out of the box.
Which DGX Spark is best?
The README does not compare DGX Spark models, so this project cannot answer it. What it does document is a serving profile measured on one host with 121 GiB of unified memory, where the GPU budget is capped at MemTotal minus HOST_RESERVE_GIB.
Community notes