DeepSeek-v4-Flash-DSpark-2x-DGX-Spark: a two-node vLLM recipe for DeepSeek-V4-Flash on 2x DGX Spark
DeepSeek-v4-Flash 0731 recipe for 2x DGX Sparks
At a glance
- What is it?
- MiaAI-Lab's repository packages a two-node DGX Spark deployment of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp: vLLM with TP=2, DSpark speculative decoding, a 1M-token context ceiling and nvfp4_ds_mla KV cache. It is a cluster recipe, not a single-machine quickstart, and the README is explicit about what it does not support.
- Who is it for?
- Adopt this if you already have two DGX Sparks on a working RoCE/NCCL fabric and want the Vision-Exp checkpoint served with vLLM TP=2, DSpark speculation and a 1M-token ceiling without writing the launcher yourself. Do not adopt it for a single Spark, for video input (the official weights have no video encoder), or if you cannot pin both nodes to the same container image.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 2 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 17, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What the 2x DGX Spark recipe actually deploys
The repository is not a model. It is an operations layer around deepseek-ai/DeepSeek-V4-Flash-Vision-Exp, the Vision-Exp variant of DeepSeek V4 Flash, served by vLLM across two DGX Spark nodes with tensor parallelism set to 2. The README describes DSpark speculative decoding, a 1M-token context ceiling, and nvfp4_ds_mla as the KV cache format. Native image support is described as a startup hotfix on the Anemll 0.1.1 runtime, combining the ViT and Aligner from the Vision-Exp checkpoint and accepting OpenAI image_url inputs or the <image>path</image> form.
The audience is narrow on purpose. You need two DGX Sparks, a working RoCE/NCCL fabric, and the same container image on both machines. The README states that start refuses to launch if either node is missing the image, so the recipe assumes you control both hosts. Anyone looking for a laptop or single-GPU path is in the wrong repository. The README also notes that the old Qwen3-VL sidecar and MCP path has been removed, so older write-ups that describe a sidecar encoder no longer match the tree.
How the two-node layout and KV ceiling fit together
Everything runs from the head node. The default image is ghcr.io/anemll/dspark-vllm-gx10:0.1.1, and the default checkpoint is the official Vision-Exp revision 86f746b36186f0e567729a5c06a8c918caba82a9 with ABLITERATED=0. The served model name is deepseek-v4-flash-vision-exp, the context ceiling is MAX_MODEL_LEN=1048576, concurrent sequences default to MAX_NUM_SEQS=6, and MAX_NUM_BATCHED_TOKENS defaults to 8192. Speculative decoding uses MTP_NUM_TOKENS=6, which the README says must be at least the dspark_block_size of 5 and divisible by the Vision-Exp n_predict of 3.
The interesting part is how the README frames the 1M ceiling. It states plainly that max_model_len and max_num_seqs are ceilings, not reservations, and that the real limit is sum(live tokens) ≤ KV pool. The boot log quoted in the README shows 17.04 GiB of KV cache memory, 2,331,430 tokens of GPU KV cache size, and 2.22x maximum concurrency for 1,048,576 tokens per request. Six normal agent turns fit; six simultaneous full-1M requests do not. That distinction is the one most operators get wrong when they read a 1M number and assume six parallel long-context jobs.
Weights can live on both nodes or only on the head. With the default DSPARK_WORKER_HF_NFS=0, prepare downloads the checkpoint onto the worker as well. Setting DSPARK_WORKER_HF_NFS=1 keeps weights on the head and has the worker mount that cache over NFSv4 on the ConnectX link, following the same pattern the README attributes to Qwen3.8-Flash-vLLM. The repository also ships a separate three-node launcher, start-tp3.sh, which needs WORKER2_HOST; the README says it is separate so that .env cannot accidentally flip the two-node path.
Installing it: env, image, weights, then start
The README gives a numbered quick start. First copy the environment template, then edit it. The fields it says to set at minimum are WORKER_HOST, MASTER_ADDR, NCCL_IB_HCA, NCCL_SOCKET_IFNAME (plus matching TP_ and GLOO_ interface names), VLLM_HOST_IP, WORKER_VLLM_HOST_IP and HF_CACHE. The example fabric block in the README looks like this, and you are expected to change the interface and GID for your own ring:
WORKER_HOST=10.0.0.2
MASTER_ADDR=10.0.0.1
VLLM_HOST_IP=10.0.0.1
WORKER_VLLM_HOST_IP=10.0.0.2
NCCL_IB_HCA=rocep1s0f1
NCCL_SOCKET_IFNAME=enp1s0f1np1
DSPARK_VLLM_IMAGE=ghcr.io/anemll/dspark-vllm-gx10:0.1.1Second, pull the image on both nodes. The README warns that start refuses to launch if either node is missing it:
docker pull ghcr.io/anemll/dspark-vllm-gx10:0.1.1Third, prepare the weights on the head. The --official flag selects the official checkpoint; --abliterated or --yes reads ABLITERATED from .env.dspark. The README says abliteration is gated behind HF_TOKEN and that prepare downloads an 18 KiB direction file rather than the 157 GiB checkpoint once you have agreed on the Keys Hub repo:
./prepare-dspark-model-cache.sh --officialFourth, the README offers an optional CPU-only gate. It states this does not measure tokens per second:
bash scripts/ci-validate.shFifth, start the worker first and then the head:
./start-deepseek-v4-flash-dspark.shFinally, check that the server is up. The README says to expect "id": "deepseek-v4-flash-vision-exp" and "max_model_len": 1048576:
curl -fsS http://127.0.0.1:8888/v1/models
./smoke-deepseek-v4-flash-dspark.sh
./status-deepseek-v4-flash-dspark.shThe API is served at http://HEAD_NODE_IP:8888/v1, with VLLM_HOST=0.0.0.0 by default; for head-only tests the README suggests VLLM_HOST=127.0.0.1. A one-shot bind override exists in the form ./start-deepseek-v4-flash-dspark.sh --host 0.0.0.0 --port 9000. Day-to-day operations use ./status-…, ./logs-… and ./stop-….
The restart exit code 3 trap and other sharp edges
The most concrete failure mode in the README is not a crash. After a reboot, dockerd may already have restored the ranks because the containers use restart: unless-stopped. In that case start exits 3, meaning already running, rather than 1. The README calls this expected and warns against running ./stop unless you want a cold start. For systemd users it suggests SuccessExitStatus=3. If you wire this into a supervisor without reading that note, you will treat a healthy cluster as a failed unit and restart it unnecessarily.
Two configuration rules are stated as absolutes. start-*.sh exports GPU_MEMORY_UTILIZATION from GPU_MEMORY_UTILIZATION_TEXT, and the README says not to set GPU_MEMORY_UTILIZATION by hand. It also says not to set DSPARK_MODEL by hand. Flipping a meaningful switch in .env.dspark requires restarting both ranks, not just the head.
The default profile is tuned for concurrency, not for one huge prompt. The README offers an alternative for long coding or big prompts that keeps the 1M ceiling but changes the balance: MAX_NUM_SEQS=4, MAX_NUM_BATCHED_TOKENS=16384 and GPU_MEMORY_UTILIZATION_TEXT=0.87. It also notes that the Vision-Exp ViT consumes more weight RAM than the 0731 checkpoint, which is why the KV pool lands at 17.04 GiB on this cluster at utilization 0.835.
Two smaller but real constraints. The README says to disable earlyoom on both hosts, because it can kill vLLM under deep-context load. And it warns against unsetting VLLM_USE_BREAKABLE_CUDAGRAPH, which defaults to 0; the README states that leaving it unset is slower. Vague as that is, it is the only performance guidance the README gives, and the repository points to results/RESULTS-2026-08-14.md for dated tables and method rather than quoting numbers in the README itself.
No video encoder, and what the removed sidecar means
The limitation most likely to surprise a multimodal team is stated in the opening paragraph: there is no video encoder in the official weights, and a GIF is treated as a still frame. If your pipeline expects frame-level video understanding, this checkpoint will silently degrade to a single image rather than fail loudly. That is a design boundary of the weights, not a bug in the launcher.
The README also says the old Qwen3-VL sidecar and MCP path has been removed. Anyone who bookmarked an earlier version of this recipe, or copied its docker-compose files, should expect the current tree to differ. The repository still contains docker-compose.dspark.yml, docker-compose.dspark-nfs.override.yml and docker-compose.stage-c.override.yml, but the README does not document a rollback path to the sidecar arrangement. If you depended on that path, the README is silent on how to restore it.
Finally, the recipe is the wrong tool below two nodes. The README frames the whole quick start around a head and a worker, with WORKER_HOST and MASTER_ADDR as required fields. A single DGX Spark is outside the supported shape here; the three-node launcher is the only other topology the README acknowledges.
Alternatives: what you would trade away
The closest comparison is running the same checkpoint under plain vLLM without this repository's launcher. You would still choose a container image, set tensor parallel size, and point vLLM at deepseek-ai/DeepSeek-V4-Flash-Vision-Exp. What you give up is the assembled set: the .env.dspark contract, prepare-dspark-model-cache.sh for the checkpoint and the gated abliteration direction, start/stop/status/logs scripts, the smoke test, the CPU-only validation script, the NFS weight-sharing override, and the separate TP=3 launcher. You also lose the specific defaults the README documents, including MTP_NUM_TOKENS=6 with its divisibility rule against n_predict=3 and the nvfp4_ds_mla KV configuration. In exchange you avoid a recipe that assumes two hosts, a specific image tag, and a working RoCE fabric.
A second alternative is the 0731 non-Vision checkpoint. The README repeatedly distinguishes the two, noting that the Vision-Exp ViT takes more weight RAM than 0731, which is why the KV pool is 17.04 GiB at utilization 0.835. If you do not need image input, the 0731 lane is the lighter one in this repository's own framing. The README points to docs/DEEPSEEK_V4_FLASH_0731.md for checkpoint and encoder details.
Licence, maintenance and the cost of staying current
The repository is MIT licensed, which covers the scripts and configuration in this tree. It does not relicense the model weights, the Anemll container image, or vLLM; those carry their own terms, and the README's abliteration path adds a gated Hugging Face repository behind HF_TOKEN. Treat the MIT file as covering the glue only.
On maintenance, the last push was on 2026-09-16 and the repository is not archived, so the tree reflects very recent work. There are no retrieved releases, so upgrades are tracked through commits and the CHANGELOG.md file rather than versioned artifacts. That matters for cost: the recipe pins ghcr.io/anemll/dspark-vllm-gx10:0.1.1 as the default image and pins the checkpoint to revision 86f746b36186f0e567729a5c06a8c918caba82a9. Moving either pin means re-running prepare-dspark-model-cache.sh and restarting both ranks, because the README says start syncs .env.dspark to the worker.
The upgrade surface is wider than a single container. The tree includes patches/, vllm_patch_gb10/, build-dspark-vllm-runtime.sh, Dockerfile.gb10-dsv4-dspark and dspark-numeric-knobs.sh, which suggests the runtime is built or patched rather than consumed as-is. The README does not document a supported upgrade sequence from 0.1.1 to a later image, so budget for a cold start on both nodes whenever you move the tag.
Editorial conclusion
Adopt this if you already have two DGX Sparks on a working RoCE/NCCL fabric and want the Vision-Exp checkpoint served with vLLM TP=2, DSpark speculation and a 1M-token ceiling without writing the launcher yourself. Do not adopt it for a single Spark, for video input (the official weights have no video encoder), or if you cannot pin both nodes to the same container image. Verify three things before trusting a deployment: that ghcr.io/anemll/dspark-vllm-gx10:0.1.1 is present on both hosts, that ./start-deepseek-v4-flash-dspark.sh exits 0 rather than 3 on the first run, and that curl http://127.0.0.1:8888/v1/models returns max_model_len 1048576.
Frequently asked questions
Can DeepSeek V4 Flash run on DGX Spark?
This repository targets exactly that, but with two DGX Sparks rather than one. The quick start requires a head node and a worker, with WORKER_HOST and MASTER_ADDR set, and it runs vLLM with TP=2 and the DeepSeek-V4-Flash-Vision-Exp checkpoint.
Which DGX Spark is best for this recipe?
The README does not compare DGX Spark models. It specifies the requirements instead: two DGX Sparks, a working RoCE/NCCL fabric, and the same container image on both nodes, with the default image being ghcr.io/anemll/dspark-vllm-gx10:0.1.1.
What is a DGX Spark used for in this project?
Here the Sparks are the two inference nodes serving deepseek-v4-flash-vision-exp through vLLM. The head node runs the launcher and the API on port 8888, and the worker joins over NCCL, with weights either copied to both nodes or mounted from the head over NFSv4 when DSPARK_WORKER_HF_NFS=1.
Community notes