sparkrun: a scheduler-free launcher for LLM inference on DGX Spark nodes

The gap sparkrun fills between a bare GPU box and a cluster scheduler

A DGX Spark machine is a single node. Serving a model larger than one box on it means deciding how to split the model, how the nodes find each other over RDMA, how the container image and weights reach every host, and how to stop the whole thing cleanly. The README frames the project as an alternative to that work: launch, manage, and stop LLM inference workloads on one or more NVIDIA DGX Spark systems, with no Slurm and no Kubernetes. The audience is therefore narrow and identifiable. It is someone who has DGX Spark hardware, wants vLLM, SGLang or llama.cpp serving, and does not want to operate a scheduler to get it. The repository topics list dgx-spark, inference, llama-cpp, sglang and vllm, which matches that scope exactly.

Recipes, runtimes and how --tp maps to physical hosts

The core abstraction is the recipe. A recipe names a model and a runtime, and the CLI accepts it as a single argument, as in sparkrun run qwen3-1.7b-vllm. Multi-node tensor parallelism is expressed through --tp, and the README states that TP maps to node count on DGX Spark. So sparkrun run qwen3-1.7b-vllm --tp 2 means two hosts, with InfiniBand and RDMA detection handled automatically rather than configured by hand. Recipes come from git-based registries. The README names three: official recipes maintained by the Spark Arena team, community recipes, and benchmarked recipes published through Spark Arena, plus the ability to add your own registries. That registry model is the part worth paying attention to, because it means the set of runnable configurations is not fixed in the package. It is fetched, and sparkrun update refreshes it. The README also mentions watched launches reporting Docker-start TTR and TTFT through a rank-local streaming readiness check for Docker vLLM and SGLang launches, with per-recipe overrides documented in docs/STARTUP_READINESS.md. I have not run this, so I cannot say how stable those timings are in practice. What the material does establish is that readiness detection is per-rank and streaming rather than a fixed sleep, which is the more useful design for multi-node launches.

Getting it running: uvx, the setup wizard and what the wizard touches

Installation is a single command: uvx sparkrun setup. The README describes this as installing sparkrun and then launching a guided setup wizard that creates a cluster, configures an SSH mesh, detects ConnectX-7 NICs, sets up sudoers and enables earlyoom. That list is the honest description of the project's footprint. It writes SSH configuration, it needs sudoers entries, and it changes early OOM behaviour on the target machines. Anyone evaluating sparkrun for a shared or locked-down environment should read those four items as the actual integration surface, not as conveniences. Day-to-day commands are short: sparkrun run, sparkrun logs, sparkrun stop, sparkrun status. The README makes one behavioural promise that matters operationally: Ctrl+C detaches from logs and never kills the inference job. Detaching rather than terminating is the right default for a serving workload, and it is the kind of detail that is usually wrong in homegrown launch scripts. VRAM estimation is available before launch through sparkrun show <recipe>, which is the command to run when you are unsure whether a model fits on the node you have.

Update channels and the downgrade risk in switching back to stable

sparkrun update upgrades the tool when installed via uv tool and refreshes recipe registries. Four channels exist. --stable installs from PyPI, --beta from the develop branch, --alpha from develop-next, and --yolo is documented as an alias for --alpha. Running update with no flag stays on the current channel; passing a flag switches and the choice is remembered for later updates. The version strings differ in a way that tells you which channel you are on: stable prints a plain version such as 0.2.40, while beta and alpha append a channel suffix and commit, as in 0.3.0-alpha+g1a2b3c4. The README states plainly that switching from a preview channel back to --stable may downgrade. That is the one upgrade caveat worth carrying into a maintenance plan. A cluster that has been running an alpha build with a recipe that depends on newer behaviour can lose that behaviour on a switch back, and the recipes themselves are refreshed by the same command, so the tool version and the recipe set move together unless you manage registries separately. The same channel flags work with sparkrun setup install and sparkrun setup update, so the channel choice is not confined to one subcommand.

Telemetry is on by default and the config file is where it lives

sparkrun sends anonymous usage telemetry to telemetry.sparkrun.dev by default. The README is unusually specific about the payload, which makes it possible to evaluate rather than guess. Events include a random installation id stored in ~/.config/sparkrun/config.yaml, the sparkrun version, OS and version, system architecture, and command-specific metadata: run runtime, model, parallelism, source and hardware counts; benchmark category, framework, profile and result keys; update version and registry counts; and setup-wizard step choices. The README states that telemetry excludes personally identifiable information, hostnames, usernames, local file paths, tokens, secrets, logs, private Hugging Face or local models, and full command arguments. Two controls exist. sparkrun setup telemetry --disable turns it off persistently, sparkrun setup telemetry --enable turns it back on, and SPARKRUN_NO_TELEMETRY=1 opts out for a single process. A single-process environment variable is the right shape for CI or a one-off benchmark run where you do not want to change machine state. The installation id living in a YAML config file also means the opt-out is inspectable, which is more than many tools offer.

Where sparkrun stops being the right answer

The README's own framing is the limitation. There is no scheduler here. If you need queueing, fair sharing across teams, gang scheduling with all-or-nothing placement, or preemption, sparkrun does not describe any of that, and the absence is structural rather than a missing feature. Distribution is over SSH, and setup creates an SSH mesh plus sudoers entries, which assumes you control the nodes and can grant that access. The parallelism model is also tied to a specific assumption: --tp maps to node count, which is a deliberate simplification for DGX Spark and would be wrong on a heterogeneous cluster where you want more GPUs per host than hosts. Runtime support is stated as vLLM, SGLang and llama.cpp, so anything else is outside the tool's scope. Finally, the startup readiness timing described in the README applies to Docker vLLM and SGLang launches specifically, which suggests the readiness path is not uniform across all three runtimes. I cannot confirm from the supplied material how llama.cpp launches report readiness, and the README does not say.

Compared with writing your own launch script plus docker run

The realistic alternative for a two-node DGX Spark setup is a shell script around docker run with explicit NCCL and RDMA environment variables, plus manual scp or rsync of weights and images. That approach has no dependency beyond Docker and SSH, and it is completely transparent: you can read every line. What it does not give you is recipe registries, VRAM estimation before launch, readiness detection, or a stop command that reliably tears down a multi-node job. It also drifts, because the InfiniBand and RDMA detection that sparkrun performs automatically becomes a set of hardcoded interface names in your script that break when hardware changes. On the other side, the script approach has no telemetry, no sudoers modification, and no update channel that can move your recipes underneath you. The trade is between a small amount of ongoing configuration maintenance and a tool that takes over SSH, sudoers and earlyoom on your nodes. For a single node running one model, the script is probably enough. The value of sparkrun grows with node count, because that is where the SSH mesh and the automatic RDMA detection stop being conveniences and start being the reason the thing works at all.

Licence, maintenance and what to verify before rolling it out

sparkrun is Apache-2.0. That is a permissive licence with an explicit patent grant, and it places no copyleft obligation on the recipes or configurations you write around it. It does not, by itself, settle the licensing of the model weights or container images a recipe points at, and recipe registries are separate git repositories with their own terms. The release cadence visible in the material is fast: v0.3.6, v0.3.7 and v0.3.8 landed between late August and early September 2026, with the last push to main on 2026-09-10. Frequent releases on a young tool mean the update command is not optional maintenance, it is part of normal operation, and the alpha channel exists for people who want to track develop-next. The practical cost is that a pinned deployment needs a deliberate decision about which channel it follows, since the channel is remembered across updates. Before adopting, verify three things on a machine you can rebuild: what the setup wizard actually writes to SSH config and sudoers, what sparkrun show reports for the recipe you intend to serve, and whether telemetry is acceptable under your policy or should be disabled with sparkrun setup telemetry --disable. The README points to sparkrun.dev, docs/STARTUP_READINESS.md and a Discord invite for support, so the documentation surface extends past the repository.

Editorial conclusion

sparkrun fits teams running one to a handful of DGX Spark boxes who want vLLM, SGLang or llama.cpp serving without standing up Slurm or Kubernetes, and who accept SSH-based distribution and anonymous telemetry by default. It is the wrong tool if you need gang scheduling, queueing across many tenants, or non-NVIDIA hardware. Before adopting, run sparkrun setup on a non-production node and read ~/.config/sparkrun/config.yaml to confirm what the install id and registry entries look like, then decide whether sparkrun setup telemetry --disable matches your policy.

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. spark-arena/sparkrun on GitHub