# sparkrun drives DGX Spark inference over SSH instead of a scheduler

> A Python tool for launching, managing and stopping Docker-based inference workloads across one or more NVIDIA DGX Spark systems, with no Slurm and no Kubernetes, recipes distributed through Git registries, update channels that can downgrade you, and a bundled routing plugin that is AGPL while the rest is Apache.

**spark-arena/sparkrun** — sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems.  Live support: https://discord.com/invite/GH5kRgv6ZD

- Repository: https://github.com/spark-arena/sparkrun
- Website: https://sparkrun.dev
- Stars: 540 · Forks: 60
- Language: Python
- License: Apache-2.0
- Published: 2026-09-10 · Updated: 2026-09-10 · Language: en
- Canonical page: https://hysenlabs.com/projects/spark-arena-sparkrun

## The setup wizard does the cluster work before you run anything

Installation is one command and it does considerably more than install a package:

```bash
uvx sparkrun setup
```

The description in the README lists what the guided wizard does next, and the list is the specification of the tool's scope. It creates a cluster, configures an SSH mesh between the machines, detects ConnectX-7 network cards, sets up sudoers, and enables earlyoom. That is five pieces of infrastructure work that a small cluster needs before anyone can serve a model, and doing them by hand across several machines is where a first attempt usually stalls. The SSH mesh in particular is what everything else builds on, since the README also says models and container images are synced to cluster nodes over SSH automatically. A separate vendored runtime directory and a set of Docker assets sit alongside that, and the repository keeps a versions file at the root that pins the versions the tool considers current, which is a sensible way to make a fleet reproducible. Earlyoom is the one to note on a machine with unified memory: a memory-hungry workload that starts to swap will take the whole system down rather than just its own process, so a watchdog that kills the offending job is close to mandatory rather than a nicety.

## Ctrl+C detaches from the logs and never touches your job

The quick start is four commands and one promise:

```bash
# Run an inference workload
sparkrun run qwen3-1.7b-vllm

# Multi-node tensor parallelism (TP maps to node count on DGX Spark)
sparkrun run qwen3-1.7b-vllm --tp 2

# Re-attach to logs, stop a workload, check status
sparkrun logs qwen3-1.7b-vllm
sparkrun stop qwen3-1.7b-vllm
sparkrun status
```

The promise is stated immediately below them: Ctrl+C detaches from the logs, it never kills the inference job, and the model keeps serving. That is a small design decision with a large effect on how you work, because with most container tooling the same keystroke that stops your output also stops the thing producing it. Here, attaching is a separate action from stopping, and stopping is a separate action from launching, so a detached workload is a normal state rather than an accident. The launch also watches readiness: watched launches report time to readiness and time to first token for Docker-based vLLM and SGLang launches, measured with a rank-local streaming check, with the configuration, timing boundaries and per-recipe overrides kept in a separate document.

## Tensor parallelism equals node count on this hardware

The comment on the tensor parallelism flag is the sentence to read carefully: TP maps to node count on DGX Spark. A DGX Spark is a single system with unified memory, so asking for a degree of two is asking for two machines, and asking for a degree of four is asking for four. The tool then handles the interconnect itself, with automatic InfiniBand and RDMA detection named in the highlights. That removes the most tedious part of multi-machine inference, which is making sure the fabric is actually there and configured, and it means the flag you would recognise from a single-node setup has a different meaning here. The runtimes supported are vLLM, SGLang and llama.cpp, described as working out of the box, and the packaging declares runtime plugin entry points, one per runtime, which is how a fourth would be added without touching the tool. Read together with the readiness reporting, the design is coherent: the tool owns the parts that are fiddly, which are SSH, the fabric, model distribution and knowing when the server is actually answering, and leaves the inference engine to the engine. There is also a VRAM estimation command, spelled as a show subcommand taking a recipe, whose purpose is to tell you whether your model fits before you launch it, which is the difference between a fast check and a ten-minute failure. There is a standalone shell script in the repository for running a recipe directly, which is what makes the tool usable from a job script without the Python entry point in the way.

## Update channels are remembered, and going back to stable can downgrade

The update system has four flags and one warning, and the warning is the interesting part:

```bash
sparkrun update --stable   # PyPI stable release (default)
sparkrun update --beta     # develop branch preview
sparkrun update --alpha    # develop-next branch (bleeding edge)
sparkrun update --yolo     # alias for --alpha
```

Run with no flag and you stay on whichever channel you are already on; pass a flag and it switches, and the choice is remembered for future updates. The same flags work with the setup subcommands for install and update. Stable prints a plain version number while the preview channels print the channel and the commit, in a form like a version with an alpha suffix and a short hash, which is what makes it obvious at a glance whether you are running a published build or a commit someone is still pushing. The warning is that switching from a preview channel back to stable may downgrade you. That is the honest consequence of a stable channel that is simply older than a preview branch, and it is worth knowing before you run the command on a machine where the running version is what your team is testing against. The same command also refreshes recipe registries, so an update changes two things at once: the tool and the set of recipes it will offer. The maintainers keep a change log file at the root, and the current release line moves quickly, with the three most recent tags published in September 2026 and the manifest version matching the newest of them.

## Recipes come from Git, and the community hub is a benchmark board

Distribution is the design choice that makes the tool feel like something other than a wrapper script. Models are not built into the binary: a recipe is fetched from a Git registry, and there are three named kinds, official recipes maintained by the team behind the community hub and tested for this hardware, community recipes contributed by others, and benchmarked recipes. All three live in GitHub repositories, so you can read what a recipe does before you run it, fork one, or add a registry of your own. The highlights also list VRAM estimation as a first-class feature, exposed as a show subcommand that takes a recipe name, which means the answer to will this fit is a query rather than an experiment. The community hub doubles as a leaderboard of sorts: it is described as a place to browse benchmark results and then run them directly, which closes a loop most tools leave open, since you can see that a configuration was measured on this hardware and reproduce that exact configuration rather than guessing at it. The project is maintained under an organisation also named for the arena, the sponsoring company is named in the README, and the package author is the same organisation, which is worth knowing when you are deciding how much of the roadmap depends on one team. The README is deliberately short and points at a documentation site for the CLI reference, the quick start and the recipes index, so the pages here are an overview rather than a manual.

## Apache-2.0 for the tool, AGPL-3.0 for the routing plugin, and the channel decides

The licensing is a two-part answer and the packaging metadata states it as a compound expression rather than a single licence. The project declares Apache-2.0 combined with AGPL-3.0-only, and lists three licence files accordingly: the main one, the plugin's own, and an exception file. The reason is a bundled integration called SparkRoute, described in a README inside the plugin directory as AGPL-3.0-only with an additional permission for combining and distributing it with sparkrun. Its notices and immutable source provenance are said to ship in the installed plugin package, and the routing executable acquired separately is AGPL-3.0-only. Then there is the part that will bite you in a policy review: the routing plugin defaults on for the alpha channel while LiteLLM defaults on for the stable and beta channels. So two installs of the same tool, on different channels, ship different default routing behaviour, and the documentation for the channels and for managed plugin updates is kept in a separate file. If you need to know which router is in your build, you have to check the channel rather than the version.

## Telemetry is on by default, and the README lists what it excludes

The telemetry section is unusually explicit, which makes it easy to assess. Basic anonymous usage data goes to a dedicated host by default, and the events enumerated are a random installation identifier stored in a configuration file under your home directory, the sparkrun version, the operating system and its version, the system architecture, and command-specific metadata: runtime, model, parallelism, source and hardware counts for runs; category, framework, profile and result keys for benchmarks; version and registry counts for updates; and which steps you chose in the setup wizard. Then comes the exclusion list, which is the part worth reading in an organisation with a data policy. The README states the data does not include personally identifiable information or anything that might reveal trade secrets, and specifically not hostnames, usernames, local file paths, tokens, secrets, logs, private models on Hugging Face or locally, or full command arguments. It can be disabled persistently with a setup subcommand, re-enabled the same way, or suppressed for a single process with an environment variable, which is the one to use in a job runner where you cannot rely on configuration.

## Textual draws the interface, and three dependencies are pinned for advisories

The dependency list is short and annotated, and the annotations are the interesting part. Nine runtime packages, of which three are pinned for reasons other than compatibility: a serialization library appears twice as a transitive dependency pinned for security reasons, an identifier library is pinned as a transitive dependency of the HTTP and model hub clients to the fix for one advisory, and a syntax highlighting library is pinned as a transitive dependency of the terminal framework to the fix for another. Pinning transitives is normally a smell, and doing it with a comment naming the advisory turns it into a record of what the project is carrying and why. The other dependencies are an application framework from the same organisation as the author, a validation package, the click command line library, a YAML parser, the model hub client and the terminal interface toolkit. That last one explains the interface without the README ever saying so: this is a terminal user interface built with a text widget library, which is also why a rendering library appears as a transitive dependency. Python 3.12 or newer is required, and the package is classified as alpha, with classifiers for Linux only and for Python 3.12 and 3.13. The development extra pins the test tooling exactly as well, so a contributor's environment is reproducible for the same reason the transitives are pinned.

## Conclusion

Adopt sparkrun if your inference runs on a small number of DGX Spark systems and you would rather not operate a cluster manager for them, because the whole value proposition is replacing Slurm and Kubernetes with SSH and a recipe. The guided setup, the tensor parallelism that maps to node count, and the registry model are the parts that would cost you the most to write. Do not adopt it for heterogeneous clusters or for anything that needs a real scheduler's fairness and preemption, since that is the trade being made. Four things to check first. Which channel you are on, because preview channels can downgrade you on the way back to stable. Whether the AGPL routing plugin is acceptable in your deployment, since it defaults on for the alpha channel and its licence is different from the rest. Whether telemetry is acceptable, since it is on by default with a documented exclusion list. And whether your Python is 3.12 or newer, which is a hard floor rather than a suggestion.

## FAQ

### what is sparkrun

A Python tool for launching, managing and stopping Docker-based LLM inference workloads on one or more NVIDIA DGX Spark systems, without Slurm or Kubernetes. It installs with a single setup command that creates a cluster, configures an SSH mesh, detects ConnectX-7 cards, sets up sudoers and enables earlyoom, then runs vLLM, SGLang or llama.cpp recipes fetched from Git registries.

### Does sparkrun run on anything other than DGX Spark?

The package metadata declares Linux as the only supported operating system, requires Python 3.12 or newer, and the tool is described throughout as targeting NVIDIA DGX Spark systems, including automatic InfiniBand and RDMA detection for multi-node tensor parallelism.

### How do sparkrun update channels work?

A stable channel installs the PyPI release, beta installs a preview from the develop branch, alpha installs from develop-next, and yolo is an alias for alpha. Running the update command with no flag stays on the current channel, a channel flag switches and is remembered, and switching from a preview channel back to stable may downgrade the installed version.

### What does sparkrun collect as telemetry?

A random installation id stored in your config directory, the sparkrun version, operating system and architecture, plus per-command metadata such as runtime, model, parallelism, benchmark category and registry counts. The README states it does not collect hostnames, usernames, local paths, tokens, secrets, logs, private models or full command arguments, and it can be disabled with a setup subcommand or per process with an environment variable.

### What licence is sparkrun under?

The packaging metadata declares Apache-2.0 combined with AGPL-3.0-only, with three licence files. The AGPL part covers a bundled routing plugin that carries an additional permission for combining and distributing it with sparkrun, while the routing executable acquired separately is AGPL-3.0-only, and the routing plugin defaults on for the alpha channel.

## Sources

- [License: Apache-2.0](https://github.com/spark-arena/sparkrun/blob/main/LICENSE)
- [Project website](https://sparkrun.dev)
- [README](https://github.com/spark-arena/sparkrun/blob/main/README.md)
- [Releases](https://github.com/spark-arena/sparkrun/releases)
- [spark-arena/sparkrun on GitHub](https://github.com/spark-arena/sparkrun)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/spark-arena-sparkrun