Parallax: distributed LLM serving across machines you already own

The problem Parallax targets: one model, many uneven machines

Most serving stacks assume a homogeneous cluster. You buy N identical GPUs, put them behind one scheduler, and the model fits or it does not. Parallax starts from the opposite premise. The README describes it as letting you "build your own AI cluster for model inference onto a set of distributed nodes despite their varying configuration and physical location." That phrase is doing the real work: varying configuration and physical location.

The intended user is someone with hardware that does not match. A workstation with one large GPU, a couple of machines with smaller cards, and a Mac, all sitting on different networks. Parallax's feature list names pipeline parallel model sharding, cross-platform support, and a Mac-specific path with paged KV cache management and continuous batching. Those are three separate answers to the same question: how do you use what you have instead of what a datacenter would have.

The topics list on the repository includes decentralized-inference alongside llm-serving and distributed-systems, which tells you the project sits closer to the volunteer-compute end of the spectrum than to the Kubernetes-operator end. That positioning matters for who should read further.

How the pieces fit: Lattica for transport, SGLang, vLLM or MLX LM for compute

The README's architecture section is short but specific. Peer-to-peer communication is handled by Lattica, a separate GradientHQ project. The GPU backend is SGLang or vLLM. The Mac backend is MLX LM. Nothing in the README suggests Parallax implements its own attention kernels or scheduler internals; it composes existing engines and adds a distribution layer above them.

That composition explains the model support table. DeepSeek, MiniMax, GLM, Kimi-K2, Qwen, gpt-oss and Step models are listed with HuggingFace collection links, which means Parallax is not shipping weights, it is pointing at weights that the underlying engines can already load. The practical consequence: your supported-model list is bounded by what SGLang, vLLM or MLX LM support, not by Parallax alone. When the README says Parallax "now supports OpenClaw integration," that is a client-side integration, not a new inference capability.

The pipeline parallel sharding is the part that has to be Parallax's own. Sharding a model across nodes with different memory sizes and different network distances is where the scheduling and routing logic described as "dynamic request scheduling and routing for high performance" would live. The README does not document the algorithm, the sharding policy, or how it decides which layers land on which node. That is a gap, and it is the gap you would want closed before trusting the framework with a model too large for any single machine you own.

Getting it running: the commands the README actually gives

The quick install block is four lines. Clone the repository, change into it, run ./install.sh, activate the virtual environment with source .venv/bin/activate, then start serving with parallax serve -m Qwen/Qwen3.5-0.8B.

Several things are worth noting about that sequence. The install script creates a .venv in the repository root, so the install is self-contained rather than system-wide. The serve command takes a HuggingFace-style model identifier via the -m flag. The example model is small, an 0.8B parameter Qwen variant, which makes sense as a smoke test: you can confirm the plumbing works before pointing it at a 120B model.

The README does not show flags for specifying which nodes participate, how they discover each other, or what port the server binds to. Those details are presumably in docs/user_guide/install.md and docs/user_guide/quick_start.md, both of which are linked but not reproduced. If you are evaluating Parallax for a real deployment, those two files are the ones to read before running anything, because a four-line quick start that omits node configuration cannot be the whole story for a distributed system.

The Mac path is the most distinctive claim, and the least documented

Paged KV cache management and continuous batching are standard techniques in GPU serving. The README lists them explicitly "for Mac," which is unusual. On Apple silicon, unified memory changes the constraints: you are not moving tensors across a PCIe bus, and the memory ceiling is the machine's total RAM rather than dedicated VRAM. Applying paged KV cache and continuous batching in that environment is a different engineering problem from doing it on an H100.

Parallax routes Mac inference through MLX LM rather than SGLang or vLLM. That means a Mac node in a Parallax cluster is not running the same engine as a GPU node. The framework has to reconcile two different runtimes inside one pipeline-parallel deployment, which is a harder integration than a cluster of identical backends. The README asserts this works; it does not describe how the boundary between an MLX LM stage and an SGLang stage is managed, what happens to KV cache state at that boundary, or whether the paged cache implementation is inherited from MLX LM or written by Parallax.

For anyone whose reason to look at Parallax is the Mac support, that missing detail is the first thing to chase in the docs directory.

Where Parallax is the wrong choice

If you have one machine and it fits your model, Parallax adds a peer-to-peer layer, a distribution layer, and a multi-runtime integration on top of an engine you could run directly. SGLang and vLLM both serve models on a single node without any of that. The overhead is not documented in the README, but the architecture makes the tradeoff visible: Lattica sits in the request path.

A second case: latency-sensitive single-request workloads. Pipeline parallelism splits a model into stages across nodes, and each token has to traverse every stage. When those stages are on machines in different physical locations, as the README explicitly allows, the per-token network cost compounds. The README claims "dynamic request scheduling and routing for high performance" but gives no numbers, no latency targets, and no guidance on what network conditions make the approach viable. Without that, treating Parallax as a drop-in replacement for a colocated multi-GPU server is a guess.

A third case: teams that need a documented operational story. Three releases exist, v0.1.0 through v0.1.2, with the latest dated 2025-12-02. The README is a landing page with links outward. There is no operations guide in the material supplied, no failure-mode documentation, and no capacity planning guidance. If your adoption process requires those documents before a pilot, Parallax is not there yet.

Compared with running vLLM or SGLang directly

The honest comparison is not Parallax versus some other distributed framework. It is Parallax versus the engines it wraps. vLLM and SGLang both appear in the README's partner logos and both serve as Parallax backends, which means the interesting question is what the extra layer buys.

Running vLLM directly on one node gives you a mature serving path, tensor parallelism within that node, and a well-known configuration surface. You give up the ability to span nodes that are not in the same machine. Running Parallax gives you pipeline parallel sharding across nodes, a P2P transport, and a uniform serve command across GPU and Mac hardware. You give up direct control of the engine's own tuning parameters, or at least you give up the documented version of that control, since the README does not show how Parallax flags map onto backend flags.

There is a second alternative worth naming: a Kubernetes-based serving deployment with vLLM workers and a router in front. That approach assumes a controlled network and identical or near-identical nodes, and it comes with a large operational surface. Parallax's bet is that you do not have that environment and do not want to build it. Whether the bet pays off depends on how well the sharding handles genuinely uneven hardware, which is exactly what the README does not quantify.

Licence, maintenance and what three releases tell you

Parallax is Apache-2.0. That licence permits commercial use, modification and redistribution, and it includes an explicit patent grant. It does not require you to open-source your own code that links against Parallax. If you are embedding Parallax in a product, the Apache-2.0 terms are permissive; the usual obligations around preserving notices and stating changes apply. This is a description of the licence text, not legal advice, and the dependencies matter too: SGLang, vLLM and MLX LM carry their own licences, and Lattica is a separate repository whose terms you would need to check independently.

On maintenance: the release history shows v0.1.0 in November 2025, v0.1.1 later the same month, and v0.1.2 in December 2025. The last push to the default branch is dated 2026-07-01, which is later than the most recent release, so development activity continues between tagged versions. The 0.1.x version numbers indicate the project is pre-1.0, and the README's own news section still references the 0.0.1 release from October 2025 alongside a February 2026 OpenClaw integration note. Anyone pinning Parallax in production should pin a specific tag rather than tracking main, given the pre-1.0 versioning and the visible gap between commits and releases.

Upgrade cost is hard to estimate from the README. Because Parallax delegates inference to SGLang, vLLM and MLX LM, a backend upgrade can change model support without Parallax changing at all. That coupling is the main maintenance risk: your Parallax version and your backend versions have to move together.

Editorial conclusion

Parallax is aimed at teams or individuals who already have several machines (a mix of GPU boxes and Macs) and want one served model across them rather than a single-node deployment. It is the wrong tool if you have one machine, if you need documented capacity numbers before committing, or if you cannot accept that the README is a pointer to other documents rather than a complete operations manual. Before adopting, verify three things from the repository itself: the install script's assumptions in docs/user_guide/install.md, the exact model identifiers accepted by the serve command in docs/user_guide/quick_start.md, and whether the backend for your hardware (SGLang, vLLM or MLX LM) is listed as supported for the model you intend to run.

Official sources

1. GradientHQ/parallax on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases