SwarmLLM: Running a 27B Model Across Browser Tabs with WebGPU and WebRTC
Every device brings a slice. Together they run the whole model. Peer-to-peer LLM inference across browser tabs: a from-scratch WebGPU engine and a WebRTC runtime that split a 27B model over the devices in a room.
At a glance
- What is it?
- SwarmLLM splits one large language model across the devices in a room, each browser tab holding a slice of layers and passing 10 KB activations over WebRTC. It is an MIT-licensed research project with a real 27B demo and a thin compatibility list.
- Who is it for?
- Adopt SwarmLLM if you have a room of WebGPU-capable browsers and want to experiment with layer-sliced inference without installing anything on the peers. Do not adopt it if you need Firefox, Linux Chromium or Safari on a Mac to host, or if you need private prompts: the README states a room is a shared conversation and mid-model activations are not private against a determined peer.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 9 days ago.
- What is it written in?
- Mainly JavaScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 16, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What SwarmLLM actually solves
A single consumer device often cannot hold a 27B model in memory. SwarmLLM's answer is to stop trying. It splits the model by layers across the devices in a room, so a laptop, a phone and a desktop each hold a slice they can afford, and the full model exists only as the union of those slices. The README's demo records Qwen 3.8 27B, 15 GB of Q4_0 weights, split across a MacBook and an iPhone in browser tabs on the same Wi-Fi, producing 400 tokens at 10.7 tok/s.
The target user is not a production team. It is someone with several WebGPU-capable browsers in one place who wants to run a model larger than any one of them can hold, without provisioning servers or installing a runtime on each peer. The README is explicit that nothing is installed on the joining devices: you open a URL, create a room, share the code, and each device downloads only its layers, cached for next time.
How the layer split and the WebRTC activation path work
The architecture diagram in the README shows the flow plainly. The host embeds the last token into a hidden state of 5,120 floats, roughly 10 KB, and sends it over WebRTC. Peer A runs layers 0 through 21, peer B runs 22 through 42, peer C runs 43 through 63, each on its own GPU. The result returns to the host, which applies the final norm, the LM head, sampling, and a draft head that guesses the next token.
Two design decisions stand out. First, the engine is memory-bound by nature, since every generated token reads all the weights once. The README describes the response: 4-bit blocks with f16 scales, 64 threads sweeping each row together, dequantization in registers, reduction in shared memory, and one command submit per token. Second, the runtime tries to make each network lap carry more work. Prefill sends 16 tokens per round trip, and decode chains speculative drafts so a slow link still moves several tokens per lap. The speculative path is verified in a single batched pass with exact rollback of the recurrent state, and the README claims the speculative stream is bit-identical to plain decoding.
Installing SwarmLLM and running a first room
The fastest path needs no clone at all. The README says to open swarmllm.ai/room, create a room, share the code, pick a model, and start. Every device downloads only its layers.
To run the same thing locally, the README gives a static-server recipe. Any static server works; the example uses serve on port 8080.
git clone https://github.com/Nehanth/swarmllm && cd swarmllm
npx -y serve -l 8080 . # any static server works; then open http://localhost:8080/roomAfter that command, open http://localhost:8080/room in a WebGPU-capable browser and create a room. The joining devices need only the URL and the room code.
Working on the engine itself is a different setup. It needs Deno 2.x and a WebGPU-capable GPU, and model files go under models/, with details in docs/models.md. The package.json scripts are the entry points: npm test runs unit tests without a GPU, npm run test:gpu runs golden tests on Qwen3 0.6B, and npm run test:q38 runs the 27B suites including the speculative-versus-plain equality check. npm run bench:q38 measures decode and prefill throughput.
Where SwarmLLM breaks: Safari hosting, Firefox, and prefill
The compatibility list is short and the README is honest about it. Chrome on macOS is the tested host. Safari on an iPhone can join a room and hold a small slice. Safari on a Mac reloads the tab under memory pressure when it holds the 27B's large slice, so the README says not to host from it. Firefox and Linux Chromium need WebGPU enabled and are untested by the project. That rules out a large share of the browsers a casual room would contain.
Prefill is the other weak point, and the README names the cause: the DeltaNet recurrence is serial and the prefill GEMM is young. The numbers show the gap. On the GB10, decode is 9.0 tok/s plain and 16.1 tok/s speculative, ahead of a native llama.cpp build measuring 8.0 decode on the same file and GPU. Prefill is 44 tok/s against 377 for the native build. A long prompt is where this hurts: a MacBook plus iPhone room needed 8.5 s for a 169-token prompt.
Privacy is a boundary, not a bug. The README states a room is a shared conversation, so everyone in it sees the questions and answers by design. No server sees them, but the devices running layers work on mid-model activations, which the README says are not private against a determined peer either, pointing to SECURITY.md.
SwarmLLM compared with exo, llama.cpp rpc-server and Petals
The closest comparison is exo, which also splits a model by layers across machines. The difference is what each node must run. exo needs a Python package per node and machines that run MLX or tinygrad. SwarmLLM needs a browser with WebGPU and nothing else, which is why phones can join a room. The trade is reach: exo targets machines you control, while SwarmLLM targets whatever browser is in the room.
llama.cpp's rpc-server also slices by layers, but it requires a binary and an open port per node, and the README notes the docs say it is not for untrusted networks; in practice it is a LAN tool. SwarmLLM's rooms span networks via WebRTC, which removes the port-opening problem but adds a signaling dependency on PeerJS. Petals splits across server GPUs in a public swarm, so it assumes a different resource entirely. distributed-llama wants Linux boxes and Raspberry Pis.
Against WebLLM, MLC and transformers.js the split is categorical: those run the whole model in one browser tab, so they are limited by that tab's memory. Ollama and llmman run the whole model on one machine and route between machines rather than splitting. SwarmLLM's engine is its own WGSL, not WebLLM, MLC or llama.cpp; weights and tokenizer come from Qwen, hosting from Hugging Face.
Maintenance, licence and what a fork inherits
SwarmLLM is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and licence text are retained. That is the whole of the licence implication here; the repository ships a LICENSE file and the package.json declares "license": "MIT". Nothing in the repository suggests a dual-licence or a contributor agreement that would complicate a fork.
The repository is not archived, and the last push was on 2026-09-09, eight days before this writing, with v0.2.0 released on 2026-09-07. Despite the recent activity, the project describes itself as young in places: the prefill GEMM is "young" in the README's own words, and Firefox and Linux Chromium are untested. The repository carries the usual governance files (GOVERNANCE.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md, CITATION.cff, RELEASE.md), so the process for changes exists on paper.
Upgrade cost is mostly model files and tests. The engine is JavaScript with WGSL kernels, so there is no compiled artifact to rebuild. The npm scripts pin the verification path: npm test for unit tests, npm run test:gpu for Qwen3 0.6B golden tests, npm run test:q38 for the 27B suites. A fork that changes kernels is expected to keep the golden tests passing, since the README says every optimization is gated on them.
A realistic first evaluation
Before trusting a room with anything that matters, check two things the project itself makes checkable. First, confirm your model file is on the supported list in docs/models.md: Qwen 3.8 27B in GGUF Q4_0, Qwen3 0.6B, 1.7B and 4B in GGUF Q8_0 or Q4_0, and SmolLM2 135M in safetensors f32. A model outside that list has no stated path.
Second, reproduce the equality claim on your own hardware. The README says the speculative path produces the same stream as plain decoding, and npm run test:q38 is the suite that checks it. If speculative decoding is the feature you care about, that test is the one that decides whether the claim holds on your GPU.
For a quick sanity check of the room itself, the cross-internet row in the performance table is the relevant expectation: 3.5 to 6 tok/s for a host plus one peer. If that is too slow for your use, the same-Wi-Fi configuration is the one the demo used.
Editorial conclusion
Adopt SwarmLLM if you have a room of WebGPU-capable browsers and want to experiment with layer-sliced inference without installing anything on the peers. Do not adopt it if you need Firefox, Linux Chromium or Safari on a Mac to host, or if you need private prompts: the README states a room is a shared conversation and mid-model activations are not private against a determined peer. Before committing, run the repository's own golden tests on your target hardware and read docs/models.md to confirm your model file is supported.
Frequently asked questions
Does SwarmLLM require any installation on the devices that join a room?
No. The README says to open swarmllm.ai/room, create a room, share the code and pick a model; each device downloads only its layers, cached for next time. Running it locally only needs a static server serving the repository.
Which browsers can host a SwarmLLM room?
Chrome on macOS is the tested host. Safari on an iPhone can join and hold a small slice, but Safari on a Mac reloads the tab under memory pressure when it holds the 27B's large slice, and Firefox and Linux Chromium are untested.
Is SwarmLLM private if no server sees the prompts?
The README states a room is a shared conversation, so everyone in it sees the questions and answers by design. It also says the devices running layers work on mid-model activations, which are not private against a determined peer, and points to SECURITY.md.
Community notes