Swiftlet: running 35B and 80B Qwen MoE models on Apple hardware by streaming expert weights
Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices by streaming expert weights from storage, enabling 35B and 80B models to run with low RAM, including on iPhone.
At a glance
- What is it?
- Swiftlet is a Swift and Metal runtime that keeps only a model's dense core in RAM and reads routed Mixture-of-Experts weights from disk on demand. It makes 35B and 80B Qwen models fit on Macs and iPhones, at speeds that depend heavily on which Mac you own.
- Who is it for?
- Swiftlet is for Apple Silicon owners who want to run a 35B or 80B Qwen MoE model locally without the RAM those weights would normally need, and who accept the decode speeds the README reports for their specific chip. It is the wrong tool if you need fast generation on a base M1, if you have no free SSD for an 18 to 42 GB container, or if you need factual recall rather than fluent writing, since only about 3B parameters are active per token.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 4 days ago.
- What is it written in?
- Mainly Swift, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 16, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The RAM wall that Swiftlet is built to get around
A Mixture-of-Experts model stores far more weights than it uses per token. Qwen3-Next-80B-A3B has 78.8 GiB of weights in its 8-bit form, which the README says cannot be resident on a 64 GB machine at all. A conventional runtime loads the whole model before it answers anything, so the model simply does not run. Swiftlet's approach is to keep only the small dense core resident and read the routed expert weights from storage when a token needs them. The README reports that the same 8-bit 80B container is served in 6.9 GiB at about 4.8 tok/s, and that a 4-bit 397B container of 207.6 GiB runs in 12.6 GiB at about 1.4 tok/s. The audience is Apple Silicon owners: macOS 14+ or iOS 17+, with the 35B running on an iPhone 17 in about 2.5 GB of RAM at about 1 tok/s. The project is Apache-2.0 and the last push was on 2026-09-14.
Dense core resident, experts streamed: the mechanism
The README describes the split directly: Swiftlet keeps the small dense core of a model in memory and streams the routed Mixture-of-Experts weights from storage on demand. Decode speed then tracks active parameters rather than container size, which is why the README notes that both the 35B and the 80B activate about 3B parameters per token, while the 397B activates about 17B. That is the design bet: container size sets your disk bill, active parameters set your speed.
The README also describes where the bottleneck sits on different hardware. On an M1 with an 8-core GPU the decode loop is close to compute bound, so the expert cache is mostly a memory knob: on a short prompt --cache-gb 2 matches --cache-gb 8 while saving several GB of RAM. On a 503-token prompt a larger cache lifts decode about 8 percent from 2 to 6 GB by trimming the CPU gap between command buffers. On an M5 the README says the decode loop is dispatch bound, not IO bound, and that there is headroom in kernel speed. Those are two different bottlenecks, and they explain why the same cache setting is not the right answer on every machine.
Building Swiftlet on a Mac and running a first prompt
The README's quick start clones the repository and builds with Swift Package Manager in release configuration. The binary lands in .build/release.
git clone https://github.com/leonickson1/Swiftlet.git && cd Swiftlet
swift build -c releaseNext, fetch a container with swiftlet-repack. The README recommends the R2 mirror over Hugging Face because anonymous Hugging Face downloads are rate-limited and get slower the longer they run. This command pulls the 4-bit 35B, which is 18 GB on disk and about 2.6 GB of peak RAM on an M5.
.build/release/swiftlet-repack \
--from-url https://pub-c0cfece2dbc340dbb2cd9d94310a7d68.r2.dev/qwen3.6-35b-qpack \
--output ~/models/qwen3.6-35b.qpackThe README lists the equivalent Hugging Face sources as --from-hf Leonickson/Qwen3.6-35B-A3B-qpack, Leonickson/Qwen3.6-35B-A3B-8bit-qpack and Leonickson/Qwen3-Next-80B-A3B-qpack. It also states that the same command repacks raw MLX checkpoints via --from-hf mlx-community/... or --source /path/to/checkpoint. Once a download finishes, scripts/verify_container.py checks it against the source hashes.
Then chat. The README says the chat command applies the model chat template, disables the reasoning block, and keeps conversation state so follow-ups prefill only the new turn.
.build/release/swiftlet chat ~/models/qwen3.6-35b.qpack \
"What is the capital of Spain?" "And what about France?"For a single prompt with stats, the README gives swiftlet generate with --gpu and --chat. There is also an OpenAI-compatible server, which the README says binds to loopback only.
.build/release/swiftlet-server --model ~/models/qwen3.6-35b.qpack --port 8080The honest expectation: 3B active parameters and slow base hardware
The README sets one expectation plainly: only about 3B parameters are active per token, so these models chat and write like large models but recall facts like small ones. If your workload is retrieval of specific facts rather than fluent generation, the streaming design does not change that, and a smaller fully resident model may serve you better.
The speed figures are the other limit. On a base M1 with an 8-core GPU and 16 GB, the README reports about 2.45 tok/s for the 4-bit 35B and about 1.74 tok/s for the 8-bit, with prefill running at roughly decode speed, so a long system prompt is expensive on that class of machine. On an M4 Max with a 40-core GPU the 4-bit 35B reaches about 19.5 tok/s. That is a wide spread, and it means the experience is decided by your chip, not by the runtime alone. The README's own guidance on a 16 GB machine is that --cache-gb 2 is usually the right trade.
Storage is a hard prerequisite rather than a tuning knob. The containers are 18 GB for the 4-bit 35B, 34 GB for the 8-bit 35B and 42 GB for the 80B. There is no smaller tier listed, and the README does not document a way to run these models without the container on disk.
How Swiftlet differs from llama.cpp and MLX
llama.cpp and MLX both quantize and run models locally on Apple hardware, but the usual pattern is to load the weights you intend to use into unified memory and rely on quantization to make them fit. Swiftlet's difference is that it treats storage as the home of the expert weights and memory as a cache for the routed ones. That is why an 8-bit 80B container of 78.8 GiB can be served in 6.9 GiB, and why a 4-bit 397B container of 207.6 GiB runs at all in 12.6 GiB.
The cost of that approach is visible in the same numbers: the 397B decodes at about 1.4 tok/s, and the 80B at about 4.8 tok/s on the hardware the README cites. A runtime that keeps everything resident would be faster per token if the model fit, which is the case Swiftlet is not designed for. Swiftlet also repacks raw MLX checkpoints, so it is not a replacement for the MLX ecosystem so much as a different execution strategy for the MoE family it targets: Qwen3-Next and Qwen3.5/3.6.
Maintenance, licensing and what the repository does not say
The repository is not archived and the last push was on 2026-09-14, three days before this writing, so the codebase is moving. The README states that the current focus is kernel speed because the decode loop is dispatch bound rather than IO bound, which implies further tuning is expected. There are no releases listed, so building from main is the path the README describes.
Swiftlet is Apache-2.0, and the repository carries a THIRD_PARTY_NOTICES.md file, which is where dependency attributions live. Apache-2.0 permits commercial use and modification with the usual notice and patent terms; that is a statement about the licence text, not legal advice, and the model containers themselves are separate artifacts with their own terms on Hugging Face.
Several things the README does not document: there is no rollback procedure for a container that fails verification, no guidance on migrating between container formats, and no changelog. The iPhone path currently depends on an app version the README says is still in App Store review, so the Experimental Models feature may not appear for a couple of days. The README does not state a timeline for that review.
Editorial conclusion
Swiftlet is for Apple Silicon owners who want to run a 35B or 80B Qwen MoE model locally without the RAM those weights would normally need, and who accept the decode speeds the README reports for their specific chip. It is the wrong tool if you need fast generation on a base M1, if you have no free SSD for an 18 to 42 GB container, or if you need factual recall rather than fluent writing, since only about 3B parameters are active per token. Before committing, check the disk requirement for the container you want, run swiftlet-repack to fetch it, and measure decode speed on your own machine with swiftlet generate --gpu --chat.
Frequently asked questions
What is Swiftlet?
Swiftlet is a Swift and Metal runtime for the Qwen3-Next and Qwen3.5/3.6 MoE hybrid model family. It keeps only the dense core of a model in memory and streams the routed expert weights from storage on demand, which lets 35B and 80B models run on Apple devices with low RAM.
How do I install Swiftlet on a Mac?
The README's quick start clones the repository and runs swift build -c release, then uses .build/release/swiftlet-repack to download a container such as the 4-bit 35B from the R2 mirror or from Hugging Face with --from-hf. Requirements are Apple Silicon, macOS 14+ or iOS 17+, and free SSD space for the container.
What is Swiftlet AI?
It is the same project: a local inference runtime written in Swift with Metal kernels, distributed under Apache-2.0. The README describes it as an open runtime anyone can build on, with the 35B model also shipping inside the Priv AI app for iPhone.
Community notes