exo

A Python project that joins several devices into one AI cluster so models larger than a single machine can run locally, with automatic discovery and a Thunderbolt networking option.

The GitHub description is two words: run frontier AI locally. exo does that by connecting all of your devices into an AI cluster, which lets you run models larger than would fit on a single device. The README adds a detail that shapes the whole project: day 0 support for RDMA over Thunderbolt, which means models run faster as you add devices. The project is Python under the Apache-2.0 license and has grown past 46,000 stars.

The feature list leans on hardware you might already own. Automatic device discovery needs no manual configuration. RDMA over Thunderbolt is described with a claimed 99 percent latency reduction. Topology aware auto parallel splits models based on a real time view of device topology, and tensor parallelism comes with claimed speedups on 2 and 4 devices. MLX is used as the inference backend, which points the project squarely at Apple silicon.

The benchmarks section cites runs on 4 M3 Ultra Mac Studio units with tensor parallel RDMA, covering models such as Qwen3 in 8 bit, DeepSeek v3.1 in 8 bit, and Kimi K2 Thinking in native 4 bit. Worth noting is where these numbers come from: the source is attributed to a third party video about a large VRAM Mac setup, not to the maintainers' own lab. That attribution matters when reading the speedup claims.

The quick start describes a mostly hands off setup. Devices running exo automatically discover each other, and each device provides an API and a dashboard at a local address. Running from source on macOS requires Xcode for the Metal ToolChain and brew for package management, with a note about accepting a binary cache to avoid building the Xcode Metal ToolChain. The tradeoff is laid out up front, which is more than many projects bother to do.

• Source 1 · Project repository
• Source 2 · README