pplx-garden is four inference projects under one repository, not one library

Four projects with their own docs, papers and licences

pplx-garden is described as the Perplexity AI open source garden for inference technology, and the word garden is accurate. It is a container for independent pieces rather than a single library with one API. The README lists fabric-lib for the RDMA TransferEngine and the P2P MoE dispatch and combine kernel, pplx-unigram as a Unigram tokenizer encoder, and lily as a Rust and Metal inference server for Qwen3.6-35B-A3B on Apple Silicon. Each one carries its own documentation file, its own blog post, and in the case of lily its own licence.

That structure is also visible in the top-level tree. There are separate directories for fabric-lib, lily, p2p-all-to-all, pplx-unigram, python-ext, python and rust, alongside benchmarks, docker, docs, scripts and tests. The repository is a workspace in the ordinary sense: several codebases that happen to be released together.

The licence split is the detail to note first. The repository root carries an MIT licence, while lily is Apache-2.0 with third-party notices in a NOTICE file. Those are compatible licences, but they are not the same licence, and a product that bundles one component and not another carries only the terms of what it actually includes.

The audience is infrastructure engineers working on large model inference. The fabric-lib work comes with an MLSys'26 paper on RDMA point-to-point communication for LLM systems, and the blog posts cover trillion-parameter models on AWS EFA, weight transfer for RL post-training in under 2 seconds, and disaggregated prefill and decode.

The Python package is a shell around a PyO3 extension

The Python side of the repository is thin by design. The build system is setuptools with setuptools-rust and setuptools-scm:

[build-system]
requires = ["setuptools>=80", "setuptools-rust>=1.12", "setuptools-scm>=9"]
build-backend = "setuptools.build_meta"

The package is named pplx_garden, its version is dynamic, and setuptools_scm writes that version into python/pplx_garden/_version.py. The package directory maps to the python directory, and the native part is declared as an extension module named pplx_garden._rust, built from python-ext/Cargo.toml with a PyO3 binding:

[[tool.setuptools-rust.ext-modules]]
target = "pplx_garden._rust"
path = "python-ext/Cargo.toml"
binding = "PyO3"

Only numpy is listed as a runtime dependency, which is a good sign about how little the Python surface promises. The README does not document an install command, so the practical route is to build the wheel yourself with the declared backend, which means a Rust toolchain is needed even for the Python-facing parts.

The lint configuration shows the house style, and it is unusually thorough. Ruff runs with a line length of 88 and a rule set that includes pycodestyle, Pyflakes, pyupgrade, isort, bugbear, annotations, blind except, logging format, numpy and refurb rules, with E501 ignored and Any annotations allowed. Import ordering treats pplx_garden, tests and benchmarks as first party.

The Cargo workspace holds the real work, and leaves lily out

The Rust side is a Cargo workspace with a resolver set to version 2 and an explicit member list:

[workspace]
members = [
  "fabric-lib",
  "fabric-lib/fabric-debug",
  "p2p-all-to-all",
  "pplx-unigram",
  "python-ext",
  "rust/build-utils",
  "rust/cuda-lib",
  "rust/logging-lib",
  "rust/proc-lib",
  "rust/thread-lib",
  "rust/torch-lib",
]
exclude = ["lily"]
resolver = "2"

The exclusion is the interesting line. lily is a separate build, consistent with it carrying its own Apache-2.0 licence, while everything else shares the workspace dependency table and the root MIT licence. The supporting crates are named for what they do, and the names suggest the shape of the codebase: a CUDA library, a torch library, and smaller utilities for logging, threads and processes.

The shared dependency table pins versions tightly, which is what you expect from a project that has to interoperate with a vendor network stack. Entries include tokio 1.48.0, pyo3 0.27.1 with the anyhow feature, clap 4.5.51, serde 1.0.228, cxx 1.0.187 for C++ interop, bindgen 0.72.1 for generating bindings, and syscalls 0.7.0. A rust-toolchain.toml and a rustfmt.toml at the root pin the toolchain and formatting, and Cargo.lock is committed, so builds are expected to be reproducible rather than floating.

fabric-lib moves bytes between hosts, and p2p-all-to-all does the MoE pattern

The two directories that overlap most are worth separating, because they solve different problems inside the same cluster.

fabric-lib is a general transport. It provides an RDMA TransferEngine and a P2P MoE dispatch and combine kernel, and its documentation is in docs/fabric-lib.md. The work behind it is published as the paper fabric-lib: RDMA Point-to-Point Communication for LLM Systems, with companion writing on enabling trillion-parameter models on AWS EFA and on weight transfer for RL post-training in under 2 seconds. The through-line is that moving weights between machines over the network is often the bottleneck in large scale training and inference, and that point-to-point RDMA transfers beat going through a server.

p2p-all-to-all is the specific collective pattern behind that idea, and it is listed in the directory structure as the P2P MoE All-to-All implementation. The difference in approach is real. fabric-lib is the mechanism that makes a direct transfer possible, exposing a transfer engine and a kernel that can dispatch tokens to experts and combine the results. p2p-all-to-all is the pattern of having every expert talk to every other expert, which in a mixture-of-experts deployment is the traffic shape that makes a point-to-point transport worthwhile in the first place.

The blog post on disaggregated prefill and decode sits alongside these, and it points at a third deployment idea, splitting the prefill and decode phases of inference across separate machines so each can be scaled on its own.

pplx-unigram attacks tokenizer CPU cost, and lily targets the laptop

The other two projects are much smaller and sit at opposite ends of the stack.

pplx-unigram is a Unigram tokenizer encoder, documented in docs/unigram.md. The research write-up attached to it is titled Improving Unigram Tokenizer CPU Performance, which tells you the concern: tokenization is close enough to the model that it can matter, and on CPU it competes with the inference work itself. A tokenizer encoder written in Rust and exposed to Python through the same extension mechanism is the obvious way to remove that overhead.

lily is a different proposition. It is a Rust and Metal inference server for Qwen3.6-35B-A3B on Apple Silicon, and it provides greedy text generation through a minimal OpenAI-compatible HTTP API. Greedy decoding and a minimal API are deliberate limits: this is not a serving stack with batching, streaming and sampling controls, it is a way to run one model on local hardware through an interface that existing clients understand. Its documentation is lily/README.md, and its licence and third-party notices are separate files inside the subproject.

Placing them together gives the garden its shape. fabric-lib and p2p-all-to-all are for a datacenter, pplx-unigram is a component that could sit in either place, and lily is for the machine on your desk.

No releases, but a repository laid out like an engineering project

There are no GitHub releases, so the components are consumed from the repository rather than from versioned artefacts on a package index, apart from the Python package which is built from source. The repository is not archived and the last push was on 2026-09-02, so the code is being worked on.

What the tree does show is a set of engineering conventions rather than a pile of experiments. There is a benchmarks directory, a tests directory, a scripts directory and a docker directory at the root, and a MANIFEST.in alongside the Python packaging. A separate fabric-debug crate in the workspace suggests a dedicated tool for inspecting transfers rather than a debug mode bolted into the main library. The documentation directory holds one markdown file per subproject, and each project is described in the README through its docs, its paper and its blog posts rather than through a quickstart, which tells you the intended entry point for each is the written research alongside the code.

For an adopter, the practical consequence is that there is no supported upgrade path in the usual sense. You pin a commit, you read the docs and the blog posts for that commit, and you rebuild.

Editorial conclusion

Adopt pplx-garden if you are building large scale inference and need RDMA point-to-point transfers, a Unigram tokenizer encoder, or a Metal serving path for Apple Silicon, and you are willing to work with components that have no tagged releases. Do not adopt it as a single installable library, since the repository is a garden of independent projects with their own licences, and the lily subproject is Apache-2.0 while the root is MIT. Verify first that your hardware and drivers match what fabric-lib expects for RDMA, that the Python extension builds against your toolchain through setuptools-rust and PyO3, and that the licence of each subproject you copy into a product matches the terms you owe.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. perplexityai/pplx-garden on GitHub
4. README