uzu: running Qwen3.5 locally from Rust, Python, Swift and TypeScript

The problem uzu targets: inference inside the app, not behind an endpoint

The README frames the value in three claims: zero latency, full data privacy, and no inference costs. Those three only hold together under one deployment shape, which is a model running on the same device as the caller. A network round trip reintroduces latency and puts prompts on someone else's machine. A hosted API reintroduces per-token billing. So uzu is aimed at people building desktop, mobile or embedded applications where the model ships alongside the binary and the user's text never leaves the process. The Python and TypeScript packages on PyPI and npm suggest a second audience: people who want to try a local model from a script without writing Rust, and people embedding chat into a web or Node application. The Swift package descriptor targets iOS and macOS, which is where the unified memory claim in the feature list matters most. If your workload is batch inference on a server with a GPU pool, uzu is not describing your problem.

Engine, downloader, session: the three objects the API actually exposes

The quick-start examples across all four languages follow the same sequence, and that sequence is the architecture. First you construct an Engine from an EngineConfig. In Rust that is Engine::new(engine_config).await, in Python await Engine.create(engine_config), in Swift try await Engine.create(config: engineConfig), and in TypeScript await Engine.create(engineConfig). The engine is the long-lived object that owns model resolution. Second you ask it for a model by identifier string. The README uses alibaba:qwen3.5:0.8b:mirai:mirai-m:4, a colon-delimited identifier that appears to encode publisher, family, size, and two further qualifiers. The call returns an optional, and every binding handles the miss explicitly: Rust uses ok_or("Model not found"), Python and Swift return early on None or nil, TypeScript throws. Third, engine.download(model) returns something iterable that yields progress updates. Rust and Swift expose an update.progress() method returning a fraction; Python and TypeScript expose a progress field or property. Fourth, engine.chat(model, ChatConfig) produces a session, and session.reply(messages, ChatReplyConfig) returns a list of replies. The last element carries a message with separate reasoning() and text() accessors, which means the engine distinguishes a reasoning channel from the final answer rather than returning one flat string. The README also lists traceable computations as a feature, which implies the engine can be checked against a reference implementation, but no example of that appears in the material.

Getting it running: four package managers, one model identifier

The install lines are the most concrete part of the README and they differ per binding. Rust uses a git dependency rather than a crates.io release: uzu = { git = "https://github.com/trymirai/uzu", branch = "main", package = "uzu" }. That means a Rust build tracks the main branch, not a tagged version, so a rebuild can pull different code than the previous build. Python uses uv add uzu==0.5.26, pinned to the release that matches the README. TypeScript uses pnpm add @trymirai/uzu@0.5.26, also pinned. Swift uses Swift Package Manager with .package(url: "https://github.com/trymirai/uzu.git", from: "0.5.26"), and the Package.swift file is what declares iOS and macOS support. After installation the code path is identical in shape: create the engine, resolve the model, download it with a progress loop, open a chat session, send a system message plus a user message, and read the last reply. The Rust example imports ChatMessage, ChatConfig and ChatReplyConfig from uzu::types::session::chat and uses builder methods ChatMessage::system() and ChatMessage::user() with with_text(). Note that the Rust example needs a Tokio runtime, since main is annotated with #[tokio::main] and the engine constructor is awaited.

Where the README stops: model weights, licensing and platform limits

Several things a reader would need before adopting uzu are not in the repository README. The model catalogue is a link out to trymirai.com/local-models, so the set of supported architectures cannot be checked from the repo alone. The identifier format is shown by example but never specified, so it is unclear whether the four colon-separated segments are stable or whether a user can point the engine at a local file path or a Hugging Face repository. There is no mention of quantization formats, context length limits, or memory requirements per model size, which matters because the headline feature is unified memory on Apple devices and the example model is a 0.8B one. The MIT licence covers the repository, but the README says nothing about the licence of the model weights the downloader fetches, and those are separate artifacts from a separate publisher. The Python and TypeScript bindings live under a path containing crates/legacy/uzu/bindings, and the directory name legacy is a signal worth investigating before depending on them. Finally, the release cadence visible in the supplied data is fast: 0.5.23 on 2026-09-03, 0.5.25 on 2026-09-04, 0.5.26 on 2026-09-06. Rapid patch releases are normal for young projects, but they also mean the git branch dependency in the Rust example will move under you.

How uzu differs from llama.cpp and MLX

The obvious comparison is llama.cpp, which also runs models locally and also ships bindings for multiple languages. The difference in approach is what the API returns. llama.cpp exposes a lower-level interface around context creation and token sampling, and its Python binding mirrors that: you manage the context, the sampling parameters and the token loop. uzu wraps the whole thing in an Engine that owns model resolution and download, and hands back a session whose reply already separates reasoning from text. That is a higher-level contract, and it costs you control over sampling and KV cache behaviour, which the README does not expose at all. On Apple hardware the closer comparison is MLX, which is Apple's own array framework and provides the primitives to build inference rather than an inference engine with a chat session API. MLX gives you more room to modify the model; uzu gives you a chat session in about twenty lines. If you need to run a model architecture that is not in the trymirai.com catalogue, the higher-level contract becomes a wall rather than a convenience.

Maintenance cost: pinned packages against a moving git branch

The upgrade surface splits by binding. Python and TypeScript consumers pin an exact version, so an upgrade is a deliberate edit of one line in a manifest and the changelog between releases is the thing to read. Swift consumers use from: "0.5.26", which allows any later 0.5.x, so a rebuild can pick up a new patch without a manifest change. Rust consumers using branch = "main" get no version boundary at all, and given three patch releases in four days, that is the highest-churn option of the four. If you are building on uzu from Rust, switching the dependency to a tag or a crates.io release would make builds reproducible, assuming a tagged release exists. The MIT licence is permissive and imposes no source disclosure obligation on your application, but it covers the code in this repository. It does not automatically cover downloaded model weights, and the README does not address them. That is a question for whoever publishes the weights, not a question the uzu repository answers.

What to check before you build on uzu

The repository gives you enough to run the quick-start and not much more. Three checks are worth doing first. Run the Python or TypeScript example against the exact model identifier you intend to ship, since the catalogue lives off-repository and the identifier format is only shown by example. Read Package.swift and the bindings directories to confirm that the platform you target is actually covered, rather than assuming the Swift badge implies every Apple platform. And compare the release notes for 0.5.23 through 0.5.26 to see whether the API in the README is stable across that window, because a project shipping three patches in four days is still settling. If those checks pass, the multi-language surface is the real argument for uzu: one mental model, four bindings, and a chat session that fits on one screen in each.

Editorial conclusion

Adopt uzu if you are shipping an app that needs on-device chat and you want the same engine call from Rust, Python, Swift and TypeScript rather than a different SDK per platform. Do not adopt it if you need a documented HTTP server, a training path, or a supported model list that lives in the repository instead of on trymirai.com. Before committing, verify three things: that the model identifier you actually need resolves through engine.model(), that the pinned package version on PyPI, npm or SPM matches the git branch you plan to build against, and that the licence terms you care about are covered by the MIT file in the repository, since the README does not discuss model weights at all.

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. trymirai/uzu on GitHub