AI Runner: A Local Inference Stack With a Desktop Canvas and a Headless Daemon

The Two Products Inside One Repository

AI Runner is described as two interlocking experiences rather than one tool. The first is a companion: a named entity with an assigned personality, a chosen voice and memory that accumulates across sessions, with awareness of time, date and local weather. The second is a layered canvas where you sketch, paint, generate and filter, converting sketches to images and compositing on layers. The README frames both as running without an internet connection, an API key or a subscription. That framing matters more than it looks. A companion with long-term memory is a stateful application, and stateful applications are the ones that break when their storage moves. Image generation is largely stateless per request. Putting both behind one runtime means the daemon has to serve two very different workloads. The stated audience is people who want the model weights, the conversation history and the generated images to stay on their own disk. If you are comfortable sending prompts to a hosted API, most of this repository is unnecessary weight.

How the GUI, Daemon and Sidecars Fit Together

The package diagram in the README is the clearest statement of architecture. A native launcher and bundle helpers sit at the top. From there, arrows run to the desktop client in src/ and to the headless daemon in services/. The GUI also talks to the daemon directly. Both the native layer and the daemon reach the llama.cpp and whisper.cpp sidecars, and both reach the scripts/ tooling. Both also read and write a data directory identified as AIRUNNER_BASE_PATH. The package table confirms the split: src/ holds the desktop GUI client, the daemon client bridge, widgets and entry points; services/ holds the headless daemon, a FastAPI server, a runtime registry, orchestration, downloads and persistence; native/ holds the Python launcher entry point and runtime layout helpers; scripts/ holds the test runner, UI build, install helpers and quality reports. Two design consequences follow. First, the GUI is a client, so a daemon running elsewhere on the network is a supported shape rather than a hack. Second, the actual inference for text and speech does not happen in Python at all. It happens in the compiled sidecars, which is why the install process builds them.

Installing It: Two Modes, One Script Each

The README states that AI Runner currently runs as a Python application with two install paths. The dev mode is for contributors working from a repo checkout and is invoked as ./scripts/install.sh. According to the documentation, this reuses ./venv by default, installs the Python packages in editable mode, and builds the pinned llama.cpp and whisper.cpp sidecars under build/runtime-sidecars/linux/. Rerunning the command reuses the existing environment rather than recreating it and refreshes the local editable installs without re-solving the full dependency graph. Two flags are documented: --refresh-deps forces a full dependency refresh, and --sidecars-cuda builds CUDA-enabled native sidecars. The distributed mode is for operators who want the daemon and GUI client installed separately, including split-machine setups, and uses ./deployment/install_distributed.sh with a --role argument, shown as --role daemon and --role gui-client. The distinction is worth taking literally. If you run the dev script you get a checkout-shaped environment with editable installs; if you want a daemon on one host and a client on another, the distributed script is the documented route. Python 3.13 is the stated target. The minimum hardware table lists an RTX 3060, 16 GB of RAM and 22 GB of storage, with 100 GB or more recommended, and notes that actual usage varies.

Where the Model Weights Come From

Model acquisition is built in. The feature table lists a model management component with HuggingFace and Civitai downloaders and support for multiple local LLMs and image models. On the generation side the README names SDXL and Z-Image Turbo, plus LoRA, embeddings, image-to-image, inpainting, post-process filters and background removal. On the text side it names Mistral among the topics and describes support for multiple local LLMs without enumerating them. This is the part of the project that ages fastest. Downloader integrations depend on the remote site's API staying stable, and model support depends on someone writing the loading path for each architecture. The README does not describe a fallback when a Civitai or HuggingFace download fails, and it does not list which LLM architectures beyond Mistral are wired up. If your workflow depends on a specific checkpoint, confirm it is supported before you build an environment around this.

The Safety Filters Are Configurable, and That Cuts Both Ways

The feature table describes configurable NSFW output filtering alongside an always-on prompt classifier for illegal content. The wording separates two mechanisms with different status. One is configurable, meaning an operator can change its behaviour. The other is described as always-on, meaning it is not presented as a setting. That is a deliberate posture for a tool that generates images locally and stores them locally, and it will satisfy some users and annoy others. If you are evaluating AI Runner as a general-purpose local inference front end, be aware that a component you cannot turn off sits in the prompt path. The README does not say what the classifier does when it fires: whether it blocks the request, returns an empty result, or logs the event. That is a gap worth closing by reading the code in services/ before relying on it in an automated workflow.

Platform Reality: Linux First, Windows Experimental

The system requirements table lists Ubuntu 22.04 and Windows 10 as minimums, but the footnote attached to Windows is explicit: Windows support is experimental and community-maintained, and Linux is the primary supported platform. The recommended configuration is Ubuntu 22.04 on Wayland. This is the single most consequential constraint in the document. A desktop application with a PySide6 GUI, native compiled sidecars and a CUDA build step has a long list of things that can diverge between operating systems, and the project is telling you it has not closed that gap on Windows. Language support has a similar shape. English has TTS, LLM, STT and GUI support. Japanese has TTS, LLM and GUI but no STT. Spanish, French, Chinese and Korean have TTS and LLM only, with no STT and no GUI. The GUI column being false for four languages is a strong hint that the interface itself is not localised, even where the models handle the language.

What It Does Not Do, and What to Use Instead

AI Runner is not a serving layer. The services/ package contains a FastAPI server, but the README presents it as the daemon that the GUI client talks to, not as a documented HTTP API for third parties. If your goal is to expose an OpenAI-compatible endpoint to a fleet of applications, this is the wrong tool, and the difference is architectural rather than cosmetic. A dedicated inference server such as vLLM or llama.cpp's own server mode is built around request batching, concurrent clients and an API contract. AI Runner is built around one operator, a persistent companion and a canvas. Running it as a shared backend means working against its design. The same reasoning applies to the sidecars. The project pins and builds llama.cpp and whisper.cpp itself, which buys reproducibility and costs you the ability to track upstream releases on your own schedule. There is also a maintenance cost implied by the release cadence: three releases in the eight days before the last push, all patch versions. Fast patch cadence on a project this size usually means active bug fixing, and it also means you should expect to update rather than pin and forget.

Licence and the NOASSERTION Flag

The repository metadata reports the licence as NOASSERTION, while the README carries a GPL v3 badge linking to the GNU licence text. Those two signals disagree, and the disagreement is not something to resolve by assumption. GPL v3, if that is what applies, has consequences for anyone embedding this in a larger product: the copyleft terms reach derived works distributed to others. Running it internally, or using it as an end user, is a different situation from shipping it inside a commercial application. The README does not discuss the licence beyond the badge, and I cannot tell you from the supplied material which file governs or whether the NOASSERTION flag reflects a custom or dual arrangement. Read the LICENSE file in the repository and, if the answer changes your plans, get proper advice rather than a blog post's opinion. The presence of a vulnerability reporting template in the README suggests the maintainers have at least thought about security disclosure, which is a point in favour, but it says nothing about the licence question.

Editorial conclusion

Adopt AI Runner if you want a single self-hosted application that covers SDXL and Z-Image Turbo generation, TTS and STT voice conversation and an LLM companion with RAG recall, and you have an NVIDIA GPU plus 32 GB of RAM on Ubuntu 22.04. Do not adopt it if you need Windows as a first-class target, since the README marks Windows support experimental and community-maintained, or if you need STT outside English, which the language table lists as unsupported. Before committing, verify three things yourself: that the install mode you pick matches how you intend to run it, that the pinned llama.cpp and whisper.cpp sidecars build on your machine, and that the licence file in the repository matches the GPL v3 badge, because the metadata reports NOASSERTION.

Official sources

1. Capsize-Games/airunner on GitHub
2. Issues
3. Project website
4. README
5. Releases