AI-Horde: a crowdsourced inference cluster you can run yourself

The problem AI-Horde solves: a queue where the compute belongs to strangers

Most people who want to generate an image or a paragraph with a modern model have two options. Pay a hosted provider, or own a GPU large enough to run the model locally. AI-Horde targets the gap between those two. The README describes it as a free community service where volunteers share computer power through workers, in the spirit of Folding@home and SETI@home. The comparison is deliberate: the compute is donated, not rented.

The audience is therefore split. On one side are users who cannot afford a GPU and want image or text generation through a REST API. On the other are operators who have idle hardware and want to contribute it, earning kudos that give their own requests priority. The README is explicit that kudos can never be bought or sold and that doing so violates the Terms of Service, and that kudos never expire. That single rule is what separates this from a marketplace.

The second audience is less obvious and more interesting: teams that want the middleware without the public crowd. The README states the AI Horde middleware can be deployed privately within any enterprise environment, installed within hours and scaled within days. That is a claim about the software, not the public service, and it is the part worth evaluating if you already have a rack of GPUs and no scheduler.

Requests, jobs and workers: the mechanism behind the kudos ledger

The README describes the flow in plain terms. A user makes a request, such as a painting of a sunset over mountains, and the system finds available volunteer computers that can handle the job. The ride-sharing analogy in the README is accurate up to a point: the dispatcher matches supply to demand rather than running the model itself.

What the repository adds is the vocabulary. A worker is a machine running the horde worker software. A job is one unit of generation handed to a worker. Kudos is the accounting layer that records who contributed and who consumed. The README links a dedicated kudos explanation, and the repository keeps a kudos accounting explanation under docs/. The important design property is that kudos is not a payment rail: it is a priority signal. Users with more kudos get faster service, anyone can use the service for free, and anonymous use is supported.

The stack underneath is conventional and that is a feature. The README lists Python 3.9 or later, PostgreSQL, Redis for caching, and Docker for containerization. The pyproject file is stricter than the README: requires-python is >=3.12,<3.14. If you are planning a deployment, trust the pyproject constraint, not the README sentence. The API layer is Flask with flask-restx, served by waitress, with flask-limiter for rate limiting and Flask-Caching in front of Redis.

One dependency deserves attention. The pyproject comments describe horde_sdk as the single source of truth for image sampler capabilities, covering which solver knobs each sampler accepts, how many model evaluations a step costs, and which sampler, scheduler and baseline combinations cannot produce an image. Request validation, pricing and the published constraints endpoint all read from it. That is a sensible way to stop the API, the workers and the clients from drifting apart, but it also means an SDK version bump can change what the API accepts.

Bringing up a private instance with docker-compose

The repository ships a docker-compose.yaml with three services: aihorde, postgres and redis. The aihorde service builds from the local Dockerfile, publishes port 7001, sets PROFILE=docker, mounts .env_docker into /app/.env_docker and mounts ./horde into /app/horde so you can change source without rebuilding. It depends on postgres being healthy and redis having started. The compose file comment notes that the port before the colon is the one exposed to the outside, so you can change it and reach the service on localhost at your chosen port.

Start it with the standard compose command from the repository root:

docker compose up -d

Postgres uses the image ghcr.io/haidra-org/ai-horde-postgres:latest, sets POSTGRES_PASSWORD to changeme, and publishes 5432. The compose file comment explains that PostgreSQL 18 and later keep major-version-specific directories below /var/lib/postgresql, which is why the volume is mounted at that parent path rather than a versioned subdirectory. The healthcheck runs pg_isready -U postgres every 2 seconds. Change that password before the container is reachable from anywhere but your own machine.

The Dockerfile is worth reading before you build. It is a two-stage build on python:3.12-slim. The build stage installs a pinned uv (AI_HORDE_UV_VERSION defaults to 0.9.18), exports dependencies from uv.lock with uv export --frozen --no-dev, and builds wheels. The run stage installs those wheels. There is a notable workaround: the git URL for patreon-python is rewritten to the bare package name with sed, because that project's setup.py unconditionally requires pytest-runner, which is not available offline. If you maintain a fork, that sed line is the kind of thing that breaks quietly when upstream changes.

To exercise the API once the stack is up, or against the public instance at aihorde.net, the README points to README_integration.md and README_return_codes.md. The public API is documented at aihorde.net/api. The README also notes that OAuth2 registration is the recommended account path, with pseudonymous and anonymous usage as alternatives, and that the public site lists services built on top of the horde.

Where AI-Horde is the wrong tool

The design that makes AI-Horde cheap is the same design that makes it unpredictable. Workers drop in and drop out. The README frames this as scalability, and for throughput it is. For latency it is not. If your application needs a p99 response time, a volunteer cluster cannot give you one, because the dispatcher is matching against whatever hardware happens to be online.

Reproducibility is the second limit. A job goes to whichever worker can handle it. The pyproject comment about horde_sdk being the shared vocabulary for sampler capabilities implies that the accepted parameter surface is centrally defined, but that is about validation, not about which model checkpoint a given worker loads. If you need a specific model version for every request, the public horde is the wrong venue. A private deployment narrows this, because you control which workers join, but the middleware itself does not appear to pin a job to a named worker.

The third limit is operational. The stack is Flask, PostgreSQL and Redis, which is easy to run and easy to run badly. The compose file ships a default database password. The Dockerfile pins a uv version and rewrites a dependency URL. Neither is exotic, but both mean upgrades are not a matter of pulling a tag and restarting. The repository does ship a CHANGELOG.md, and the pyproject version is 5.1.11, so there is a version history to read before you move.

Finally, consider the licence before you build a product on it. AGPL-3.0 is a strong copyleft licence with a network clause. Running a modified AI-Horde as a network service is exactly the situation that clause addresses. That is a question for your own counsel, not for this article, but it belongs on the checklist before the first commit, not after.

AI-Horde compared with a hosted model router

The natural alternative for a developer who wants text or image generation through an API is a hosted router that fronts several commercial model providers. The difference is not the interface. Both expose a REST API and both hide the hardware. The difference is who owns the capacity and who sets the terms.

With a hosted router, you get a contract: a provider commits to availability, you get a key, and you pay per token or per image. Model versions are chosen by the provider and change on the provider's schedule. With AI-Horde, capacity is donated and the README is clear that access is meant to stay equitable rather than guaranteed. Kudos buys priority, not a reservation. You are trading a service level agreement for a community resource, and the trade is visible in the README's own framing.

There is a second alternative that is closer in spirit: running your own inference server and skipping the dispatcher entirely. If you have one GPU and one model, a queue is overhead. AI-Horde earns its complexity when you have many workers, intermittent availability, and more demand than any single machine can serve. Below that threshold, the middleware is a liability rather than an asset. The README's own claim that a private deployment can be installed within hours is plausible for the software; it says nothing about the time to recruit and operate the workers.

Maintenance, upgrades and what the licence commits you to

The repository is not archived, and the last push was on 2026-09-04. That is recent enough that the project is not dormant, but it is also not a signal about release cadence: no releases were retrieved for this article, so the version history you can rely on is CHANGELOG.md and the version field in pyproject.toml, currently 5.1.11.

Upgrade cost concentrates in two places. The first is the dependency graph. The Dockerfile resolves from uv.lock with uv export --frozen, so the lock file is the contract. The pyproject pins Flask and Werkzeug to the 2.2 series, Redis to 4.3.5, and horde_sdk to 0.31.0, with horde_model_reference constrained to >=8.4.0,<9.0.0. Those are tight ranges. A major bump in horde_sdk, which the pyproject describes as the shared vocabulary for sampler capabilities, is the upgrade most likely to change observable API behaviour.

The second is the database. The compose file mounts the Postgres data volume at /var/lib/postgresql rather than a versioned subdirectory, specifically so that an image update cannot reinterpret an older data directory. That is a thoughtful detail, and it also means your data directory layout is now tied to that convention. Back up before you change the postgres image tag.

On licence: the project is AGPL-3.0, and the source files carry SPDX headers naming AGPL-3.0-or-later for code and CC0-1.0 for requirements.txt. The AGPL network clause is the part that matters for anyone planning to expose a modified instance to other users. Whether your specific deployment triggers it is a legal question, and this article cannot answer it. What can be said is that the licence is not permissive, and that is a deliberate choice consistent with a project whose README states it is committed to staying free and community-run.

Editorial conclusion

Adopt AI-Horde if you want a REST API in front of a pool of volunteer or internal GPUs and you accept that a job is only as good as whichever worker picks it up. Do not adopt it if you need a guaranteed latency SLA, a fixed model version, or per-request isolation: the README describes drop-in and drop-out compute, which is the opposite of a pinned deployment. Before committing, verify three things against your own environment: that your Python version satisfies the pyproject constraint of >=3.12,<3.14, that the aihorde service in docker-compose.yaml answers on port 7001, and that your .env_docker sets the PostgreSQL and Redis endpoints your containers actually expose. The kudos accounting is the part most likely to surprise you, so read docs/ and the kudos explanation before you expose the API to anyone else.

Frequently asked questions

Official sources

1. Haidra-Org/AI-Horde on GitHub
2. Issues
3. License: AGPL-3.0
4. README