# av/harbor: one command starts a pre-wired local LLM stack

> Harbor is a Python CLI and companion app that orchestrates Docker Compose stacks of LLM backends, frontends and support services so they arrive already connected. It is convenient and opinionated, and its release notes show that keeping dozens of upstream images working is the ongoing cost.

**av/harbor** — Project brief: Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.

- Repository: https://github.com/av/harbor
- Website: https://discord.gg/8nDRphrhSF
- Stars: 3,236 · Forks: 228
- Language: Python
- License: Apache-2.0
- Published: 2026-08-08 · Updated: 2026-08-18 · Language: en
- Canonical page: https://hysenlabs.com/projects/av-harbor

## The problem Harbor targets: an LLM stack that is wired before you open it

Running a model locally is the easy half. The hard half is everything around it: a backend such as Ollama, llama.cpp or vLLM, a frontend such as Open WebUI, a search service for retrieval, a speech service for voice, an image generator. Each has its own container, its own port, its own environment variables, and its own idea of how to reach the others. The README describes Harbor as a CLI and companion app that spins up that whole set with one command, handling "the Docker Compose orchestration, configuration, and cross-service connectivity". The audience is the person who wants to use models rather than maintain a compose file. It is not aimed at teams that already have a deployment pipeline, and it is not a hosted service: the homepage is a Discord invite, so the project's own support channel is chat, not a ticket queue.

## How the CLI, compose.yml and the services directory fit together

The repository layout tells most of the story. There is a `harbor/` directory for the Python CLI, a `services/` directory for per-service definitions, `profiles/`, `boost/`, `app/`, and a root `compose.yml` whose own comment states that Harbor works by combining multiple compose files together and is orchestrated by the CLI. That is the mechanism: the CLI assembles a merged compose configuration from the services you name, then runs it. The README's example is explicit about the payoff. `harbor up` starts a fully configured Open WebUI and llama.cpp, and adding `searxng speaches` to the same command gives Open WebUI web RAG and text-to-speech and speech-to-text. Cross-service connectivity is therefore not something you configure; it is a property of the service definitions. Two consequences follow. First, the CLI is the interface, so anything the CLI does not expose is effectively out of reach without ejecting. Second, the project ships both a Python package (`llm-harbor` in pyproject.toml) and an npm package (`@avcodes/harbor`) whose `bin` entry points at `harbor.sh`, which is why installation paths differ by platform.

## Installing Harbor and running your first two commands

The README points to the wiki page "1.0.-Installing-Harbor" for CLI and app installation, and the repository root contains `install.sh`, `install.ps1`, `requirements.sh` and `harbor.sh`. There is also an `install.md` file described in the README as an agent install prompt for Claude Code, Codex and similar tools, which is the route to take if you would rather have a coding agent perform the setup. The npm package exposes a `harbor` binary, so a Node-based install is available as well; the README does not show the exact npm invocation, so check the wiki page before assuming one.

Once the CLI is on your PATH, the first real use is the command from the README:

```bash
# Starts fully configured Open WebUI and llama.cpp
harbor up
```

What you should see is Open WebUI and llama.cpp brought up as a connected pair, with the frontend already pointed at the backend. The README notes that a failure to reach Open WebUI on its default port triggers a message saying Harbor does not appear to be installed or Open WebUI is not running on the default port, so that text is the signal to check the stack rather than the browser.

The second command extends the same stack with search and speech:

```bash
# Now, Open WebUI can do Web RAG and TTS/STT
harbor up searxng speaches
```

SearXNG supplies web search for retrieval, and Speaches supplies voice input and output. Before relying on any of this, run the health check:

```bash
harbor doctor
```

The release notes for v0.5.1 describe doctor as gaining timeout-guarded compose checks and an early exit for `--check` mode, and v0.5.2 mentions a fix so it no longer hangs on non-interactive stdin. Those two notes together are a fair warning that a diagnostic command which inspects running containers is only as reliable as the containers themselves.

## The maintenance surface is the real limitation

Read the release notes as a maintenance log rather than a feature list. v0.5.4 describes repairs to first-boot and integration failures across more than twenty services, found by a new runnable integration suite. v0.5.5 describes a large repair sweep restoring dozens of services on current upstream images, alongside workspace file ownership changes across twenty-plus services. That is the honest picture of a project that aggregates many third-party images: the integration work is continuous, and a service you depend on can break because an upstream image moved, not because Harbor changed its own code. Plan for upgrades accordingly. If you need a frozen environment where nothing shifts between deployments, Harbor's model of tracking current upstream images works against you, and a hand-written compose file with pinned digests is the better tool. The same applies if you must run without the CLI in the loop: the compose.yml comment says the CLI orchestrates the combination, and `harbor eject` exists precisely to produce your own configuration when you want to leave. The README does not document a rollback procedure for a bad upgrade, so treat version pinning as something you arrange yourself.

## Where Harbor sits next to Ollama and a hand-rolled compose file

The closest comparison is not a single product but the two things people usually do instead. The first is installing Ollama and Open WebUI separately and connecting them by hand. That gives you two well-documented pieces and full control over versions, and it costs you the wiring: every additional service is another container, another port mapping, another environment variable. Harbor's answer is that the wiring is the product, and its v0.5.0 notes show llama.cpp replacing Ollama as the default backend, which means the backend choice is a Harbor decision you inherit rather than a default you set. The second alternative is your own `compose.yml`. You get reproducibility and no extra dependency, and you give up the pre-built service definitions under `services/` and the cross-service defaults that make `harbor up searxng speaches` sufficient. Harbor is worth it when you want to try many services quickly; it is the wrong choice when the stack is small, stable and already scripted.

## Licence, upgrade cost and what the README leaves open

Both package manifests declare Apache-2.0, and the repository carries a LICENSE file, so the CLI and the service definitions are permissively licensed. That covers Harbor's own code. It does not cover the third-party images Harbor pulls in, each of which carries its own terms, and the README does not enumerate them, so check the licence of any service you intend to run commercially. On upgrade cost: the release cadence visible in the notes is frequent, with v0.5.3, v0.5.4 and v0.5.5 landing within about a month, and the last push was on 2026-08-16. Frequent releases with repair sweeps mean upgrades are routine but not free, and the notes themselves are the changelog you should read before moving. The README does not describe a supported downgrade path, and the wiki is where installation and app details live, so the wiki is the first place to look when an upgrade misbehaves.

## Conclusion

Adopt Harbor if you want a local stack that is wired together before you open the browser, and you accept that the CLI owns the compose files, the .env and the ports. Skip it if you need a pinned, reproducible deployment or a stack that must survive without the harbor CLI, since `harbor eject` is the documented way to take your configuration elsewhere. Before committing, run `harbor doctor` and read the wiki page "1.0.-Installing-Harbor" to confirm your platform's install path, then start with `harbor up` and `harbor up searxng speaches` to see exactly which services come up pre-connected on your machine.

## FAQ

### How do I install av/harbor?

The README links to the wiki page "1.0.-Installing-Harbor" for CLI and app installation, and the repository root contains install.sh, install.ps1 and requirements.sh. The npm package @avcodes/harbor exposes a harbor binary, and install.md is described as an agent install prompt for Claude Code, Codex and similar tools.

### What does harbor up actually start?

The README gives the example that harbor up starts a fully configured Open WebUI and llama.cpp, and that adding searxng and speaches to the same command enables web RAG and TTS/STT in Open WebUI.

### Does av/harbor work with Docker?

Yes. Harbor orchestrates Docker Compose, and the root compose.yml comment states that it works by combining multiple compose files together under the control of the CLI. The compose.yml also declares a harbor-network network that is not external.

### What is the licence for av/harbor?

Both package manifests, package.json and pyproject.toml, declare Apache-2.0, and the repository contains a LICENSE file. That covers Harbor's own code, not the third-party service images it starts.

## Sources

- [Official documentation](https://discord.gg/8nDRphrhSF)
- [Official README](https://github.com/av/harbor#readme)
- [Project repository](https://github.com/av/harbor)
- [Release notes](https://github.com/av/harbor/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/av-harbor