LlamaFarm: A Local AI Platform With a YAML File at the Center

The Problem: Document AI That Never Leaves the Machine

Most teams that want retrieval over their own PDFs end up assembling four or five things: an embedding model, a vector store, an OCR step for scanned pages, a generation endpoint, and a scheduler to keep ingestion from blocking the API. LlamaFarm's pitch is that this assembly is the product. The README states the platform runs entirely on your hardware, that data never leaves the device, and that it works offline once models are downloaded. The target reader is someone with a pile of documents and a GPU who does not want per-token billing or a data processing agreement. The capability table makes the scope concrete: RAG over PDFs, docs and CSVs, text classifiers trained with 8 to 16 examples via SetFit, more than twelve anomaly detection algorithms for batch and streaming, OCR and structured extraction from images and PDFs, named entity recognition, and tool calling over the Model Context Protocol. That is a wide surface for one project, and the breadth is the first thing to weigh. A team that only needs embeddings and a vector index is paying for OCR, anomaly detection and a Designer UI whether or not it uses them.

Three Services, Three Ports, One Config File

The architecture section lists three components. The Server is a FastAPI REST API on port 14345 that also serves the Designer web UI and handles project management. The RAG Worker is a Celery worker for asynchronous document processing and has no fixed port in the table. The Universal Runtime handles ML model inference, embeddings, OCR and anomaly detection on port 11540. Configuration is consolidated into llamafarm.yaml, which the README describes as the place where all settings live, with no scattered settings or hidden defaults. The claim is worth testing rather than trusting, because runtime provider settings, dataset definitions and processing strategies all appear to route through that one file. The data flow implied by the CLI is a loop: a dataset is created and linked to a processing strategy and a database, files are uploaded, the Celery worker processes them asynchronously, and queries hit the resulting store through semantic search with optional metadata filtering. Keeping ingestion asynchronous is the right call for large PDF batches, and it is also why the RAG worker exists as a separate process rather than a background thread inside FastAPI.

Choosing a Runtime Provider in llamafarm.yaml

The runtime block is the part most teams will edit first, and the README gives three shapes for it. The Universal Runtime is the recommended option and points at a local base_url of http://127.0.0.1:11540/v1 with a HuggingFace model id such as Qwen/Qwen2.5-1.5B-Instruct. Ollama is the simpler path for GGUF models, using base_url http://localhost:11434/v1 and a tag like qwen3:8b. The OpenAI-compatible provider covers vLLM, Together, Mistral API or any endpoint speaking that dialect, with an api_key field that reads from an environment variable via ${OPENAI_API_KEY}. That third option is the honest escape hatch: the project markets local execution, but the same config key lets you point at a hosted API when a local model is not good enough. Note what the YAML does not do. It does not choose between OCR engines, even though the Universal Runtime lists Surya, EasyOCR, PaddleOCR and Tesseract. It does not set anomaly detection algorithms, though Isolation Forest, One-Class SVM, Local Outlier Factor and autoencoders are named. Those selections must live somewhere else in the project, and the README does not show where.

Getting It Running: Desktop App or Three Terminals

There are three installation paths, and they cost very different amounts of effort. The desktop app is a direct download for Mac universal, Windows, Linux x86_64 and Linux ARM64, and the README says it needs no additional setup. The CLI path installs via a shell script from the repository, then runs lf init my-project to generate llamafarm.yaml and lf start to bring up services and open the Designer UI at http://localhost:14345. Chat is lf chat for an interactive session or lf chat "Hello, LlamaFarm!" for a single message. The source path is the heaviest: clone the repository, install Nx globally, run nx init --useDotNxInstallation --interactive=false, which the README marks as required on first clone, then start three services in separate terminals with nx start server, nx start rag and nx start universal-runtime. The RAG workflow itself is four commands: lf datasets create -s default -b main_db research, lf datasets upload research ./papers/*.pdf, an optional lf datasets process research if you passed --no-process, and lf rag query --database main_db "Your query". There is also lf rag health for checking the pipeline, and lf models list plus lf chat --model powerful for model selection.

Where the Local-First Design Stops Being the Right Tool

The privacy guarantee is also the ceiling. Once models are downloaded the platform works offline, which means every capability depends on a model that fits your hardware. A 1.5B parameter instruct model configured in the README's example is not a frontier model, and the documentation does not describe a routing layer that sends hard queries to a larger remote model while keeping documents local. The OpenAI-compatible provider can point at a hosted endpoint, but that trades away the property the project is built around. The second constraint is operational. Three services on two named ports plus a Celery worker is a real deployment, and the source instructions require three separate terminals and a global Nx install before anything runs. The third is versioning. The most recent release listed is v0.0.34 from May 2026, following v0.0.33 and v0.0.32 at roughly two to three week intervals. A 0.0.x line moving that fast is not a stable API contract, and anything you build against the YAML schema or CLI flags should be pinned. Finally, the README does not state memory or VRAM requirements for any runtime option, so capacity planning is guesswork until you measure it yourself.

How LlamaFarm Differs From a Plain Ollama Setup

The obvious comparison is Ollama alone. Ollama serves GGUF models over an OpenAI-compatible endpoint at port 11434, and LlamaFarm can use exactly that as its provider, which tells you the two are not competitors at the model layer. The difference is everything above it. Ollama gives you inference and nothing else; you still write the ingestion loop, choose an OCR library, pick an embedding model, wire up a vector store and build a query interface. LlamaFarm ships those as named services with a CLI in front of them, and its Universal Runtime adds OCR engines, reranking cross-encoders, SetFit classification, NER and anomaly detection that Ollama does not attempt. The cost of that convenience is the three-process footprint and the config surface. If your workload is chat against one model, Ollama plus a small script is less to operate. If your workload is PDFs in, structured answers out, with classification or anomaly checks alongside, LlamaFarm is doing work you would otherwise write yourself. The reranking option matters here too: cross-encoder reranking is a documented part of the Universal Runtime and is a common quality lever for retrieval that a bare inference server cannot provide.

Licence, Upgrade Cadence and What That Costs You

The project is Apache-2.0, which permits commercial use, modification and redistribution, and includes an explicit patent grant. That is a permissive licence, and it means embedding LlamaFarm in a product does not obligate you to publish your own source. It does not resolve the separate licences of the models you load through it: Qwen, Llama, Gemma and Mistral weights each carry their own terms, and the Apache-2.0 badge on the repository says nothing about them. Check the model licence before shipping, and treat that as a distinct review from the code licence. On maintenance, the release history shows v0.0.32 in late April 2026, v0.0.33 in mid May and v0.0.34 in late May, with the last push to main in June 2026. That cadence is active but pre-1.0, so upgrades can require YAML edits or renamed CLI subcommands. Pin the version you install, keep llamafarm.yaml under version control so schema drift is visible in a diff, and re-run lf rag health after each bump to confirm the worker and runtime still agree on the pipeline.

Editorial conclusion

Adopt LlamaFarm if you need document ingestion, embeddings and small-model inference to stay on hardware you control, and you accept running a FastAPI server, a Celery worker and a runtime on separate ports. Do not adopt it if you want a single binary or a stable API surface, since the latest release is v0.0.34 and the CLI, YAML schema and Designer UI can move between releases. Before committing, run lf init, lf start and lf rag health on your own machine, then confirm your GPU or NPU appears in the acceleration path the documentation claims for Apple Silicon, NVIDIA and AMD.

Official sources

1. License: Apache-2.0
2. llama-farm/llamafarm on GitHub
3. Project website
4. README
5. Releases