# pdf-brain: a local PDF and Markdown knowledge base with vector search

> pdf-brain extracts text from PDFs and Markdown, asks a local Ollama model to enrich each document, turns it into embeddings, and stores the result in a local libSQL vector index with an HNSW structure. It is agent-first, with a JSON envelope on stdout and an MCP server for assistants.

**joelhooks/pdf-brain** — 🧠 Local PDF knowledge base with vector search using libsql w/vectors

- Repository: https://github.com/joelhooks/pdf-brain
- Stars: 662 · Forks: 59
- Language: TypeScript
- License: MIT
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/joelhooks-pdf-brain

## A knowledge base that ends in a local database

The repository description calls pdf-brain a local PDF knowledge base with vector search using libSQL with vectors, and the README broadens that to PDF and Markdown files in one unified, searchable knowledge base. The local-first framing is the design decision that everything else follows from. The feature list says everything runs on your machine with no API costs, and that is achieved by making the language model a local process rather than a remote endpoint, so the enrichment step and the embedding step both go through a model server you start yourself. The second design decision is the storage engine. Rather than reaching for a dedicated vector database service, the project uses libSQL with vector support, and the architecture diagram in the README shows the last stage as an HNSW index with cosine similarity, which means the embeddings, the metadata, the full text and the taxonomy all live in one database file that you can copy, back up or delete. That has a practical consequence for anyone evaluating it, because the entire index is a local artefact rather than a service you have to operate, and the same artefact can be moved between machines or committed to a repository if you choose to.

## The four stage pipeline, and what each stage is for

The README draws the pipeline as a row of boxes, and the diagram is the clearest description of the system in the project. Documents go in as PDF or Markdown, they are extracted with pdf-parse for PDFs and a Markdown parser for notes, and then the flow splits into two model calls. The first is enrichment, where a local LLM, shown as llama3.2:3b, reads the document and produces the metadata. The second is embedding, where a separate local model, shown as mxbai-embed at 1024 dimensions, turns the text into a vector. The vectors and the metadata land in libSQL, where they are stored in an HNSW index and searched with cosine similarity. Separating the two model calls is the part that matters for cost and for control. The embedding model is required, because without it there is nothing to search, while the LLM for enrichment is optional and only makes documents nicer to browse and organise. The same separation is what lets the tool offer an auto-tag mode, described as faster and using heuristics plus a light LLM, and a full enrichment pass, described as the slower one that asks the model for everything.

## Agent-first output and the capabilities contract

The README is explicit that pdf-brain is agent-first and emits a single JSON envelope to stdout by default, with a --format text option for human readable output and for TUI and progress rendering. The global flags are listed plainly, with --format accepting json, ndjson or text and defaulting to json, a --pretty flag to pretty print the JSON, a --quiet flag, described as an alias for --no-hints, that omits the nextActions field, and a --log-level flag taking silent, error, info or debug, with logs going to stderr so they never corrupt the envelope on stdout. That last point is the detail that makes the design usable by a program rather than a person. The contract is also discoverable at runtime, because there is a capabilities command that exposes the full command and tool contract including JSON Schemas.

```bash
pdf-brain capabilities
```

So an agent does not have to scrape help text to find out what it may call. It asks. The output format flags then let the same binary serve two audiences, a person at a terminal who wants readable text and a spinner, and a model or script that wants a single parseable object with a hint field describing sensible next steps. The package manifest shows the tooling that supports this, with the binary mapped to a CLI entry point and a bin field, and the build script producing a standalone bundle with the bun runtime targeted.

## Installing the binary, Ollama and the two models

The install path is short, and the README gives a standalone binary first, with an npm route as the alternative.

```bash
# Standalone binary (no runtime needed)
curl -fsSL https://raw.githubusercontent.com/joelhooks/pdf-brain/main/scripts/install.sh | bash

# or via npm
npm install -g pdf-brain
```

Ollama is the one hard prerequisite, because it provides the embeddings. The README lists the platform routes for it, Homebrew on macOS, a shell installer on Linux, and a download for Windows, and then the two model pulls.

```bash
# Required: Embedding model (1024 dimensions)
ollama pull mxbai-embed-large

# Recommended: Local LLM for enrichment
ollama pull llama3.2:3b

# Start Ollama server
ollama serve
```

Note the comment on the second pull, which calls the local model recommended rather than required, and the note in the prerequisites section that the LLM is optional while Ollama itself is not. That split is what makes a minimal install possible, in which the tool indexes and searches documents but produces less structure for them. The two remaining setup steps are the library initialisation, which creates the database and seeds a starter taxonomy, and adding the first document.

```bash
pdf-brain init
pdf-brain add ~/Documents/paper.pdf --enrich
```

## Three search modes over the same index

Search comes in three flavours and the distinction between them matters when you are indexing a large collection. The default is semantic, using the embeddings, so a query about context engineering patterns can find a document that never uses that phrase. The full-text mode is selected with --fts and the README notes that it is faster and uses no embeddings, which is the mode you want when you are grepping for an exact token and do not want to pay for the vector path. The hybrid mode combines the two.

```bash
# Semantic search (uses embeddings)
pdf-brain search "context engineering patterns"

# Full-text search only (faster, no embeddings)
pdf-brain search "context engineering" --fts

# Hybrid search (combines both)
pdf-brain search "machine learning" --hybrid
```

There are two modifiers that shape the result set. A limit flag caps the number of results, and an expand flag widens the context around each match by a number of characters, which is what you want when a hit is a sentence inside a chapter rather than the whole document. The document management commands follow the same pattern of small, explicit verbs, with add accepting a local path, a URL for a PDF or a raw Markdown file, a tags option and the enrich flag, and then list, read, remove and tag for the library itself, where list can be filtered by tag and tag replaces the tag set for a document.

## SKOS taxonomy and the fields the model fills in

The organisation layer is a taxonomy, and the README specifies the standard it uses, SKOS, the Simple Knowledge Organization System, for hierarchical concept organisation. The command group is a small vocabulary of its own, with list and tree to inspect, a tree subcommand that can start from a named concept, search to find concepts, add to create one with a label and a broader parent, assign to attach a concept to a document, and seed to load a taxonomy from a JSON file.

```bash
# Add a new concept
pdf-brain taxonomy add ai/transformers --label "Transformers" --broader ai-ml

# Assign concept to document
pdf-brain taxonomy assign "doc-id" "programming/typescript"

# Seed taxonomy from JSON file
pdf-brain taxonomy seed --file data/taxonomy.json
```

The enrichment step is what fills the metadata, and the README lists exactly what it produces: a clean, properly formatted title, the author or authors if detectable, a two to three sentence summary, a document type such as book, paper, tutorial, guide or article, a primary category, five to ten descriptive tags, the concepts matched from your taxonomy, and proposedConcepts, which are new concepts the model suggests adding. That last field is the interesting one, because it closes the loop with the taxonomy, turning the model into a source of candidates for the vocabulary rather than only a consumer of it. The model backend is configurable, and the manifest shows that the AI SDK is used with Anthropic and with an OpenAI compatible provider, while the README notes that the local Ollama provider is the default.

## Bulk ingest, and the limits of a local index

Ingest is the command that turns a personal document collection into a knowledge base, and it takes directories rather than files, recursing into them and accepting several at once, with the README calling out an Obsidian vault or notes folder as a use case.

```bash
# Ingest a directory with full LLM enrichment
pdf-brain ingest ~/Documents/papers --enrich

# Auto-tag only (faster, heuristics + light LLM)
pdf-brain ingest ~/docs --auto-tag

# Process only first N files (for testing)
pdf-brain ingest ~/papers --enrich --sample 10

# Disable TUI for simple output
pdf-brain ingest ~/papers --enrich --no-tui
```

The options tell the story of the practical limits. The sample flag exists so you can test the pipeline on ten files before committing to a directory of thousands, and that matters because the full enrichment pass calls the language model once per document. The auto-tag mode is the escape hatch, using heuristics and a light model pass instead of the full enrichment, and the no-tui flag exists so the command can run unattended in a script where a spinner in a terminal would be noise. Beyond those knobs, the constraint is structural. The index is local, so there is no shared multi user library, and the quality of the structured metadata is bounded by a three billion parameter model running on your own hardware, which will produce shorter and shallower summaries and less reliable concept matching than a hosted model would. A large corpus is also a queue of model calls, and the ingest commands are the point at which that becomes obvious. The maintenance picture is fine rather than busy, with the last push on 2026-06-25 and releases v1.3.1 on 2026-02-06 and v2.0.0 on 2026-02-07, and the repository shows a serious release setup with changesets, husky hooks, a secretlint configuration and a supply chain security document.

## Conclusion

Adopt pdf-brain if you want to search a personal document collection semantically without sending anything to a hosted API, and if you are willing to run Ollama locally and accept that enrichment quality is bounded by a three billion parameter model. It does not fit a team that needs shared, multi user indexing, since the whole design is local first with one database on your machine, and it does not fit a corpus that needs a stronger summariser than llama3.2:3b can produce. Before you index anything, run pdf-brain check and pdf-brain capabilities so you know the exact command and tool contract of the build you have, and decide whether the auto-tag mode is enough for a bulk ingest, since the full enrichment pass calls the model for every document.

## FAQ

### What is pdf-brain and what does it index?

It is a local PDF and Markdown knowledge base with semantic search and AI powered enrichment, using libSQL with vectors for the store. It indexes .pdf and .md files through the same workflow, including research papers, books, notes, documentation and Obsidian vaults.

### What do I need to install before using pdf-brain?

Ollama is required because it provides the embeddings, and the README lists the install route per platform. You then pull mxbai-embed-large for embeddings, which is required, and llama3.2:3b for enrichment, which is recommended but optional, start the server with ollama serve, and install pdf-brain itself as a standalone binary or with npm install -g pdf-brain.

### What is the difference between the three pdf-brain search modes?

The default is semantic search using the embeddings, the --fts flag gives full text search only, which the README describes as faster and using no embeddings, and the --hybrid flag combines the two. Results can be capped with --limit and widened with --expand.

### What does the LLM produce when pdf-brain enriches a document?

The enrichment step extracts a clean title, the author if detectable, a two to three sentence summary, a document type, a primary category, five to ten descriptive tags, the concepts matched from your taxonomy, and proposedConcepts, which are new concepts the model suggests adding.

### How does pdf-brain work with AI assistants?

It ships an MCP server, so it can be used with Claude, Cursor and other AI assistants. The CLI is also agent-first: output is a single JSON envelope on stdout by default, with --format text for human readable output, and the full command and tool contract including JSON Schemas can be discovered at runtime with pdf-brain capabilities.

## Sources

- [Issues](https://github.com/joelhooks/pdf-brain/issues)
- [joelhooks/pdf-brain on GitHub](https://github.com/joelhooks/pdf-brain)
- [License: MIT](https://github.com/joelhooks/pdf-brain/blob/main/LICENSE)
- [README](https://github.com/joelhooks/pdf-brain/blob/main/README.md)
- [Releases](https://github.com/joelhooks/pdf-brain/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/joelhooks-pdf-brain