flepied/second-brain-agent: a Markdown indexing agent with an MCP server

What Second Brain AI agent indexes, and what it refuses to index

The input is a directory of Markdown notes. The README describes the author's own setup as Obsidian, but nothing in the described pipeline depends on Obsidian specifically: any editor that writes Markdown files into a folder works, because the watcher is looking at files, not at an application. That is the useful boundary. If your notes live in a proprietary database, a hosted workspace, or a format that is not Markdown, this project has nothing to attach to.

The second input is the links inside those notes. transform_md.py extracts text from the Markdown file itself, then follows the links it finds and converts them too. The README lists the supported forms explicitly: local PDFs such as ~/Documents/report.pdf, remote PDFs such as an arxiv.org PDF URL, ordinary web pages, YouTube video URLs, and file:// URLs pointing at a PDF. So a note that is three lines of your own thinking plus a link to a paper ends up indexed as both.

The intended user is someone whose notes already function as a reference library, where the value is in the accumulated links rather than in any single note. Students collecting papers, researchers tracking sources, and anyone running the Tiago Forte style of note-taking the README cites. It is a worse fit for someone who wants a chat interface over a handful of short notes, because the conversion work happens on every link regardless of whether the link is worth converting.

The pipeline: Markdown to chunks to ChromaDB to an agent

The README gives the data flow as a short diagram: Markdown files from your editor, to text files from Markdown and pointers, to text chunks, to a vector database, to an AI agent. Each arrow is a separate stage, and the stages are visible as separate files in the repository root.

transform_md.py handles Markdown and its links. transform_txt.py handles plain text. lib.py holds shared code, and test_lib.py tests it. The vector store is ChromaDB, and the compose.yaml file runs a Chroma server rather than an embedded store. That is a meaningful choice: the database is a separate process with its own lifecycle, its own port, and its own volume, so notes can be reindexed without restarting the agent and the agent can be restarted without rebuilding the index.

Retrieval is served two ways. qa.py is a question answering entry point against the OpenAI model the README names. mcp_server.py is a Model Context Protocol server, described in the README as the way to pull relevant context from your notes and plug it into the LLM or workflow of your choice. The distinction matters: qa.py is this project answering your question, while the MCP server is this project handing context to something else that answers. The MCP path is the more durable design, because it does not tie your notes to one model vendor.

Two more root files, similarity.py and smart_connections.py, are not explained in the README text available here. Their names suggest similarity search and link suggestion, but the documentation does not describe them, so treat them as unexplored until you read the source.

Installing Second Brain AI agent and indexing a first folder

The project is not installed from a package index in any instruction shown here. The repository ships pyproject.toml, uv.lock, a Makefile and a set of shell scripts, which means you work from a clone and let the lockfile pin the dependency set. Python is constrained to >=3.10,<3.12 in pyproject.toml, so a 3.12 or 3.13 interpreter will not satisfy the requirement.

The Makefile drives the environment through uv. The uv.lock target syncs all extras, including the test group:

make uv.lock

Running that creates or refreshes the locked environment. The Makefile sets UV_CACHE_DIR to /tmp/uv-cache for its own invocations, so the cache lands in a temporary directory rather than your home directory.

The compose.yaml file defines the Chroma server. Note the comment in that file about where persistence lives, because it is the kind of detail that quietly loses data if you assume otherwise.

services:
  server:
    image: chroma-local:sort
    volumes:
      - $DSTDIR/Db:/chroma/chroma:z
    ports:
      - 8000:8000
    environment:
      - IS_PERSISTENT=TRUE
      - ALLOW_RESET=TRUE

The volume comment states that the local Chroma server persists under /chroma/chroma, not /data. Set DSTDIR to a real directory before starting it, or the mount will point somewhere unintended. The service listens on port 8000.

For the agent side, run-server.sh and mcp-server.sh are the entry scripts, and install-systemd-services.sh exists to register them as systemd units. example.env is the template for configuration; the README does not enumerate its keys, so read the file itself before assuming a variable name. The repository also ships monitor.sh and integration-test.sh, which suggests a workflow where the indexer and the server are supervised separately.

Where the documentation stops and you are on your own

The README is a pitch, not a manual. It explains what the project does and why that is appealing, then hands off. It does not document rollback, it does not describe what happens when a link fails to convert, and it does not explain how the index is invalidated when a note changes. The README says the system processes any change in these files automatically, which tells you a watcher exists but not how it decides what changed or what it does when a conversion errors out.

That gap is the real cost of adoption. A YouTube video with no transcript, a PDF behind a paywall, a web page that returns a consent interstitial: all of these are normal inputs for a link-following indexer, and none of their failure behaviour is described. The dependencies hint at the surface area. yt-dlp and youtube-transcript-api for video, pymupdf and unstructured for documents, assemblyai and pydub for audio transcription. Each of those is a place where a fetch can fail, and the README does not say whether a failure aborts the note, skips the link, or retries.

The other limitation is the release cadence. The most recent release listed is 0.6.0 from 2024-03-07, while pyproject.toml declares version 0.7.0. The last push to the repository was on 2026-04-05, so work has continued past the last tagged release, but anyone pinning to a release is pinning to something two years older than the current tree. If you need a versioned artifact, you are building from a commit.

How this differs from a general-purpose RAG framework

The obvious alternative is LangChain itself, plus a vector store, assembled by hand. This project is built on LangChain and ChromaDB, so the difference is not the underlying machinery. It is that the link-following step is already written. A hand-rolled LangChain pipeline over Markdown gives you chunking and embedding; it does not give you PDF extraction, YouTube transcript retrieval and web page scraping wired into the same loader. That is the part you would otherwise build and maintain yourself.

The second alternative is a hosted notes product with built-in AI search. Those handle the ingestion problem for you and remove the operational work entirely. The trade is that your notes live in their format and their infrastructure, and the retrieval is theirs. This project's value proposition is the opposite: your notes stay as files you own, and the retrieval is exposed over MCP so the consuming model is your choice. If you do not care about either of those properties, the hosted option is less work.

The third comparison is to a plain grep or ripgrep over the same folder. For exact strings, that is faster and needs no infrastructure. It fails on the case this project targets, which is asking a question whose answer is spread across a PDF and a video transcript you never read closely. Vector search earns its cost there and nowhere else.

Licence, maintenance and upgrade cost

The licence is GPL-3.0, declared both in the repository and in pyproject.toml. For personal note indexing this is unlikely to matter. If you intend to embed this in a product you distribute, the copyleft terms apply to the combined work, and that is a question for a lawyer rather than for this article.

The dependency list is long and includes fastapi, uvicorn, kubernetes, opentelemetry, sentence-transformers and onnxruntime. Several of those are heavy, and kubernetes in particular is unusual for a personal notes tool, which suggests the deployment target is a cluster rather than a laptop. uv.lock exists to pin the resolution, and the Makefile rewrites the Chroma image tag in compose.yaml from the frozen chromadb version, which is a deliberate coupling: upgrade chromadb and the compose file follows. That mechanism is the main upgrade safeguard, and it also means a chromadb bump changes your container image reference.

The maintenance signal is mixed. The repository is not archived, and the last push was on 2026-04-05. But the newest tagged release is 0.6.0 from 2024-03-07, while the declared version is 0.7.0, so releases lag the tree. Budget for reading source rather than release notes when something breaks.

Editorial conclusion

Adopt it if you already keep notes as Markdown files and want retrieval wired into an LLM client over MCP rather than a closed notes app. Skip it if you need Python 3.12 or newer, a pip-installable release, or a project whose README covers rollback and failure recovery, because this one does not. Before committing, check which Python your environment provides, confirm the ChromaDB version your lockfile resolves to, and read run-server.sh and mcp-server.sh to see what the scripts actually start.

Frequently asked questions

Official sources

1. flepied/second-brain-agent on GitHub
2. Issues
3. License: GPL-3.0
4. README
5. Releases