Arcana: Embeddable RAG for Elixir, No Separate Vector Service Required

Why Arcana Exists and Who Should Care

Most RAG libraries are Python-first and assume you will talk to them over HTTP. That means running a vector database you do not otherwise need, maintaining two languages, and stitching telemetry together across process boundaries. Arcana rejects that model. It is an embeddable library for Elixir and Phoenix that puts everything inside your existing application: documents, chunks, embeddings, and the knowledge graph all live as tables in your current Postgres database. The README explicitly cites pgvector, supervision trees, and built-in telemetry as reasons the BEAM fits RAG well. This is for Phoenix teams that already have an Ecto repo and want retrieval augmented generation without adding a second service. If you are starting a greenfield project in another language, or if you are not already invested in Elixir, this library solves a problem you do not have.

Three Usage Modes: Advanced, Modular, and Agentic

Arcana maps its surfaces to a 2025 survey by Singh et al. that classifies RAG into naive, advanced, modular, and agentic patterns. The key axis is who decides the control flow: the author at code time or the LLM at runtime. Arcana ships three shapes. `Arcana.search/2` and `Arcana.ask/2` give you Advanced RAG with one call, including query rewriting, hybrid search, optional graph fusion, and cross-encoder reranking. `Arcana.Pipeline.*` gives you Modular RAG, where you compose steps like gate, rewrite, expand, decompose, search, reason, rerank, answer, and ground. Each step is a behaviour with a single callback. `Arcana.Loop.*` hands control to the LLM, which picks tools such as search, answer, and give_up each turn. The README is explicit that Arcana intentionally does not ship a naive mode. Even the simplest entry point already does rewriting, reranking, and graph fusion when available. That is a deliberate trade-off: you get better defaults, but you cannot strip them down to a bare embed-retrieve-generate flow.

Local-First Embeddings and the One Process Model

Arcana runs embeddings locally by default using Bumblebee with EXLA, EMLX on Apple Silicon, or Torchx. The cross-encoder reranker is also local. You can swap to OpenAI or Cohere, but you do not have to. Embedders and rerankers are `Nx.Serving` instances under your supervision tree. Telemetry events are `:telemetry` spans you can already consume. The README stresses that there is no separate RAG service to operate. This is a meaningful architectural decision. It means your Phoenix app's supervision tree now includes long-running model servers. That is fine on a BEAM VM, but it changes your deployment footprint: you need enough memory and CPU to run Bumblebee in production, or you need to configure a remote API. The pluggable design uses behaviours with a single callback and a sensible default. Custom rerankers, searchers, and answerers are described as 10-line modules. The README also notes that `Arcana.search/2` and `Arcana.Loop` both call into `Arcana.Searcher`, so a custom searcher works across all surfaces.

Getting Started: Commands and Configuration from the README

The shortest useful program in the README shows ingestion and a query. You call `Arcana.ingest/2` with text and a repo, then `Arcana.ask/2` with a question, the same repo, and an LLM specifier like `"openai:gpt-4o-mini"`. The README gives this example: `{:ok, _doc} = Arcana.ingest("Phoenix LiveView is a server-rendered UI library...", repo: MyApp.Repo)` and then `{:ok, answer} = Arcana.ask("What is Phoenix LiveView?", repo: MyApp.Repo, llm: "openai:gpt-4o-mini")`. For the Pipeline surface, you build a context with `Pipeline.new/2`, then pipe through steps like `Pipeline.rewrite()`, `Pipeline.select(collections: ["elixir", "erlang"])`, `Pipeline.decompose()`, `Pipeline.search()`, `Pipeline.rerank()`, and `Pipeline.answer()`. For the Loop, you call `Arcana.Loop.new/2` with a question, repo, and collection, then `Arcana.Loop.run/2` with a `controller_llm`. The README also shows a router/answerer split where a cheap model like `"zai:glm-4.5-flash"` picks tools and a stronger model like `"zai:glm-4.6"` writes the final answer. There is a Livebook tutorial linked in the README, which is the practical way to try it without writing a full Phoenix app from scratch.

The Loop's Lock vs Pick Semantics and Grounding

The Loop surface has a specific constraint that is easy to miss. When your corpus is split into multiple collections, you pass them all to `new/2` and the controller LLM picks which collection to search on each turn. When you pass a single collection, the loop is locked to it. The README says the controller physically cannot express anything else because the tool schema will not even include the parameter. This is a strong design choice: it prevents the model from hallucinating a collection name, but it also means you cannot dynamically add collections at runtime without changing the call. The Loop also supports optional grounding. After the loop finishes, you can call `Arcana.Loop.ground(ctx)` to get a faithfulness score between 0.0 and 1.0. The context includes `tool_history`, which records which tools the LLM picked in order, and `terminated_by`, which can be `:answered`, `:gave_up`, `:max_iterations`, or `:error`. That visibility into the agent's reasoning path is useful for debugging, but it also means you need to inspect these fields to understand why an answer is weak.

A Real Limitation: It Is Not a Standalone Service

The biggest limitation is also the core premise: everything lives inside your app. That is a benefit for deployment simplicity, but it becomes a wrong tool when you need to serve RAG to multiple applications or scale retrieval independently from your web app. If you have a separate worker fleet or a mobile backend that needs vector search without a Phoenix frontend, Arcana forces you to run a Phoenix app just to host the library. Also, the README does not mention any built-in support for vector stores other than Postgres with pgvector. If you are already running Elasticsearch or Qdrant, Arcana will not replace them. The local-first default also carries a real cost: Bumblebee models are not tiny, and running them on every application node could be wasteful. The README says you can swap to OpenAI or Cohere, but that configuration is not shown in the truncated material, so you would need to check the HexDocs for exact keys. The repository was last pushed on 2026-08-28 with v4.0.0, so it is actively maintained, but the v3 to v4 jump happened within a week, which suggests breaking changes may be frequent.

Alternatives and the Difference in Approach

The obvious alternative is a Python RAG framework like LlamaIndex or LangChain, often paired with a vector database such as Pinecone or Weaviate. The difference is architectural. Those tools run as a separate service, communicate over HTTP, and require you to manage a vector database and an orchestration layer. Arcana embeds everything into your Elixir process tree and uses Postgres as the vector store. The README's argument is that the BEAM's supervision tree and telemetry make this natural. Another alternative is to use pgvector directly with a hand-rolled Elixir module, skipping Arcana entirely. That gives you full control but no query rewriting, reranking, graph fusion, or agentic loop. Arcana's value is that it packages those steps into behaviours with defaults. The trade-off is that you inherit Arcana's opinionated flow, including the intentional absence of a naive mode. If you want a minimal embed-retrieve-generate pipeline with no rewriting, you cannot get it from Arcana without writing custom behaviours that skip those steps.

Maintenance, License, and Upgrade Considerations

Arcana is licensed under Apache-2.0, which permits commercial use, modification, and distribution with attribution, but this is not legal advice. The repository is not archived, and the latest release is v4.0.0 from 2026-08-28, preceded by v3.0.1 and v3.0.0 within the same month. That rapid release cadence suggests active development, but it also means you should pin versions and read the changelog before upgrading. The README references a companion project, arcana-adept, which is a complete Phoenix app with a pre-ingested Doctor Who corpus. That is a useful reference for seeing how the library behaves in a full application, but it is a separate repository. The library depends on Nx, Bumblebee, and EXLA or EMLX, which are heavy dependencies that will affect your compile time and release size. The documentation lives at hexdocs.pm/arcana, and there is a Livebook tutorial, so the maintenance cost is mitigated by official examples. Before adopting, verify that your Elixir and Phoenix versions are compatible with v4.0.0, since the README does not list explicit version requirements in the truncated material.

Editorial conclusion

Adopt Arcana if you run a Phoenix app with an existing Postgres repo and want RAG without new infrastructure, and if you are comfortable with Elixir's Nx ecosystem and local model defaults. Skip it if you need a standalone service, if your corpus demands a dedicated vector index beyond pgvector, or if your team cannot commit to the BEAM-specific operational model. Before adopting, verify that your Postgres version supports pgvector, that your deployment can run Bumblebee or a configured remote embedding API, and that the telemetry spans match your existing monitoring. Read the Loop guide's lock vs pick semantics carefully, because the controller's tool schema changes based on whether you pass one collection or many. The v4.0.0 release from 2026-08-28 is current, but check the changelog for breaking changes if you upgrade from v3.x.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes