Hysen Labsarcana
Embeddable RAG library for Elixir/Phoenix with agentic pipelines and dashboard
Arcana brings retrieval augmented generation inside an Elixir app
An embeddable RAG library that stores documents, embeddings, and a knowledge graph in your existing Postgres database.
Where the data lives
Arcana is built for teams that already run an Elixir or Phoenix application with an Ecto repository. Instead of standing up a separate vector database, an indexing service, and an orchestration layer, the library keeps documents, their chunks, the embedding vectors, and a knowledge graph as ordinary tables in the Postgres database the app already uses. The authors argue that the BEAM runtime is a good fit for this work because pgvector is mature, supervision trees match the shape of long running embedders and rerankers, and telemetry is part of the platform. The result is one process model with no extra infrastructure to operate.
Three ways to use it
The library maps onto three recognized RAG patterns. Arcana.search/2 and Arcana.ask/2 are the default door and represent advanced RAG, already doing query rewriting, hybrid search, optional graph fusion, and cross encoder reranking in a single call. Arcana.Pipeline.* represents modular RAG, letting you compose explicit steps such as gate, rewrite, expand, decompose, search, reason, rerank, answer, and ground, each implemented as a behaviour you can replace. Arcana.Loop.* represents agentic RAG, where the language model chooses tools like search, answer, or give_up on each turn. The project deliberately ships no naive one shot mode.
Getting started and context
The shortest useful program ingests a string with Arcana.ingest/2 and asks a question with Arcana.ask/2, passing the repo and an LLM identifier such as openai:gpt-4o-mini. Embeddings run locally by default through Bumblebee with EXLA, EMLX on Apple Silicon, or Torchx, and the cross encoder reranker is also local, though you can swap in OpenAI, Cohere, or another provider. A runnable Livebook tutorial is linked from the README, and a companion app called arcana-adept ships with the Doctor Who corpus pre ingested for experimentation. The code is Apache-2.0 licensed and the project shows 330 stars.
Editorial conclusion
Arcana is published under the Apache-2.0 license and reports 330 stars. Its documentation points to a Livebook tutorial and a prebuilt companion app, arcana-adept, that demonstrates the library against the Doctor Who corpus.
Community notes