rag_api: an ID-scoped RAG service for LibreChat deployments

The file_id keying decision and who it serves

Most retrieval stacks index a corpus and then filter results by whatever metadata the application attached. rag_api inverts that. Files are organized into embeddings by file_id, and the README states the main reason is to work with embeddings at file level so that queries can be targeted when combined with file metadata held elsewhere. The chunks themselves are the unit of ownership, not the document row in some external table.

That design assumes a host application already knows which files a user may see. LibreChat is the named primary use case, and the homepage points at librechat.ai, so the intended deployment is a companion service rather than a general-purpose vector search product. If you have no upstream system that maps users to files, the ID scheme gives you nothing: you would be building the metadata layer anyway, and a plain pgvector table with your own columns would be less indirection.

Owner scoping lives in one module, not per route

The retrieval scope section of the README describes the current authorization model in unusual detail. Every route that reads or removes stored content resolves the caller's owner set from the verified token and puts it into the store query before ranking, so a chunk outside that set is never read into the process. The owner set is built in app/scope.py rather than re-derived per route.

Doing the filter before ranking matters. A post-filter approach retrieves top-k vectors and then discards unauthorized ones, which both leaks timing information and silently reduces the number of usable results. Pushing the owner predicate into the store query avoids both, at the cost of every query carrying the scope predicate.

The README also documents what the previous behaviour was, which is the clearest signal of what this release is for. GET /ids listed every file id in the deployment. POST /query_multiple performed no authorization at all, so combining it with GET /ids disclosed the content of every file to any authenticated caller. POST /query authorized the whole result set from documents[0], so any hit behind the first was never checked, and since a file_id is chosen by whoever uploads, an attacker's own row ranking first authorized the rows behind it. GET /documents, GET /documents/{id}/context and DELETE /documents read or deleted chunks of any file id the caller could name. A chunk with no recorded user_id read as belonging to everyone. On the synchronous store path, a failed ingestion rolled back by file_id alone, so an upload under someone else's file id destroyed their chunks.

One behavioural change is worth internalizing: a file id outside the caller's scope answers not found rather than found but refused, so none of these routes is an existence oracle. That is the right choice for disclosure, and it creates the orphan problem described below.

Running it: compose files, environment variables and uvicorn

The README gives two paths. For Docker, docker compose up starts both the API and a PSQL/pgvector instance; docker compose -f ./db-compose.yaml up starts only the database, and docker compose -f ./api-compose.yaml up starts only the API. That split is useful when the database already exists elsewhere.

For a local run, the README says to configure a .env file based on the environment variables section, set DB_HOST to the correct database hostname, then install and start:

pip install -r requirements.txt uvicorn main:app

A clean reinstall section covers removing existing dependencies after requirements.txt changes, though the supplied excerpt cuts off mid-command. The README does not enumerate the environment variables in the portion available here, so treat the .env keys as something to read from the repository rather than from this article. JWT_SECRET is named in the retrieval scope discussion as the signing key that establishes caller identity, and its absence has consequences described below.

The upgrade order is one-directional and the failure is silent

Deleting entity-owned files requires entity_id. Chunks embedded under an entity_id, an agent knowledge base for instance, are owned by that entity rather than by the uploading user, so DELETE /documents needs the same entity_id the upload used, passed as a query parameter alongside the JSON body of file ids. A delete that omits it resolves to the caller's own scope, matches nothing, and answers 404 with the chunks left in place. Because a 404 is indistinguishable from already deleted, a caller that treats it as success will orphan those chunks silently.

Deploy order matters in one direction only, and the README is explicit. A client that sends entity_id against an older build is inert, because the parameter is undeclared there and the request behaves as before. An older client against this build orphans every agent knowledge-base file it tries to delete. So upgrade the client first, or both together, never this service first. LibreChat carries the matching change and records the owner each embed was made under, with npm run migrate:embed-owners to backfill files embedded before that.

Two data migrations precede the upgrade. Chunks with no user_id are owned by nobody and are no longer readable; the README supplies an UPDATE against langchain_pg_embedding using jsonb_set on cmetadata to stamp an owner. If the deployment ever ran without JWT_SECRET, every chunk written in that period is owned by the literal string public, and the README gives a SELECT count(*) to inspect those rows before deciding whether to rewrite them. Deployments that never set JWT_SECRET are unaffected, because the read scope is public as well. Atlas MongoDB deployments must add user_id to the vector search index first.

entity_id is still caller-asserted, and that is the open hole

This is the limitation to weigh before adopting. Agent knowledge bases are owned by an agent id rather than a user id, so a caller reading one names it via entity_id. That id now widens the owner set rather than replacing the caller's identity, and the caller's own scope always remains. But nothing in a token minted today proves the caller may act for the entity it names. A caller that knows another owner's id can still name it, on read to reach that owner's chunks, and on the ingestion routes, where entity_id is what gets stamped as the owner, to write into that owner's namespace.

The README states the position plainly: deployments exposing this API to untrusted callers must continue to authorize entity access upstream, and closing the gap requires the token to carry entity authorization, which is a coordinated change with the callers that mint tokens and is tracked separately. Read that as a boundary, not a roadmap item. If your users can reach this service directly, the owner scoping fixes the file-level leaks but does not make the service safe on its own.

Where a plain Langchain pgvector store differs

The obvious alternative is using Langchain's PGVector store directly inside the application, with no HTTP hop. The difference is not the vector math, which is the same pgvector underneath, but where the ownership predicate lives. With a direct store, your application builds the filter and passes it to the retriever, and the application's own request context is the authority. With rag_api, the service derives the owner set from a verified token in app/scope.py, and the application must mint tokens that carry identity and send entity_id where relevant.

That trade is real in both directions. A direct store keeps authorization in one language and one transaction boundary, and it eliminates the 404-means-orphaned failure mode entirely, because a delete either matches rows or your code knows why. rag_api buys you a separate deployable that LibreChat can point at, a single place where the scope predicate is written, and the ability to change embedding models or vector stores behind an HTTP interface. The README says the API will evolve to employ different querying and re-ranking methods, embedding models, and vector stores, which is the stated reason for the extra hop. If you are not running LibreChat and do not expect to swap stores, that reason is thin.

What to verify before you commit

Check the release notes for v0.9.0 rather than trusting a summary, because the retrieval scope changes are the substance of that release and the README describes them as a break in behaviour for callers. Confirm which of GET /ids, POST /query_multiple, POST /query, GET /documents, GET /documents/{id}/context and DELETE /documents your clients actually call, since each had a different defect and each now behaves differently.

Run the two inspection queries against your database before upgrading, not after. The NULL user_id count tells you how much content will become unreadable, and the public count tells you whether the deployment ever ran without JWT_SECRET. Both are cheap SELECTs and both change your migration plan.

Finally, verify that whatever mints your tokens can carry entity authorization, or accept that entity access remains an upstream responsibility. The licence is MIT, which permits commercial use and modification and requires the copyright notice and permission notice to be preserved; the repository does not appear to ship a separate commercial or support tier, so maintenance cost is whatever you spend tracking releases like v0.9.0 that change authorization semantics. Nothing in the supplied material states a support commitment, a release cadence beyond the three releases listed, or a compatibility policy for the HTTP contract.

Editorial conclusion

Adopt rag_api if you are running LibreChat or a comparable application that already stores file metadata and can mint identity-carrying tokens, and if you accept that entity authorization stays upstream. Do not adopt it as a standalone multi-tenant retrieval API exposed to untrusted callers, because entity_id is still caller-asserted. Before upgrading, check for chunks whose user_id is NULL or the literal string public, and upgrade the client before the service.

Official sources

1. danny-avila/rag_api on GitHub
2. License: MIT
3. Project website
4. README
5. Releases