QMedia: a self-hosted multimodal RAG stack split into three services
An open-source AI content search engine designed specifically for content creators. Supports extraction of text, images, and short videos. Allows full local deployment (web app, RAG server, LLM server). Supports multi-modal RAG content Q&A.
At a glance
- What is it?
- QMedia is an MIT-licensed content search engine for creators, built as a Next.js web app plus separate Python RAG and model services. The split is the interesting part, and also the source of most of its operational cost.
- Who is it for?
- Adopt QMedia if you want a self-hosted, MIT-licensed starting point for multimodal RAG over your own images, text and short video, and you are willing to run a Node front end alongside two Python services. Do not adopt it if you need a stable released artifact or a single-process deployment: the repository shows no published releases, so you would be tracking the main branch.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 160 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem QMedia targets: scattered creator material with no query layer
The README frames the project around a specific audience and a specific kind of mess. QMedia is described as an AI content search engine designed for content creators, and the stated goal is to integrate unstructured text, image and short video information into a multimodal RAG question and answer system. The word unstructured is doing real work there. A creator's own archive is usually a folder tree of screenshots, saved posts, clips and notes with no consistent metadata, and ordinary full-text search cannot reach inside an image or a video at all.
The pitch is therefore not general enterprise search. It is closer to a private research desk over your own reference material: pull a query, get back content cards that show where the answer came from and how the underlying image, text or clip was broken down. The README explicitly lists content sources and the decomposition of image/text and short video information as part of the output, which is a claim about provenance rather than about answer quality. That distinction matters when you evaluate it. If what you need is a chatbot that sounds confident, this is not aimed at you. If what you need is a way to find the clip you half remember and see why it matched, the card-plus-source design is the right shape.
Three services, three languages: how the pieces divide
The architecture visible in the README is a deliberate split rather than a monolith. The web service is described as inspired by the XHS web version and implemented with TypeScript, Next.js, TailwindCSS and Shadcn/UI. The RAG search and Q&A service, and the image/text/video model service, are described as implemented with the Python framework and LlamaIndex applications. The installation section names the three units concretely: mm_server, mmrag_server and qmedia_web.
The README states that these three can be deployed separately, so you can size each one to the resources you have, and that they can be embedded into other systems for image/text and video content extraction. That last point is the more consequential one. It means the model service is not just an internal dependency of the web UI; it is presented as something you could call from your own pipeline. The data flow implied by the feature list runs from raw material through extraction (OCR on images, transcription on video, style and layout analysis) into an index that the RAG layer retrieves from, with the web app as the presentation and query surface. The README also notes support for Google content search, which sits alongside the local index rather than replacing it.
What the README does not give is a schema, an API contract between the services, or a diagram of what is stored where. The file structure section is introduced but the supplied README text is truncated at that point, so the actual directory layout for the three services cannot be confirmed from this material.
The local model stack: what runs where, and what is still unchecked
QMedia separates the model layer from the RAG application layer, and the README gives a reason: it makes different models easy to replace. It also mentions local model lifecycle management, configurable for manual or automatic release to reduce server load. That is a real engineering concern, because holding a 70B parameter model resident alongside an embedding model and a Whisper instance is not the same as holding one small model.
The named components are specific. Language models run through local Ollama switching, with llama3:8b-instruct-q4_0 listed as a lightweight local option and llama3:70b-instruct listed as the larger one. Image embedding uses the CLIP encoder, described as converting images to text feature encoding. Text embedding uses the BGE encoder, described as a multilingual embedded model with local models aligned to a GPT encoder. Image text OCR recognition uses QAnything, referenced at a specific tag, v1.4.0-python. Video transcription uses Faster Whisper, which the README notes can run on local CPU.
Read the checkboxes before you plan around this list. llava-llama3, described as an Ollama-hosted visual understanding model, is an unchecked item. So are identification of highlights in short videos, recognition of short video style types, and analysis and breakdown of short video content. Visual understanding of images beyond OCR is therefore not something the README claims is finished. If your use case depends on asking questions about what is depicted in an image rather than what text it contains, the current documented capability is OCR plus CLIP embedding, and the richer path is listed as future work.
Getting it running: the install units and the config you have to supply
The README's installation section is organised around the three services, with separate subsections for mm_server installation, mmrag_server installation and qmedia_web installation. It also describes a combined usage mode and an independent model service mode, plus a pure local multimodal path. The supplied text is truncated inside the file structure introduction, so the actual commands, dependency files and environment variable names for each service are not present in the material available here. Anyone evaluating QMedia should read the full installation section in the repository rather than rely on a summary, because the three-service layout means three separate setup paths.
What can be said from the README is the shape of the work. The web service is a Next.js application, so it follows the usual Node toolchain. The other two are Python services built on LlamaIndex, and the model service expects Ollama to be present locally for the language models. The README does not enumerate configuration keys in the text provided, so no specific config key can be quoted here without inventing it. Treat the model selection and the manual-versus-automatic model release behaviour as the two settings you will most likely need to tune, since the README calls out lifecycle management as a configurable feature and server load as the reason for it.
Where QMedia is the wrong tool
The clearest limitation is release discipline. No releases were retrieved for this repository, and the README links to a changelog rather than versioned artifacts. That means adoption is a main-branch commitment. There is no pinned version to test against, no upgrade boundary to reason about, and no way to know from this material whether the three services are guaranteed to be mutually compatible at any given commit. For a personal research tool that is a tolerable trade. For anything with an uptime expectation, it is the first thing to resolve.
The second limitation is operational shape. Three services in two languages means three sets of dependencies, and the Python side carries LlamaIndex plus OCR, embedding and transcription components. The README's own framing of separate deployment is presented as flexibility, and it is, but the cost is that a failure in the model service surfaces in the web app as a degraded or failed query with no single log to read. The manual-or-automatic model release option exists precisely because keeping models resident is expensive, which tells you the default posture is not a small footprint.
The third is scope. This is a content search and Q&A system over creator material. It is not a document management system, not a general enterprise knowledge base, and the README makes no claims about access control, multi-tenancy or audit. Local deployment is offered as a way to keep private data offline, which is a deployment property, not an authentication model.
How it differs from a general-purpose RAG framework
The obvious comparison is LlamaIndex itself, since QMedia is built on it. The difference is that LlamaIndex is a library you assemble into your own application, while QMedia is an assembled application with opinions already made: a card-based web UI, a fixed set of encoders (CLIP for images, BGE for text), OCR through QAnything, transcription through Faster Whisper, and Ollama for generation. Choosing LlamaIndex directly gives you freedom over every one of those choices and none of the UI, indexing conventions or service boundaries. Choosing QMedia gives you a working shape to modify, at the cost of inheriting its model choices and its three-process topology.
A second comparison point is the short video handling. General RAG tooling typically treats video as an audio transcription problem and stops. QMedia's feature list goes further in intent, naming video summaries, transcription and, as future work, highlight identification and style recognition. At the time of writing, transcription and LLM-based summarisation are the documented pieces; the rest are unchecked. That gap between the roadmap and the shipped list is worth weighing honestly, because a video-heavy workflow is exactly where the unfinished items matter most.
Maintenance cost and what the MIT licence does and does not settle
The repository is MIT licensed and not archived, with the last push recorded as 2026-04-09. MIT is permissive: it allows commercial use, modification and redistribution provided the copyright notice and licence text are retained. It does not grant trademark rights, and it provides no warranty. It also does not resolve the licensing of the components QMedia depends on. The README points at QAnything, Faster Whisper, CLIP, BGE, Ollama and llama3 models, and each of those carries its own licence terms, including model weights that may have separate usage conditions from the surrounding code. If you plan to ship something built on QMedia, the dependency licences are a separate review from the MIT badge on this repository. Nothing here is legal advice.
On maintenance, the practical cost is tracking three services against a moving main branch with no releases. The README's own future plans list four unchecked items, including similar image/text/video search and card content generation, so the surface area is expanding rather than frozen. Budget for reading the changelog before pulling, and for the possibility that a model swap on the Python side changes retrieval behaviour in ways the web app will not flag.
Editorial conclusion
Adopt QMedia if you want a self-hosted, MIT-licensed starting point for multimodal RAG over your own images, text and short video, and you are willing to run a Node front end alongside two Python services. Do not adopt it if you need a stable released artifact or a single-process deployment: the repository shows no published releases, so you would be tracking the main branch. Verify three things first: that the mm_server, mmrag_server and qmedia_web install steps in the README still match the current file layout, which Ollama models your hardware can actually hold resident, and whether the unfinished items in the feature list (llava-llama3 visual understanding, highlight detection, short video style recognition) sit on a path you need.
Community notes