PixelRAG: Retrieval Over Screenshot Tiles Instead of Parsed Text
https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
At a glance
- What is it?
- PixelRAG is a Python pipeline that renders documents to image tiles and embeds them with a LoRA-tuned Qwen3-VL-Embedding model, so tables, charts and layout survive retrieval. The design is coherent, but the README leaves the training stage and the index build largely undocumented.
- Who is it for?
- Adopt PixelRAG if your corpus is visually dense (tables, charts, infographics, multi-column PDFs) and you can absorb a GPU embedding stage plus a local Playwright render. Do not adopt it if your documents are plain prose, if you need a fully documented training path, or if you cannot run a browser on the ingest host.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 2 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The table that text RAG throws away
The README states the problem in one line: text-based RAG parses a page to text chunks and loses the table, so the reader cannot find the answer. PixelRAG renders the page to screenshot tiles instead, retrieves the right tile, and the reader model reads the number off the image. That is the whole thesis, and it is narrow enough to be testable. The project is aimed at engineers building retrieval over documents where the answer lives in visual structure: financial tables, chart annotations, infographic callouts, layout-dependent comparisons. If your corpus is prose, the rendering step buys you nothing and costs you storage and latency. The README says the pipeline is general-purpose and that Wikipedia's 8.28M articles ship as a pre-built index, but the general-purpose claim is asserted rather than demonstrated; the only concrete corpus named is Wikipedia.
Two components: a renderer and a fine-tuned embedding model
The README names the two pieces explicitly. First, rendering documents to images instead of parsing them to text. Second, a Qwen3-VL-Embedding model, LoRA-fine-tuned on screenshot data, that embeds page images into a space where visual content is retrievable. The second piece is the part that is hard to substitute. A generic vision embedding model will happily produce vectors for screenshots, but the README's claim is that screenshot-specific LoRA fine-tuning is what makes visual content retrievable rather than merely embedded. The repository does not, in the supplied material, document the training data, the LoRA rank, or the training recipe. The pipeline diagram in the README ends with the fragment 'train → serve', which implies a training stage exists, but no command for it appears in the stage table. Treat the fine-tuned checkpoint as a given artifact, not something you can reproduce from the README alone.
Capture, chunk, embed, index, serve: the stage split
The README lays out the pipeline as separate installable stages, which is the most useful part of the documentation. Capture is the standalone pixelshot command. The rest runs through the pixelrag umbrella as pixelrag <stage>. The table gives this mapping: pixelshot turns a document into image tiles via Playwright CDP and PDF handling, installed with pip install pixelrag. The chunk, embed and build-index stages turn tiles into vectors into a FAISS index, installed with pip install 'pixelrag[embed]'. The index stage orchestrates the full pipeline from source through ingest, embed and index, installed with pip install 'pixelrag[index]'. The serve stage exposes a FAISS search API over FastAPI, on CPU or GPU, installed with pip install 'pixelrag[serve]'. The README shows serve as independent of the index arrow chain, which is the right call: you can rebuild an index without restarting the API. The extras split means you can install the capture path on a laptop and the embed path on a GPU box, but it also means a partial install fails at a stage boundary rather than at import time, and the README does not describe what those failures look like.
Getting pixelshot onto PATH, and the Claude Code plugin
The README gives a specific installation warning worth repeating: a plain pip install into a project venv may leave pixelshot off PATH, so it recommends uv tool install pixelrag or pipx install pixelrag to keep the CLI isolated yet available. That is a real constraint, not a style preference, because the Claude Code plugin shells out to the binary. The plugin install is two commands: claude plugin marketplace add StarTrail-org/PixelRAG, then claude plugin install pixelbrowse@pixelrag-plugins. After that, claude -p "screenshot https://news.ycombinator.com and summarize the top stories" works, or the /screenshot slash command in an interactive session. The README is explicit that there is no MCP server and no backend: the skill calls pixelshot (Playwright/CDP) on your machine. For opencode users, the same tool arrives via the @startrail/pixelbrowse npm package, added to opencode.json under the plugin key with the schema at opencode.ai/config.json. The render command itself is pixelshot https://en.wikipedia.org/wiki/Python --output ./tiles.
The hosted index and what it commits you to
The README advertises a live hosted endpoint at api.pixelrag.ai serving a pre-built index of 8.28M Wikipedia pages, with no setup and no API key. The example is a POST to https://api.pixelrag.ai/search with a JSON body containing queries as a list of objects with a text field and an n_docs field set to 5. The README also says the endpoint accepts an image as the query, which is the visual-search path, and points to pixelrag.ai/docs for the API reference. This is a genuinely low-friction way to evaluate whether visual retrieval answers your questions before you install anything. It is also a scope limit: the hosted index is Wikipedia. If your documents are internal, the hosted endpoint tells you nothing about your corpus, and you are back to running the embed and build-index stages yourself. The README does not state retention or logging behaviour for queries sent to the hosted API, so do not send anything sensitive to it.
Where the documentation is thin
Several things a person would need before adopting this are absent from the supplied material. There is no documented configuration file format, no list of environment variables, and no description of how chunking decisions are made for tiles (tile size, overlap, how a tall page is split). The embed stage's model download, memory footprint and batch behaviour are not described. The README's pipeline diagram ends with 'train → serve' but no training command is given, so the fine-tuning path is effectively closed. The stage table does not say what happens when the index is rebuilt while serve is running, or whether the FAISS index format is stable across versions. The release history shows v0.3.0 in June 2026, v0.4.0 in July 2026, and a patched headless Chrome 150.0.7844.0 in June 2026, which suggests the browser dependency is versioned alongside the package rather than left to the system. That is sensible, but it means an upgrade can move your rendering engine underneath you.
The wrong tool, and what to use instead
PixelRAG is the wrong tool when your documents are already well-structured text and your questions are answered by lexical or dense text retrieval. Rendering every page to tiles multiplies storage, adds a browser to your ingest path, and inserts a vision model between the query and the answer. If your corpus is Markdown, source code, or clean HTML with semantic markup, a text chunker plus a text embedding model will be cheaper to run and easier to debug, and the visual structure you would be preserving does not exist. The concrete alternative in that case is the standard text-RAG stack: parse to text, chunk, embed with a text model, index in a vector store. The difference in approach is not a matter of quality but of what the index represents. A text index represents tokens; PixelRAG's index represents rendered pixels, which is strictly more information per document and strictly more expensive to produce. Choose the pixel representation only when the answer is in the pixels.
Licence, maintenance and upgrade cost
The repository is Apache-2.0, which permits commercial use and modification and includes a patent grant. The README does not describe the licence of the fine-tuned Qwen3-VL-Embedding checkpoint or of the LoRA weights, so if you plan to redistribute the model rather than just call it, check the model card and the base model's terms separately. The patched headless Chrome 150.0.7844.0 release is a maintenance signal: browser rendering is the fragile part of any screenshot pipeline, and pinning a patched build means the project has taken on the job of tracking Chromium. Upgrades therefore carry two kinds of risk at once, Python package changes and rendering changes, and the README does not offer a compatibility matrix. The last push recorded is 2026-09-09, so the project is active, but activity is not the same as a stable interface. Pin your pixelrag version in the same lockfile as your index, and rebuild the index when you move the embed stage, because the README gives no guarantee that vectors from one model revision are comparable to another.
Editorial conclusion
Adopt PixelRAG if your corpus is visually dense (tables, charts, infographics, multi-column PDFs) and you can absorb a GPU embedding stage plus a local Playwright render. Do not adopt it if your documents are plain prose, if you need a fully documented training path, or if you cannot run a browser on the ingest host. Before committing, verify three things: whether the hosted api.pixelrag.ai index is acceptable for your data, what the actual embedding throughput is on your hardware, and how pixelshot behaves on the pages you care about. The repository pins a patched headless Chrome 150.0.7844.0, so browser version is a dependency you inherit rather than a detail you control.
Community notes