DeepXiv SDK: an agentic reader over pre-parsed arXiv papers

The gap between ten blue links and a 50k-token PDF

The README frames the problem narrowly and correctly: an agent researching a topic has two bad options. A search API returns abstracts, which is enough to name a paper and never enough to answer a specific question such as what speedup a method reports on HumanEval. Downloading the PDF answers that question, but the README puts the cost at roughly 50k tokens per paper and describes the result as a wall of text with no structure to navigate. Neither option lets an agent decide cheaply whether a paper is worth reading. DeepXiv's answer is to pre-parse papers into sections on the service side, so retrieval runs over full bodies rather than abstracts and the client can request one section at a time. The intended user is not a human browsing arXiv. It is a coding agent or a script that needs a cited answer, and the repository ships skills/deepxiv-cli/SKILL.md described as drop-in operating instructions for coding agents, plus two worked workflows. If you are a person who wants to read a paper end to end, the layered commands are overhead you do not need.

Layered reading is the actual mechanism

The cost ladder is the design, not a feature bolted on. The README states each step costs an order of magnitude more than the last, and the command sequence makes that concrete. deepxiv paper 2512.15176 --brief returns title, TLDR, keywords and citations, which is the cheap filter. --head returns the section list plus where the tokens are, which matters because the README warns that papers do not share a common outline, so section names have to be read off --head rather than assumed. --section Experiment then fetches only that section. --preview gives about 10k characters and no flag at all gives the full markdown. This is a deliberate inversion of how most paper tooling works: instead of fetching a document and extracting from it locally, the client asks the server for a named slice of an already-parsed document. The trade-off is that section boundaries are the service's, not yours. A paper whose relevant content sits under Related Work or Appendix will not be found by guessing a section name, and the README's own instruction to take names from --head is an admission that the outline is not predictable across the corpus.

What the agentic endpoint does with a question

deepxiv ask takes a question rather than a keyword. According to the README, the service picks its own tools, reads paper bodies, and returns an answer with citations that resolve to real arXiv IDs and URLs. The example output shows a specific claim (a 5.54x speedup on HumanEval) attributed to a named paper with its arXiv ID, followed by a source count line noting one cited and ten retrieved by default, with --all-sources to see the rest. Two details in that output are worth flagging. First, the retrieved-versus-cited gap means the service is doing its own filtering, and you are trusting that filter; the flag exists precisely because the default hides what was discarded. Second, the answer goes to stdout and sources to stderr, so deepxiv ask "..." > answer.md captures the prose without the citations. That is convenient for piping and mildly annoying if you wanted both in one file. --effort high is documented for questions that span several papers, which implies the default effort level is tuned for single-paper questions. The Python equivalent is reader.agent_search(...)["answer"], and Reader takes the token explicitly as a constructor argument rather than reading the saved config.

Getting a key, and why ask needs one when search might not

Installation is a single command, pip install deepxiv-sdk. The README states that deepxiv auto-registers a token on first use, but agentic commands (ask and talent) need a registered key instead, obtained from data.rag.ac.cn/register and then stored with deepxiv config --token YOUR_REGISTERED_KEY. Every account gets 300 agentic calls per day free, on a pool the README describes as separate from the general daily limit. That two-tier arrangement is the first thing to check against your own traffic: a script that asks one question per paper across a hundred papers a day is fine, and a batch job over a thousand is not. The beta caveat is explicit. deepxiv talent is not on PyPI yet and ships in 1.1.0b1 from source, installed with pip install git+https://github.com/DeepXiv/deepxiv_sdk.git, while the scholar index is still being built out. So the PyPI package and the git install are not the same artifact, and anyone who needs talent has to accept a source install of a pre-release. No releases were retrieved for this review, so there is no changelog to consult on how the two tracks will converge.

Search filters combine with AND, which fails quietly

deepxiv search "speculative decoding" --date-from 2025-01 --min-citations 20 --limit 10 shows the filter set: --authors, --orgs, --categories, --venue and --venue-year, dates, and citation floors. The README is unusually candid about the failure mode here. Filters combine with AND, and stacking too many will legitimately return zero results, at which point the instruction is to loosen one. This is a real usability trap for agent callers, because an empty result set is indistinguishable from a query that matched nothing for a substantive reason. A human sees zero results and adjusts. A script that treats an empty list as "no prior work exists" will draw a wrong conclusion and never surface an error. There is no documented signal distinguishing "your filters were too tight" from "nothing matches", and that is the sharpest limitation in the whole interface. If you are building anything automated on top of search, budget for a fallback that drops the citation floor or the date bound and retries before you report an empty result upward.

Talent search and the web backend step outside arXiv

deepxiv talent answers a different question: who works on a topic, where, and what their record is. The flow is two-stage, with deepxiv talent search returning IDs that feed deepxiv talent survey, which produces a full profile covering bio, education, work, open source and metrics, with a --format markdown variant for the generated report. Semantic mode takes a sentence, and dropping --semantic switches to matching names and affiliations directly, which is the difference between describing a research area and looking up a person you already know. The web backend is the other escape hatch: deepxiv ask with --web runs the same question over Google plus cached page bodies, and --search-type news narrows it. The README's honesty about this path is the useful part. Pages read in full are marked with a document symbol and snippet-only ones with a link symbol, and it says to weigh them accordingly. A cached snippet is not the same evidence as a full page body, and the output marks the difference rather than flattening it. The README also notes PubMed Central, bioRxiv/medRxiv, trending papers and per-paper social metrics as available sources, with details in USAGE.md rather than in the README itself.

Where a local retrieval stack is the better fit

The alternative is building the pipeline yourself: fetch PDFs from arXiv, parse them, chunk them, embed them, and run your own retrieval and generation. The difference in approach is where the parsing and the corpus live. DeepXiv does parsing server-side and exposes named sections plus an agentic endpoint, so your client stays thin and you inherit whatever the service has already indexed. A self-hosted stack puts parsing on your machine, which means you control the chunking strategy, you can point it at any corpus including internal documents, and you can run it without a network call or a daily quota. You also own the failure modes: PDF extraction of two-column papers, tables and equations is the part that consumes the most engineering time, and it is exactly the part DeepXiv has already done. The honest split is that DeepXiv wins when your corpus is the open literature it already covers and your questions are the kind its agentic endpoint handles. A local stack wins when the corpus is private, when you need reproducible retrieval you can pin and diff, or when per-call cost and daily caps are unacceptable. The two are not mutually exclusive, but mixing them means maintaining two retrieval paths with different recall characteristics, which is its own cost.

Licence, maintenance and what to verify before adopting

The repository is MIT licensed, which permits commercial and closed-source use of the SDK code and requires only that the licence and copyright notice travel with copies. That covers the client. It does not cover the hosted service behind data.rag.ac.cn, and the README says nothing about terms for the API, rate limits beyond the 300 daily agentic calls, or what happens to a token if the service changes. Treat the SDK licence and the service terms as two separate questions. On maintenance, the last push recorded is 2026-09-04 and no releases were retrieved, so there is no version history to inspect for breaking changes between the PyPI package and the 1.1.0b1 source install that carries talent. The repository does point to a technical report on arXiv and asks to be cited, which suggests the authors intend this to be a citable artifact rather than a throwaway script, but a citation request is not a support commitment. The concrete things to verify before you depend on it: whether the token auto-registration path works in a headless CI environment where no interactive first use happens, whether reader.section returns a usable error when a section name does not exist in a given paper, and whether the 300-call pool resets on a boundary that matches your job schedule.

Editorial conclusion

Adopt DeepXiv if your workload is question-driven reading of arXiv literature and you can live with a hosted dependency: install with pip install deepxiv-sdk, register a key, and run deepxiv config --token YOUR_REGISTERED_KEY before any ask or talent call. Do not adopt it if you need offline operation, a stable versioned API surface, or coverage of publishers outside the sources the README lists. Before committing, verify three things yourself: that the 300 agentic calls/day quota matches your traffic, that --head section names line up with the papers you care about, and that the talent command installs from source at 1.1.0b1 rather than from PyPI.

Official sources

1. DeepXiv/deepxiv_sdk on GitHub
2. Issues
3. License: MIT
4. Project website
5. README