# DeepGraph: an agent that finds questions and checks its own answers > DeepGraph is a Python research agent that harvests arXiv, merges claims into an evidence graph, finds open questions with SQL detectors, runs budgeted experiments and issues verdicts on an evidence ladder. It suits a computational researcher who wants hypotheses with checkable answers, and it is an alpha-stage system with a large operational surface. **billion-token-one-task/Deepgraph** — Token-scale scientific discovery engine — autonomous hypothesis generation, experiment execution, and knowledge graph synthesis - Repository: https://github.com/billion-token-one-task/Deepgraph - Website: https://deepgraph.joulebeat.com - Stars: 1,968 · Forks: 39 - Language: Python - License: MIT - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/billion-token-one-task-deepgraph ## Five steps from a paper harvest to a verdict DeepGraph is the autonomous research agent of JouleBeat, described as an open source RSI lab, and the README summarises its purpose in a line: it reads the literature, finds open questions, runs the experiments, and lets reality decide what is true. The mechanism is a five step pipeline, and the table is the clearest description of the system. Step one is Read, which harvests arXiv at scale and extracts claims, methods and results. Step two is Map, which merges them into an evidence graph of entities, relations and contradictions. Step three is Ask, where ten structural detectors find open questions in pure SQL, with no LLM in the path. Step four is Test, which turns a question into a pre-registered, budgeted experiment on CPU or GPU. Step five is Judge, where an evidence ladder issues a verdict only when the comparison is fair, and the result feeds back into what gets searched next. Two of those steps carry unusual weight. The question finding deliberately excludes a language model, which means the open questions come from structural properties of the graph rather than from a model guessing, and a question that can be checked is the precondition for the rest of the pipeline. And the Judge step is gated on fairness, so the agent is not allowed to declare a winner from a comparison that was not set up properly. The feedback closes the loop: the result updates what gets searched next, so the agent is doing a search rather than answering a list. ## The system that retracted its own conclusions The most informative paragraph in the README is the one about failure. Every verdict is auditable, and the system audits itself: in August 2026 it found and retracted 11 of its own conclusions that rested on empty model outputs. The README adds the observation that a search loop is only as good as the filter that decides what is true. That is a genuinely unusual thing to publish. A research agent that reports the number of conclusions it withdrew, and the reason, is telling you where its own weak point is, and the reason given is not subtle. Empty model outputs are the classic failure of a pipeline where a model is asked to produce structured data and quietly returns nothing; those empty results then flow downstream as if they were evidence, and the agent builds a conclusion on a hole. Finding eleven of them in one month and retracting them is the difference between a system that is aware of its own error rate and one that is not. The live production numbers for August 2026 are given in the same document, and they are worth reading as a scale indicator rather than as a performance claim. 24,407 papers harvested with 7,005 going through the full evidence pipeline. An evidence graph of 248,441 entities, 735,913 relations and 238 contradiction clusters. 25,652 mapped research opportunities. 150 audited verdicts across six research agendas. And 921 million LLM tokens invested. Put those together and the ratio is telling. Twenty-four thousand papers were harvested to produce 150 verdicts, which is what a pipeline with strict evidence gates looks like from the inside. ## The harness and benchmark loop behind the agent The Why DeepGraph section explains the project in terms of a bet about where progress comes from, and it is a different framing from most agent frameworks. JouleBeat builds self-improving AI by letting harnesses and benchmarks evolve together. The argument runs like this. How an agent works, its harness, meaning tools, memory, workflow and verification, now moves capability by as much as a model generation, and open source has made harness variants cheap to generate. What is still scarce is the benchmark that tells real progress from a lucky score. So both go in the loop: harnesses compete under benchmarks, and benchmarks compete on how well they predict results that arrive later. DeepGraph is positioned as the task source for that loop. Computational research is the right kind of work because a conclusion is confirmed or refuted by rerunnable code, held-out data or an independent recomputation, in minutes to days. The two audiences are named: researchers, who get a research agent that delivers conclusions they can check, and JouleBeat itself, for which every task is a judged trial for the next harness and the next benchmark. The stack table lists DeepGraph first, with the harness and benchmark libraries next and the Evolution Kernel below, which is the engine that proposes, evaluates, and accepts or rolls back, open source and installable with pip install evolution-kernel. The status column is worth reading honestly: DeepGraph is in service with paying research groups, harness versioning is running, and benchmark validation is in build. So the research agent is the product, and the self-improvement loop around it is the longer-term project. ## Running it locally, and the configuration surface The quick start is short, and the requirements are Python 3.12 or newer and an LLM API key, with PostgreSQL enabling the full control plane. ```bash python3.12 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt cp .env.example .env # set at least DEEPGRAPH_LLM_API_KEY export $(grep -v '^#' .env | xargs) python3.12 main.py # open http://localhost:8080 ``` That last line matters, because the local run opens a web interface on port 8080, which the manifest confirms with waitress in the dependencies and a console script named deepgraph that points at main:main. A fuller deployment with PostgreSQL, systemd and remote GPU backends is documented separately in docs/DEPLOY.md. The .env.example is where the operational surface shows. The defaults live in deepgraph.toml and the .env file holds machine-local overrides and secrets, which is a sensible split. The profile is set to machine_learning with the root node id ml, and the comment shows the alternative open_science profile with science as the root, so the tree the agent searches is a configuration choice rather than a fixed assumption. The LLM settings are more detailed than most projects expose. There is a base URL, an API key and a model, plus an optional extra providers list for OpenAI-compatible routes beyond the primary and secondary. Then there are stability knobs for flaky providers and gateways: connect and request timeouts, transient retry count, a backoff, and a cooldown. That last group of settings is a telling detail. A system that talks to multiple third party endpoints in a loop has learned what it means for a provider to go down mid-run, and the cooldown in particular suggests the failure mode was repeated rather than theoretical. The pipeline settings are the same story. Concurrency is 4 by default, arXiv results per query cap at 100, graph backfill on start is on, and merge candidates refresh on start is on. Paper clustering needs a minimum of 10 papers. There are optional single-process automation flags for the pipeline and for auto research, with batch size 100 and an interval of 120 seconds, and the comment is explicit that these are for when you are not already running scripts/run_pipeline_forever.sh or another external processor. There are GPU worker slots, visible devices, a default model name and a default VRAM figure in gigabytes, with a remote dual-card example given. ## An alpha package with a broad repository The package metadata is modest and honest about where the project is. The name is deepgraph, the version in the manifest is 0.2.0 while the tags are v0.3 and v0.2, the description is open scientific opportunity mapping from literature, evidence tables and plain-language domain briefs, and the classifier says Development Status :: 3 - Alpha. The runtime dependencies are five: flask, httpx, pydantic, pymupdf, waitress. The requirements file adds psycopg for PostgreSQL, matplotlib and numpy. Pymupdf is the PDF reader, which is what makes it possible to extract claims and results from paper text rather than only from abstracts, and the five package dependencies are a small surface for something that does this much. The repository itself is much broader than the package, and the top level list explains why. There are agents/, ingestion/, orchestrator/, db/, web/, plugins/, prompts/, research_agendas/, meta_harness/, ops/, deploy/ and compat/ directories, plus contracts/, scripts/, tests/, docs/ and third_party/. There are docker-compose.postgres.yml and docker-compose.grobid.yml, the latter for a document annotation service, which is a detail that tells you the ingestion path is not limited to arXiv metadata. There is also a web/ directory whose package data includes templates, static CSS and static JavaScript, so the interface is server-rendered rather than a separate front end application. The package.json is minimal and holds only Playwright for end to end tests, with a single script that runs a homepage hierarchy spec, and playwright.config.ts is at the top level. Other things worth noting: there is a CLA.md and a check_cla.py with a cla-signers.json, so contributions go through a contributor licence agreement, and the licence itself is MIT. The last push to the master branch was on 2026-10-16. ## Conclusion Adopt DeepGraph if your research question is computational, because a conclusion confirmed by rerunnable code is exactly the kind of task this agent is built to take on, and the SQL detectors mean the question finding does not depend on a model's judgement. Do not adopt it for a literature review with no experiment attached, since the whole pipeline is built around producing a verdict from a run. Before you start, read docs/RESULTS.md, because the README reports that the system retracted 11 of its own conclusions in August 2026, and start with the local profile rather than the broad open science tree, then check that your provider and model are configured in .env, since the default base URL points at a third party endpoint. ## FAQ ### What are the five stages of the DeepGraph pipeline? Read, which harvests arXiv and extracts claims, methods and results; Map, which merges them into an evidence graph of entities, relations and contradictions; Ask, where ten structural detectors find open questions in pure SQL with no LLM in the path; Test, which turns a question into a pre-registered, budgeted experiment; and Judge, where an evidence ladder issues a verdict only when the comparison is fair and the result updates what is searched next. ### How do I run DeepGraph locally? You need Python 3.12 or newer and an LLM API key. Create a virtual environment, run `pip install -r requirements.txt`, copy .env.example to .env and set at least DEEPGRAPH_LLM_API_KEY, export the variables, and run `python3.12 main.py`, which opens the interface on port 8080. PostgreSQL is needed for the full control plane, which is documented in docs/DEPLOY.md. ### Does DeepGraph have a track record of wrong conclusions? Yes, and the project reports it. The README states that in August 2026 the system found and retracted 11 of its own conclusions that rested on empty model outputs, and notes that a search loop is only as good as the filter that decides what is true. The measured results are published in docs/RESULTS.md, read directly from the production database. ### Which knowledge tree does DeepGraph search by default? The .env example sets DEEPGRAPH_PROFILE to machine_learning with DEEPGRAPH_ROOT_NODE_ID of ml, and the comments show the alternative open_science profile with science as the root node. The defaults live in deepgraph.toml, with .env holding machine-local overrides and secrets. ## Sources - [billion-token-one-task/Deepgraph on GitHub](https://github.com/billion-token-one-task/Deepgraph) - [License: MIT](https://github.com/billion-token-one-task/Deepgraph/blob/master/LICENSE) - [Project website](https://deepgraph.joulebeat.com) - [README](https://github.com/billion-token-one-task/Deepgraph/blob/master/README.md) - [Releases](https://github.com/billion-token-one-task/Deepgraph/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/billion-token-one-task-deepgraph