DeepGit: an agentic GitHub search layer that spends LLM calls only when it is unsure

The problem DeepGit targets: search that loses your intent

GitHub search is a keyword interface. You type a few tokens, you get results ranked largely by signals like stars and recency, and the parts of your request that do not fit into tokens are discarded. A query like "a fast, embeddable key-value store in Go" carries at least three separate constraints, and the search box has no place to put any of them. The README frames this directly, describing DeepGit as keeping "the full intent" instead of collapsing a question "into a handful of keywords". The intended user is an engineer who already knows roughly what they want and is tired of scrolling a ranked list to find it. The secondary user is an AI assistant, since the project ships an MCP server so that Claude Desktop, Cursor or VS Code can call DeepGit as a tool. That second use case is the more distinctive one. A human can tolerate a mediocre search and refine it. An assistant calling a tool mid-conversation needs the first call to return something usable, which is why the project's emphasis on hard constraints (a C library must not win a C++ query) matters more than its ranking polish.

The adaptive pipeline: cheap first, escalate on contested picks

The mechanism is a staged escalation rather than a fixed chain. The first LLM call converts the request into a plan containing search angles, GitHub topics, must-haves, nice-to-haves and anti-patterns. Gathering then happens at zero token cost: batched GraphQL keyword queries plus star-sorted topic sweeps, which is how the project claims to reach flagship repositories whose descriptions omit the obvious words. The README names apache/kafka as the example. Everything seen during gathering is upserted into a local vector index built on LanceDB and fastembed, running CPU-only, and near-neighbours from that accumulated corpus are pulled back in. The project describes this as recall that compounds across searches. A prefilter then uses free signals (activity, health, licence, tests and CI, relevance) to cut the field to a shortlist without spending tokens. The second LLM call is a batched ranking over compact cards containing README text and those signals. Only after that does the confidence gate fire. If the top picks are contested, DeepGit reads real source, tests and manifests, re-judges from that code, and a reflection critic reviews the final order. The README states a typical query costs two LLM calls. The architectural bet is that most queries are easy and do not deserve code reading, and that a cheap uncertainty check is a better use of budget than uniformly deep analysis.

Installing it and wiring up the two credentials it needs

DeepGit requires Python 3.11 or newer and a GitHub token. The README's install path is a clone, a virtual environment, and pip install -e ".[semantic]", with the semantic extra marked optional but recommended. Dropping that extra removes LanceDB and fastembed, and the project states that keyword plus topic search still runs without it. You lose the compounding recall, which is the feature the project leans on hardest, so installing without it is a meaningfully different product. Configuration goes through a .env file or environment variables. GITHUB_API_KEY takes a PAT with public-repo read access. LLM_PROVIDER accepts groq, openai, anthropic or vertex_ai, with the corresponding key (GROQ_API_KEY is the example given), or VERTEX_PROJECT plus GOOGLE_APPLICATION_CREDENTIALS for Vertex AI. Two credentials, two failure surfaces. The CLI exposes deepgit search "..." and deepgit mcp, and the library exposes search for a markdown report and search_structured for a result object, where the README shows result.top.repo and result.top.fit_score. There is a Dockerfile that bundles both the CLI and the deepgit-mcp server and starts the MCP server over HTTP on port 8080 by default, run with docker run --rm -p 8080:8080 -e GITHUB_API_KEY=ghp_xxx deepgit. The README's Docker section is truncated mid-command, so the one-off CLI invocation inside a container is not fully documented in the supplied material.

The benchmark is reproducible but small, and you should treat it that way

The project ships a benchmark suite under bench/ and reports, for a held-out set of twelve unseen queries with the adaptive controller on, Hit@1 of 75 percent, Hit@3 of 92 percent, MRR of 0.833, zero wrong picks at rank one, and an average of 2.0 LLM calls per query. Two things deserve attention. First, twelve queries is a small sample. A single changed outcome moves Hit@1 by more than eight points, so the gap between 75 percent and something notably worse is one query wide. Second, the held-out set was constructed by the same people who built the system, and the README does not describe how queries were selected or how the correct answer for each was determined. That is not an accusation of bad faith, it is a statement about what the number can support. The useful part is that the suite is runnable: python -m bench.run_bench --heldout for the unseen set, python -m bench.run_bench for the core set, and python -m bench.run_bench --edge for robustness cases. Each run writes a per-case report to _bench_report.md. The per-case file is worth more than the summary table, because the summary tells you the average and the report tells you which kinds of query fail.

Where the adaptive gate can go wrong

The confidence gate is the load-bearing component, and it is also the least specified in the material. The README says a "zero-cost confidence gate decides whether the fast answer is trustworthy" and that escalation happens only when top picks are contested, but it does not state what signal the gate reads, what threshold it compares against, or whether that threshold is configurable. That matters because the failure mode is asymmetric. If the gate is too permissive, DeepGit returns a confident-looking ranking built from README text alone for a query that actually needed code reading, and the user has no way to tell that the escalation step was skipped. If it is too conservative, cost climbs toward the hard-query case and the two-call average stops meaning anything. There is a second limitation in the semantic recall. The index is local and described as compounding, which means it accumulates whatever the agent has seen. A local index that grows with use has no documented eviction, no documented way to inspect or reset it, and no documented behaviour when the same repository is re-embedded after its description changes. This is a reasonable design for a personal tool and a questionable one for a shared deployment, where one user's accumulated corpus would silently shape another user's results.

How it differs from searching GitHub directly or from a research agent

The obvious alternative is the GitHub search UI or the GitHub GraphQL API used directly. The difference is not the data source, since DeepGit reads the same public repositories through a batched GraphQL client. The difference is that a direct search gives you a ranked list you must interpret yourself, while DeepGit attempts to interpret your request first and returns a judgement with a fit score attached. That trade is real but not free: you are substituting an LLM's reading of README files and signals for your own reading of repository names and descriptions, and you inherit the LLM's errors. The other comparison is a general research agent with web access. Those agents can search GitHub as one source among many, and they can read documentation sites, blog posts and issue threads that DeepGit does not appear to touch. DeepGit's narrower scope is the point: it constrains the problem to repository selection, which is why it can prefilter on CI status and licence and enforce a language constraint. If your question is "which library should I use and why", a general agent with web access has more evidence available. If your question is "which repository matches this specific brief", the narrower tool has the better-shaped pipeline. Neither is a superset of the other.

Licence, maintenance and the cost of staying current

DeepGit is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. This is a description of the licence text, not legal advice; if you plan to redistribute it inside a product, have someone qualified read the actual LICENSE file. On maintenance, the repository is not archived and the last push recorded is 2026-08-30, but no releases were retrieved, so there is no tagged version to pin. Installing from a clone of main means your build tracks whatever lands on the default branch. The dependency surface is where ongoing cost sits. The semantic extra pulls in LanceDB and fastembed, the pipeline uses LangChain and LangGraph according to the repository topics, and the LLM provider is one of four external services. Any of those moving can break an install, and with no releases there is no version boundary to fall back to. The benchmark suite is the mitigation: because bench/ is in the repository and writes a per-case report, you can re-run it after a dependency bump and see whether behaviour shifted before you trust the new state. That is the practical upgrade procedure this project supports, and it is the reason the benchmark folder is more than decoration.

Editorial conclusion

Adopt DeepGit if you already run an MCP-capable assistant and want repository discovery driven by a written brief rather than by keyword guessing, and if you are comfortable supplying both a GitHub PAT and an LLM provider key. Do not adopt it if you need a supported product with a release history, or if you cannot accept that the headline accuracy figures come from a twelve-query held-out set the project authors chose. Before trusting it, run python -m bench.run_bench --heldout on your own machine and read the per-case report it writes to _bench_report.md, because that file is the only place you will see which queries the agent got wrong and how confident it was when it did.

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. zamalali/DeepGit on GitHub