hiring-agent: HackerRank's Open-Source Resume Scoring Pipeline

What hiring-agent Is and What It Is Not

HackerRank receives between 50,000 and 60,000 intern applications each year. The README explains that no human team can read that many resumes well, so hiring-agent was built to rank them, helping decide which resumes to read first. Resumes below the cutoff score are filtered out, but the cutoff is set very low so only candidates at the very bottom of the distribution are excluded. Human reviewers make all real decisions downstream.

The README addresses misconceptions directly. This tool is not an Applicant Tracking System. It is not used to screen applications for HackerRank's own open roles. It is not a product available to HackerRank customers. Since its release, HackerRank has also shipped AI Interviewer (Chakra) for automated first-round interviews, so the pipeline is one layer of a larger system.

The default model shipped in the repository is gemma4:latest, which runs locally on most laptops without a cloud API key. The README states that actual intern resumes at HackerRank are evaluated with a top-tier Gemini model and that the repository ships a demo configuration, not the production one.

Architecture: Five Stages from PDF to Score

The pipeline runs five stages in sequence. First, pymupdf_rag.py converts PDF pages to Markdown-like text using PyMuPDF. Second, pdf.py calls the LLM per resume section using Jinja templates stored in the prompts/templates directory to extract structured JSON. Third, github.py fetches the candidate's GitHub profile and repositories, classifies projects, and asks the LLM to select the top seven. Fourth, evaluator.py runs a strict-scored evaluation with fairness constraints. Fifth, score.py orchestrates everything end to end and writes CSV output when development mode is on.

Key modules beyond those scripts include models.py (Pydantic schemas and LLM provider interfaces), llm_utils.py (provider initialization and response cleanup), transform.py (normalization from loose LLM JSON to JSON Resume format), and the prompts/ directory (all Jinja templates for extraction and scoring).

Two LLM backends are supported: Ollama for fully local execution and Google Gemini for cloud-based evaluation. The backend is configured through an environment variable and the providers.json file.

Installation and First Run

The repository requires Python 3.11. The .python-version file pins this to 3.11.13. Clone the repository, create a virtual environment, and install dependencies:

$ git clone https://github.com/interviewstreet/hiring-agent
$ cd hiring-agent

$ python -m venv .venv
# Linux or macOS
$ source .venv/bin/activate
# Windows
# .venv\Scripts\activate

$ pip install -r requirements.txt

For local LLM execution with Ollama, install Ollama from the official site and pull the default model:

$ ollama pull gemma4:latest

For better results on a higher-spec machine, pull a larger model:

# For higher system configuration
$ ollama pull gemma3:12b

Configure the environment by copying .env.example to .env and setting the default model. For Gemini, add a GEMINI_API_KEY. The .env.example file shows the structure:

DEFAULT_MODEL=gemma4:latest
GEMINI_API_KEY=your_gemini_api_key_here

The requirements.txt pins specific versions: PyMuPDF 1.26.3, pydantic 2.11.7, requests 2.33.0, pymupdf4llm 0.0.27, Jinja2 3.1.6, google-generativeai 0.4.0, and python-dotenv 1.2.2.

Known Limitations: Score Variance, Invisible Text, and Rubric Bias

Published analysis of hiring-agent has documented specific limitations that the README surfaces under a Coverage section.

One study found significant score variance across 100 runs of the same resume. The analysis identified technical skills categories as relatively stable while project quality judgments were noisy, pointing to LLM non-determinism as the root cause. A score that varies by 15 or more points across runs is not reliable enough for filtering without additional controls.

A separate analysis found a security issue: invisible text embedded in PDFs can inflate scores significantly. Resumes that carry hidden keyword text bypass the content-based scoring in the evaluator.

A third analysis argues that the GitHub-centric rubric disadvantages engineers whose work lives in private enterprise repositories. It also notes a signal degradation risk as candidates optimize for the now-public rubric.

Published recommendations for improving score reliability include standardized data formats, versioned evaluation models, ensemble scoring across multiple runs, and explainability layers. As of the last push on 2026-07-27, these improvements have not been merged into the repository.

Ethical and Legal Considerations

A Hacker News discussion of hiring-agent surfaced GDPR Article 22 concerns. Article 22 restricts automated individual decision-making that produces legal or similarly significant effects on natural persons. Filtering job candidates out of a hiring pipeline on the basis of an algorithm meets this threshold in many EU jurisdictions.

The README acknowledges the transparency argument: making the scoring logic public allows scrutiny that proprietary ATS systems never face. However, transparency does not resolve the legal compliance question for teams deploying this in the EU or in other jurisdictions with similar automated-decision regulations.

The README also notes that the cutoff is intentionally set very low, which it frames as a mitigation. The vast majority of applicants pass through to human review. Whether this framing satisfies regulatory requirements depends on the jurisdiction and deployment context.

Alternative Approaches to Resume Screening

Commercial ATS platforms such as Greenhouse, Lever, and Workday embed their own candidate scoring and filtering, with proprietary models and compliance controls built in. These are full products with support, audit trails, and legal review behind their filtering decisions. The trade-off is that their scoring logic is not inspectable.

hiring-agent's differentiating characteristic is that its full scoring rubric is public. Every Jinja template in prompts/ is readable, every weight in the scoring logic is visible in evaluator.py, and the community that has reviewed it has published detailed critiques. For teams that want to understand exactly what is being measured and why, that transparency is the primary argument for this tool over a black-box commercial system.

The tool also supports fully local execution via Ollama, which means resume data does not leave the organization's infrastructure when using a local model.

Maintenance Status and License

The repository is not archived. The last push was on 2026-07-27. The project is licensed under MIT, which permits use, modification, and redistribution in commercial and open-source projects.

The repository has no GitHub releases. Dependency versions are pinned in requirements.txt. The .python-version file pins Python to 3.11.13.

The README lists a community tool built on this repository: Resume Reality Check, a hosted tool that lets candidates score their own resume against the same rubric. This suggests the codebase is stable enough for third parties to build on.

Editorial conclusion

hiring-agent is useful for teams that receive large volumes of resumes and want to rank them for human review, not replace human reviewers entirely. The README is explicit that HackerRank uses it to rank intern applications for human reading order, with the cutoff intentionally set low so only the bottom of the distribution is filtered out. The tool is not an ATS, not available as a HackerRank product, and not the system used to evaluate HackerRank's own open roles. Before deploying it, verify that your team's legal context permits automated resume ranking, since published commentary has raised GDPR Article 22 implications. The default model ships as gemma4:latest for local use, not the production-grade Gemini model used internally at HackerRank.

Frequently asked questions

Official sources

1. interviewstreet/hiring-agent on GitHub
2. Issues
3. License: MIT
4. README