boss-zhipin-scraper: a Chrome CDP job scraper that reuses your real BOSS Zhipin login

What boss-zhipin-scraper actually solves

BOSS Zhipin renders salary figures in a font-obfuscated form on the page, so a DOM scraper reads glyphs that do not map to real digits. This project sidesteps that entirely: it calls the site's own search API through a browser you are already logged into, and the API returns a plaintext `salaryDesc` field. The README states this directly as the reason the tool exists, and it is the single design decision everything else follows from.

It is aimed at one person doing one kind of work: an engineer or analyst who wants a few pages of listings for a keyword and a city, with salary bands and repeated skill words extracted, and who is willing to keep a browser window open while it runs. It is not a fleet, not a scheduler, and not a hosted data product. The README caps `--pages` at 10, which tells you the intended scale better than any feature list does.

The second audience is agent users. The repository ships a `SKILL.md` and is listed on skills.sh, so it can be installed as an Agent Skill for Claude Code, Codex, Gemini CLI or Cursor and invoked in conversation rather than from a shell.

How the CDP session and the API call fit together

There is no Selenium and no Playwright. The tool speaks Chrome DevTools Protocol to a browser process it starts itself, using a persistent, isolated profile. That profile is separate from your main Chrome profile by default: the README notes that first launch does not copy your main login state, and that you sign in to zhipin.com inside the BOSS-specific browser window instead. `--copy-login-state` exists as a manual opt-in to import Local State and cookie files, and it is off by default.

Once the session is live, the script calls the zhipin.com search API and reads `salaryDesc` from the JSON response. Detail pages are fetched by default (`--detail` is on, `--no-detail` turns it off) to pull job descriptions for the skill analysis. Output is written incrementally, so the README claims an abnormal exit does not lose already-written rows.

There is a fallback path and it is deliberately fenced off. `--allow-dom-fallback` lets the tool extract from the DOM when the API returns nothing, and the parameter table warns that salary may then be untrustworthy. Leaving that flag off is the correct default; turning it on trades correctness for a non-empty result set.

Installing boss-zhipin-scraper and running a first scrape

The repository offers four install routes. The simplest for CLI use is a clone plus a dependency install. The two dependencies are `websocket-client` and `requests`, both pinned to major-version ranges in `requirements.txt`, and Python 3.10 or newer is required by `pyproject.toml`.

git clone https://github.com/eatmoreduck/boss-zhipin-scraper.git
cd boss-zhipin-scraper
pip install -r requirements.txt

If you prefer uv, the README lists `uv sync` as an alternative to the pip line, and the repository contains a `uv.lock`.

Next, start the dedicated browser and log in once. The setup step waits for the login to complete and, according to the README, confirms the API returns plaintext salary before finishing. Edge is available through `--setup-edge`.

python3 scripts/boss_cdp_raw.py --setup-chrome

Before spending a scrape, check the environment and optionally run a smoke test that hits the real search API without writing result files.

python3 scripts/boss_cdp_raw.py --check
python3 scripts/boss_cdp_raw.py --smoke-test

Then run the scrape. This example pulls three pages of listings for one keyword in Shanghai and requests the analysis report.

python3 scripts/boss_cdp_raw.py --keyword "AI Agent" --city 上海 --pages 3 --format csv --analysis

What you should see is a result file in the chosen format plus an analysis covering salary distribution, experience requirements and high-frequency skill words. If you want the aggregated summary and the prompt text, run the second script, which by default reads the most recent result.

python3 scripts/job_summary.py --top 15

City coverage is broader than the largest metros: the README says 300+ cities including smaller tiers, with codes synced at runtime and a local table in `data/city_codes.json`. You can list them with `--list-cities 江`. An unrecognised city name exits with an error rather than silently returning zero rows, which is the right call.

Where boss-zhipin-scraper breaks or is the wrong tool

The login state is the whole mechanism, and it is also the failure mode. If the dedicated profile's session expires, or the site challenges the account, the API call returns nothing useful and the run produces an empty or degraded result. The README's answer is `--reset-chrome-profile`, which rebuilds the profile and clears the login inside it, meaning you sign in again. There is no documented token refresh or unattended re-authentication path.

The scale ceiling is explicit: `--pages` tops out at 10. Anything resembling daily monitoring of many keywords across many cities is outside the design, and the README's disclaimer asks users not to put load on the target site. If your requirement is a recurring dataset, this tool is the wrong shape and you would be building a scheduler around something that was not meant to have one.

Platform confidence is uneven and the README says so. macOS auto-detects Google Chrome or Chromium. Windows is described as verified by unit tests and basic CLI checks, with a GBK console crash fixed, and the README explicitly invites feedback on the real scraping path there. That is an honest statement of a gap, and it means Windows users should treat the first real run as unverified.

Finally, the analysis has a boundary worth reading twice: the generated prompts are built only from job data. They do not read a local resume file and they do not score you against a posting. If you wanted a matching engine, this is not one.

How it differs from a Selenium or Playwright scraper

The README addresses this head-on in a collapsed section, and the comparison is real rather than marketing. Selenium and Playwright launch a fully controlled browser instance. That instance is large, its automation fingerprint is distinctive, and the README argues it is more likely to trigger risk controls and captchas on BOSS Zhipin.

This project inverts the relationship. It attaches to a browser you started and logged into, so the fingerprint and session are the ones the site already trusts, and it reads from the search API rather than scraping rendered DOM. The practical difference is where the fragility lives. A Playwright scraper fights the page: selectors, lazy loading, and the obfuscated salary font. This tool fights the session instead: if the login holds, the salary comes back as plain text with no decoding step.

That trade is not free. Playwright gives you a reproducible, headless, scriptable browser you can run in CI. A CDP attach to a persistent logged-in profile does not fit that model, and the README's setup flow assumes an interactive login. Choose Playwright when you need determinism and can accept DOM fragility; choose this when the login is the hard part and the API response is the prize.

Maintenance, upgrade cost and the MIT licence

The project is not archived, and the last push was on 2026-09-17. Version 2.2.0 is declared in both the README badge and `pyproject.toml`, and there is a `CHANGELOG.md` in the repository root, so version history is tracked rather than implied.

The dependency surface is small enough to audit in a sitting: two runtime libraries, both with upper bounds below the next major version. That is the good news for upgrade cost. The bad news is that the tool's correctness depends on an undocumented, private web API belonging to a third party. When BOSS Zhipin changes its search endpoint or response shape, no dependency bump fixes it; the script itself has to change. Budget for that rather than for library churn.

The licence is MIT, declared in `pyproject.toml` and present as a `LICENSE` file. MIT covers the code. It does not cover the data you collect, and the README carries a separate disclaimer pointing at BOSS Zhipin's user agreement and prohibiting commercial resale, malicious crawling, or behaviour that burdens the target site. Those are two different questions, and passing the first does not answer the second. This is not legal advice; if the output is going anywhere beyond personal research, that is a question for someone qualified to answer it.

Editorial conclusion

Adopt it if you are comfortable driving a dedicated logged-in Chrome profile and want plaintext salary fields plus a skill-frequency summary without a Selenium or Playwright stack. Skip it if you need unattended scraping at volume, a hosted service, or data whose collection you cannot justify under BOSS Zhipin's user agreement. Before trusting any output, run --check and --smoke-test, then confirm that a short --pages 1 run returns rows with a populated salaryDesc, because the README notes the DOM fallback path can produce salary values the project itself calls untrustworthy.

Frequently asked questions

Official sources

1. eatmoreduck/boss-zhipin-scraper on GitHub
2. Issues
3. License: MIT
4. Project website
5. README