x-reader: a URL dispatcher that turns ten platforms into one content schema

The problem is not fetching, it is the ten different shapes that come back

Pulling a page down is easy. The annoying part is that a YouTube link, a WeChat article, a Telegram post and an RSS item each arrive with different metadata, different body extraction rules, and different failure modes, so anything downstream has to special-case all of them. x-reader's answer is a dispatcher: the README describes the flow as any URL going through platform detection, then fetch, then a unified output, with the Python layer handling text fetching and YouTube subtitle extraction. The target user is someone building an agent or a personal archive who wants one call and one return type rather than a folder of per-site scrapers. The README's own example is a one-liner: x-reader https://mp.weixin.qq.com/s/abc123, and the library form returns an object with .title and .content. That is the whole pitch, and it is a reasonable one, because the schema is the product here, not the HTTP requests.

Platform detection followed by a fallback chain per site

There is no single extraction engine. Each platform gets its own ordered chain, and the order tells you what the maintainers trust. For X, the README lists five steps: X oEmbed for fast public tweet text, FxTwitter for structured public fallback, Jina Reader for public Articles and long-form pages, generic Jina Reader for profiles and non-status pages, and finally Playwright with a saved session for login-required content. WeChat and Xiaohongshu follow a shorter version, Jina then Playwright. YouTube uses yt-dlp subtitles with a Groq Whisper fallback. Bilibili, Telegram and RSS use their own APIs or feedparser. The important structural point is that Jina Reader is a third-party hop for a large share of the text paths, and Playwright is the last resort for anything gated. That means the reliability of x-reader is partly the reliability of someone else's reader service, and partly the freshness of a browser session you saved locally. The README is explicit that the Python layer covers text fetching and YouTube subtitles, while full Whisper transcription for video and podcast, plus AI analysis, lives in the optional Claude Code skills, not in the pip package.

Install paths, and why the pip package is only one third of the project

The README recommends installing from GitHub rather than PyPI: pip install git+https://github.com/runesleo/x-reader.git. Optional extras are declared per capability, so Telegram support is pip install "x-reader[telegram] @ git+https://github.com/runesleo/x-reader.git", browser fallback is the [browser] extra followed by playwright install chromium, and everything together is the [all] extra plus the same Playwright step. Two components are deliberately excluded from pip: the Claude Code skills, which you copy with cp -r skills/video "$CLAUDE_SKILLS_DIR/video" and the same for skills/analyzer, and mcp_server.py, which requires cloning the repository and running pip install -e ".[mcp]" before python mcp_server.py. Video and audio need external binaries, brew install yt-dlp ffmpeg on macOS or pip install yt-dlp plus apt install ffmpeg on Linux. Whisper transcription needs GROQ_API_KEY, and the README points at Groq's free tier. Configuration is a .env file copied from .env.example, with TG_API_ID and TG_API_HASH for Telegram, INBOX_FILE defaulting to ./unified_inbox.json, and OUTPUT_DIR disabled by default. There is also OBSIDIAN_VAULT, which writes into 01-收集箱/x-reader-inbox.md inside the vault.

The MCP server is the smallest layer and the one with the version ceiling

Four tools are exposed: read_url(url), read_batch(urls), list_inbox() and detect_platform(url). The README states that read_batch fetches multiple URLs concurrently, and that the server currently targets FastMCP 1.x, with the mcp and all extras pinning mcp<2. The same paragraph says moving to MCP 2.x requires a server migration rather than removing the version cap. That is an unusually candid note and worth taking at face value: if your MCP client ecosystem moves to a 2.x-only transport or API, this server does not follow by editing a dependency line. Registration is a JSON block in ~/.claude/claude_desktop_config.json pointing command at python and args at the absolute path of mcp_server.py. Note the coupling: list_inbox() reads whatever INBOX_FILE points at, so the MCP surface and the CLI share state through a JSON file on disk rather than through a service. For a single-user agent setup that is fine. For anything multi-tenant it is a shared mutable file, and the README does not describe locking or a server mode for concurrent writers.

Where it breaks: gated content, third-party hops, and the login you have to remember

The failure modes are visible in the platform table. Xiaohongshu text fetch is marked as requiring a one-time login, x-reader login xhs, which saves a session for the Playwright fallback. X Articles and login-required X pages need the same treatment via x-reader login twitter. That means the tool is not fire-and-forget on the platforms people most often want: sessions expire, and when they do the chain has already exhausted oEmbed, FxTwitter and Jina before it reaches the browser. The README also documents a privacy switch that deserves attention. By default local X cookies stay local, but setting X_READER_ALLOW_EXTERNAL_SESSION_COOKIES=1 lets Jina use your saved X session for gated Articles. That is an explicit opt-in and the default is the safe direction, but it is a real boundary you should decide about rather than inherit. Finally, the transcription story is split: YouTube gets yt-dlp subtitles and a Groq Whisper fallback inside the Python layer, while Xiaoyuzhou, Apple Podcasts and Bilibili video transcription are listed as available via the Claude Code skill. If you are not running Claude Code, that half of the advertised coverage is not reachable through the library or the MCP server.

Compared with wiring up Jina Reader and yt-dlp yourself

The honest alternative is not another reader product, it is doing the dispatch yourself. Jina Reader already handles arbitrary web pages, and x-reader uses it as the generic fallback for any web page and as a step in the WeChat, Xiaohongshu and X chains. yt-dlp already handles subtitles, and x-reader calls it for YouTube. So the delta x-reader provides is the detection layer, the per-platform ordering, the unified schema, the inbox file, and the MCP and Claude Code wrappers. If you only ever read one platform, that delta is small and you should call the underlying tool directly. If you read five or more, the ordering logic is the part you would otherwise reimplement badly, and the README's X chain is a decent worked example of how many fallbacks a single platform actually needs. The trade-off is that you inherit a dependency on Jina Reader's behaviour and on Playwright session state, and you cannot easily swap the reader backend without touching the package. The README does not describe a pluggable extractor interface, so treat the chain as fixed rather than configurable.

Maintenance cost and what the MIT licence actually covers

The release history shows v0.2.2 in May 2026 and v0.2.1 in March 2026, the latter labelled an ARM64 compatibility fix, with the repository's last push in August 2026. That is a low-cadence project with occasional platform-compatibility patches, which fits the shape of the problem: scrapers break when sites change, and the fixes arrive as point releases. Budget for upgrading when a platform changes, and expect to re-run x-reader login for XHS or X when a session goes stale, since that state lives on your machine and not in the package. On licensing, the project is MIT, which permits commercial use and modification provided the copyright notice and permission notice are retained; the README does not add terms of its own. What MIT does not settle is the terms of the sites being read. The README documents reading WeChat, Xiaohongshu and X content through Jina, Playwright and saved sessions, and it does not discuss platform terms of service, rate limits or robots directives. That question is yours to answer, and this is not legal advice. The GROQ_API_KEY, TG_API_ID and TG_API_HASH you supply are third-party credentials with their own quotas, and nothing in the repository mediates them.

Editorial conclusion

Adopt x-reader if you are already running Claude Code or an MCP client and want a single read_url tool covering YouTube, Bilibili, X, WeChat, Telegram and RSS without writing six scrapers. Do not adopt it if you need a supported scraping service with a stability contract: the X and WeChat paths fall through to Jina Reader and then to Playwright with a locally saved session, and the README itself notes that the MCP server targets FastMCP 1.x and that moving to MCP 2.x requires a server migration rather than removing the version cap. Before committing, run x-reader login twitter and x-reader login xhs against the specific gated URLs you care about, and confirm whether X_READER_ALLOW_EXTERNAL_SESSION_COOKIES should stay unset for your threat model.

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. runesleo/x-reader on GitHub