feedgrab normalizes eighteen platforms into Obsidian Markdown, one fallback chain at a time

One command in, a Markdown file with metadata out

feedgrab is a Python tool for fetching, normalizing and digesting content from self media and content platforms, and its entry point is a URL. The core loop is deliberately simple: any URL becomes a platform detection step, then a fetch, then a unified output written to a file. The output layout is documented as output/X/author_date:title.md, so the file name carries the author, the date and the title, and the content itself is Markdown with complete metadata that is compatible with Obsidian. The declared support is 18 or more mainstream platforms, and the project states that the core fetching path prefers free interfaces, so in most cases you do not have to buy a paid API and the running cost is close to zero. The commands are short, and the first use is a single URL.

feedgrab https://mp.weixin.qq.com/s/abc123
feedgrab clip
feedgrab https://x.com/elonmusk/status/123456
feedgrab x-retweeters https://x.com/ai_xiaomu/status/2051099012288356592
feedgrab x-so "AI Agent" --people

The Python layer is responsible for text fetching and for YouTube subtitle extraction. The optional Claude Code skill adds Whisper transcription of video and podcast content along with AI content analysis, and the optional MCP server exposes the same reading capability as tools.

Three installable layers that compose rather than conflict

The project is organised as three layers you can combine as needed. The Python CLI and library is the base layer, providing the basic content fetching plus a unified data structure, and it is the only required piece. The Claude Code skill adds video transcription, AI analysis and content fetching, and is installed with npx skills add iBigQiang/feedgrab, which is the form the documentation gives. The MCP server exposes the reading capability as MCP tools and is run with python mcp_server.py. The split matters because the cost and complexity jump sharply between them. The base layer is a normal Python package with a small dependency set. The skill layer assumes an agent runtime. The MCP layer assumes a client that speaks the Model Context Protocol, which in practice means an AI assistant that can call your tools. Installing the base layer and then deciding later whether you want the other two is a sensible order, since the packages directory also holds a separate desktop build for Windows.

The X fallback chain is the real architecture

The pipeline diagram in the README lists the per platform mechanisms, and the most interesting line is the one for X and Twitter: GraphQL, then FxTwitter, then Syndication, then oEmbed, then Jina, then Playwright. That is six retrieval routes for one platform, tried in order, because X is the platform where the easy paths break. Text is handled by the Jina Reader, video by the InnerTube API with yt-dlp for subtitles, and audio by Whisper transcription. The remaining platforms go through their own APIs directly, namely Bilibili, RSS, Telegram, the YouTube Data API v3, the GitHub REST API, the Feishu Open API and the Discourse Topic JSON format. So the design is a table of platform adapters over a shared normalisation layer, with the X adapter carrying most of the defensive complexity. If you are evaluating this for a single platform, the per platform adapter is the part that matters, and the adapters are not equally deep.

Private content needs your own session

Several feedgrab commands operate on content that only a logged-in account can see, and the documentation is specific about which routes those are. X bookmarks require X_BOOKMARKS_ENABLED=true and can only be read with the primary account session, and the legacy bookmark URL is still recognised alongside the current one. Batch fetching of a user's tweets needs X_USER_TWEETS_ENABLED=true, and the list route needs X_LIST_TWEETS_ENABLED=true, with X_LIST_TWEETS_DAYS controlling how many days back it goes and X_LIST_TWEETS_SUMMARY=true producing a summary table in Markdown and CSV. Likes can only be fetched for your own account, and retweeters and favoriters are read as separate list routes. The pattern is consistent: feedgrab automates the browsing you could have done yourself, but it cannot invent access. Login state comes from feedgrab login twitter, which opens a browser and saves a session, or from a cookie you paste in, or from a session file in the sessions directory.

Environment configuration decides where your data lands

The .env.example file is where the project exposes its behaviour switches, and the output settings are the first thing to set. OUTPUT_DIR sets the Markdown output root, and OBSIDIAN_VAULT points at a note vault, in which case content is written directly into the vault and takes priority over OUTPUT_DIR. FEEDGRAB_DATA_DIR sets where cookies and sessions are stored, relative to the project root or absolute, and the default is sessions. Proxying is controlled by FEEDGRAB_PROXY_ENABLED, with FEEDGRAB_PROXY_URL accepting an HTTP or socks5 URL and FEEDGRAB_NO_PROXY holding a comma separated list of hosts that should bypass it, with localhost and the local CDP port included by default. The browser user agent is unified across HTTP requests and browser interaction so that login and fetching agree, and feedgrab detect-ua reads the real Chrome user agent from the local machine and writes it into the file, which is the recommended path over the built in default. Logging is set with FEEDGRAB_LOG_LEVEL, which defaults to INFO.

Browser automation and the stealth extra define the cost of a fetch

Playwright is listed as a browser automation dependency, and the project also ships a stealth extra built on patchright, browserforge and curl_cffi. That combination is the anti-detection layer, and it tells you something about the shape of the problem. A fetch that only reads public pages does not need a stealth browser. A fetch of a logged-in feed, where the page renders through client side JavaScript and the session is protected, does. The optional dependency groups map onto this. browser pulls in playwright, stealth pulls in the detection-avoiding stack, telegram pulls in telethon for the Telegram user client, twitter pulls in XClientTransaction and beautifulsoup4, wechat pulls in markdownify and beautifulsoup4, xhs pulls in xhshow, feishu pulls in lark-oapi, and mcp pulls in the mcp package with its CLI extra. There is an all extra that installs every one of them. The Chrome CDP route is an alternative to the browser login flow: feedgrab login extracts cookies from a running Chrome over the remote debugging port, with the port defaulting to 9222, which avoids a separate login step but requires Chrome to be open and already signed in.

A 0.x project with a fast release train and a desktop build

The versioning tells you how much weight to put on stability. The package metadata lists version 0.26.2 with a description of a universal content reader, requires Python 3.10 or newer, and the README badge refers to v0.25.0. The tagged releases are named for the desktop build rather than the library, and three of them are listed: desktop-v0.1.23-20260831 from 2026-08-31, desktop-v0.1.22-20260831 from the same day, and desktop-v0.1.21-20260831 from 2026-08-30. Three desktop builds in two days says the project ships often. The last push to the main branch was on 2026-09-08 and the repository is not archived, so the code is being worked on. The licence is MIT, and the repository also carries an AGENTS.md, a CLAUDE.md, a DEVLOG.md and a tasks/ directory, which is the shape of a project developed with heavy agent assistance and a written record of each change. The Python package is built with hatchling and exposes a single console script, feedgrab, mapped to feedgrab.cli:main.

Editorial conclusion

feedgrab suits researchers and note takers who need to pull articles, tweets, videos and chat content into a personal Markdown vault without paying for platform APIs, and it suits people who want the same extraction available as a Claude Code skill or an MCP server. It does not suit anyone who needs a stable commercial pipeline, since the project is versioned 0.x and moves quickly, and it does not suit you if you cannot log in to the platforms whose private content you need, since several routes require your own session cookie. Before adopting it, run feedgrab on a public article URL to confirm the text path works, read the .env.example for the output directory and Obsidian vault settings, and confirm you are comfortable storing platform cookies in the sessions directory.

Frequently asked questions

Official sources

1. iBigQiang/feedgrab on GitHub
2. Issues
3. License: MIT
4. README
5. Releases