x-tweet-fetcher: Reading X Without an API Key, and Where That Breaks
Fetch X/Twitter tweets, replies, timelines, and articles without login or API keys — field tool for AI agents.
At a glance
- What is it?
- The project routes each X/Twitter request to one of three backends and normalizes the results into a single JSON shape. Its value is in the routing and the error codes; its weak point is that timelines and search depend on a Nitter instance you host yourself.
- Who is it for?
- Adopt it if you are building an agent or an OSINT pipeline that needs tweet text in a stable JSON shape and you are willing to run a Nitter container next to it; the fxtwitter path alone justifies the install for single-tweet lookups. Do not adopt it if you need guaranteed uptime on timeline or search calls and cannot host Nitter, because the README states public instances are unreliable and frequently dead.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 9 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap xtf fills: X has no free API and scraping gets blocked
The README opens with a short exchange between a user and an assistant, and the assistant's line is the problem statement: it cannot access X/Twitter, so the user should copy-paste the content manually. That is the failure mode this project targets. An agent that can reason about a tweet but cannot retrieve one is not much use in a research or monitoring loop, and the alternatives named in the README are all closed off: X has no free API, scraping gets you blocked, and browser automation is fragile in headless environments. The intended user is therefore not a data engineer building a large ingestion system. It is someone wiring an AI agent to a retrieval tool, or an OSINT analyst who wants a tweet, a timeline or an article as structured text without registering an application with X. The output is JSON by default, with `--text-only` for humans, which tells you which audience the authors had in mind. The exit codes reinforce that reading: `0` for success or no new mentions, `1` for an error or new mentions found, `2` for a monitor setup error. Those are cron semantics, not interactive-tool semantics.
Backend routing: fxtwitter, Nitter and a browser driver behind one interface
The mechanism is routing rather than scraping. Three backends sit behind a single call surface, and each covers a different slice of X. fxtwitter is the zero-dependency path built on the standard library, and it handles single tweets and user profiles. Nitter is reached over direct HTTP and covers timelines, search, replies and mentions. The browser backend, driven by Camofox or Playwright, covers everything the other two do plus Lists and X Articles. The default mode is `auto`, which the README describes as Nitter first with browser fallback. Lists are an exception to that ordering: the README states that lists always use the browser, so `xtf --list 1455045069516357634` will not quietly fall back to an HTTP path. All three backends normalize into one `Tweet`, `Reply`, `Profile` or `Article` schema, which is the actual selling point. If you have ever written prompt logic against three different JSON shapes from three different scrapers, the single schema is the part that saves work. The Python API exposes the same routing through a `Router` object with `fetch_tweet`, `fetch_timeline`, `fetch_replies` and `search`, and `tw.to_dict()` produces a JSON-ready dict. Errors are typed rather than stringly: `error_code` is one of `invalid_input`, `not_found`, `rate_limited`, `upstream_down`, `backend_unavailable` or `all_backends_failed`, and the last of those carries per-backend `error_causes`. That is a better interface than most scrapers offer, because an agent can branch on `rate_limited` and retry later while treating `not_found` as final.
Install and first fetch: two commands, then a Nitter container
The README gives a clone-and-install path: `git clone https://github.com/ythx-101/x-tweet-fetcher`, then `cd x-tweet-fetcher && pip install .`. After that, a single tweet works with no configuration at all: `xtf --url https://x.com/user/status/1234567890`. There is also a no-install route, `python3 scripts/fetch_tweet.py --url ...`, which takes the same flags from inside the clone. Timelines, search and replies are where setup begins. You need a Nitter instance, and the README recommends self-hosting with `docker run -d -p 8788:8080 --name nitter zedeus/nitter:latest`, then `export XTF_NITTER=http://127.0.0.1:8788`. Only after that does `xtf --user elonmusk --limit 20` or `xtf --search "openclaw" --limit 10` have somewhere to go. Configuration is entirely environment variables, with CLI flags overriding them. `XTF_NITTER` accepts a comma-separated list tried in order with failover, so `export XTF_NITTER=http://127.0.0.1:8788,https://your-backup-instance.example` is a supported shape. `XTF_BROWSER` selects `camofox` or `playwright` and defaults to camofox on port 9377 via `XTF_BROWSER_PORT`. Playwright users install the extra with `pip install ".[playwright]"` from the clone. `XTF_LANG` defaults to `zh`, which is worth noting: error messages come back in Chinese unless you set it to `en`. `XTF_CACHE_DIR` defaults to `~/.x-tweet-fetcher` and holds the mentions-monitor cache. The v1 variable name `NITTER_URL` is still honored as a fallback.
The ledger turns fetches into a queryable SQLite archive
Passing `--ledger <db>` changes the tool from a fetcher into a fetch-archive-query loop. Every timeline, search, list, replies or single-tweet fetch is written into a SQLite database, deduplicated by `tweet_id` using `INSERT OR IGNORE`, which makes repeated runs idempotent. The schema is stated to be compatible with the tweet-ledger (OpenClaw) `tweets` table, so the same file can be read by both tools. The table carries `tweet_id` as primary key plus `created_at`, `full_text`, `lang`, `source_file`, `is_reply`, `in_reply_to_status_id`, `retweeted_status_id`, `quoted_status_id`, `urls_json`, `media_json`, `raw_json` and `imported_at`. Two details show care. First, archiving never breaks a successful fetch: if the write fails, the JSON envelope carries `ledger_error` instead of the fetch failing. Second, single-tweet results from fxtwitter lack a `tweet_id` field, so the CLI injects it from the URL, and `--replies` results are archived with `is_reply=1` and `in_reply_to_status_id` pointing at the parent. Querying is offline: `xtf --ledger ~/tweets.db --query "sop"` and `xtf --ledger ~/tweets.db --stats` need no network at all. Behavior without `--ledger` is unchanged from 3.0.0. The trade-off is that the archive is a flat table of tweets, not a graph. If your question is about conversation structure rather than text search, the parent pointers are there but you will be writing the traversal yourself.
The Nitter dependency is the project's real ceiling
The README is unusually direct about this: public Nitter instances are unreliable and frequently dead, and self-hosting is strongly recommended for timeline, search and replies. Read that as the boundary of the tool. Single tweets through fxtwitter are the durable path because they need nothing but the standard library. Everything else inherits the health of your Nitter container, and Nitter itself is a separate project with its own maintenance burden and its own relationship to X's changes. If you cannot run a container, or will not keep it patched, then `--user`, `--search`, `--replies` and `--monitor` are features you are renting from someone else's uptime. The failure is at least legible: with no reachable instance you get `error_code: "all_backends_failed"` and each backend's reason under `error_causes`, such as `backend_unavailable`, rather than an empty result. The README's phrasing is that you get a clear error telling you exactly what to set, never a silent empty result. That is the right design, but it does not make the dependency go away. The browser backend is the fallback for the gaps, and it is described as the slow path; it also needs Camofox running on port 9377 or Playwright installed. A second limitation follows from the first: this is a retrieval tool, not a monitoring platform. `--monitor` is described as incremental and cron-friendly, with a cache directory, and the exit code `1` for new mentions is how you detect change. Rate limits are surfaced as `rate_limited` rather than handled for you, so scheduling and backoff are your problem.
How it compares to snscrape and to paying for API access
The obvious alternative in this space is snscrape, a long-standing Python scraper for social platforms. The difference is in the retrieval model, not the language. snscrape scrapes X directly from your machine and has no backend abstraction, so when X changes something, the tool breaks until someone patches it, and every request carries the same risk profile. x-tweet-fetcher spreads that risk across three separate paths and picks one per request, which means a single-tweet lookup and a timeline fetch are not the same bet. The second real alternative is the official X API. That path is paid, but it is contractual: rate limits are documented, results are stable, and you are not dependent on a self-hosted proxy that can be discontinued. If your workload is high volume or commercially sensitive, the paid API is the honest comparison, and the README does not pretend otherwise. It frames the project as the answer for people who have no free API access and cannot run browser automation reliably. There is also a narrower comparison worth making: the ledger schema is stated to be compatible with the tweet-ledger (OpenClaw) `tweets` table, so if you already use that tool, xtf is less a replacement than a second writer into the same file. The README links a homepage at the openclaw-qa repository, which suggests the two are meant to be used together.
Version history, licence and what maintenance actually costs you
The releases show a project that has been reshaping its own surface rather than only adding features. v1.9.0 added an Obsidian export path for X Articles and arXiv papers into a local knowledge base. v2.0.0 introduced the three-backend structure along with self-hosted Nitter and keyword search, and v3.0.0 made it an installable `xtf` package. That last step matters for upgrade cost: the README has a dedicated migrating-from-v1 section and notes that `NITTER_URL` is still honored as a fallback for `XTF_NITTER`, and that behavior without `--ledger` is 3.0.0-compatible. So the compatibility story is deliberate, which lowers the cost of staying current. The licence is MIT, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are preserved; that is a summary of the licence text, not legal advice, and you should read the LICENSE file in the repository for the operative terms. The maintenance cost that the licence does not cover is Nitter. You are running a second container, and its health determines whether half the feature table works. Budget for that in the same way you would budget for any self-hosted proxy: an upgrade path, a restart policy, and a second instance in `XTF_NITTER` if the first one matters to you. The README's own suggestion of a comma-separated list exists precisely because one instance is not enough.
Editorial conclusion
Adopt it if you are building an agent or an OSINT pipeline that needs tweet text in a stable JSON shape and you are willing to run a Nitter container next to it; the fxtwitter path alone justifies the install for single-tweet lookups. Do not adopt it if you need guaranteed uptime on timeline or search calls and cannot host Nitter, because the README states public instances are unreliable and frequently dead. Before committing, verify two things on your own machine: that `xtf --url` returns a tweet with no configuration at all, and that `xtf --user <handle> --limit 5` succeeds against your own instance at the `XTF_NITTER` address you intend to keep running.
Community notes