rss-feeds: generating feeds for the blogs that publish none

The problem, stated plainly

The project description is one sentence: generate RSS feeds for all the blogs that don't have one. That is the entire scope, and it is a surprisingly common problem.

Plenty of engineering blogs, research group sites, and company news pages publish no feed at all. There is no atom endpoint, no syndication link in the head of the document, and nothing to hand to a reader. A person who wants to follow one has three bad options: visit the site in a browser and hope they notice a new post, run a scraper themselves, or give up.

This project takes the third option away by being the scraper for everyone else. The output is a set of XML files published as static assets, each one reachable at a raw content URL, so subscribing to a blog that has no feed becomes a matter of pointing a reader at one of those URLs.

The maintainers are named at the top of the README, with a sponsorship request attached, and the project points at a companion blog post titled No RSS Feed? No Problem. Using Claude to automate RSS feeds, which is the honest account of how the scraping is done. The use of a language model to drive a browser or a parser is the modern version of the trick that used to be a hand written selector.

The project is MIT licensed, written in Python, and the last push was on 2026-10-20. There are no tagged releases, because the artefact is not the code, it is the collection of feeds that the code maintains.

The catalogue and the three states a row can be in

The README opens with a table of blogs and the feed for each, and the table is the most useful thing in the repository.

The entries cover a wide slice of the machine learning and software world. There is the AI at Meta blog, the AISI Blog, four separate Anthropic entries for engineering, news, research, and the frontier red team work, the Claude Blog, the Cohere Blog, the Cursor Blog, the Dagster Blog, the FAR.AI publications, the EleutherAI papers, the Goodfire research pages, the Google Developers Blog filtered to the AI category, the Groq Blog, the Mistral AI news, the AI FIRST podcast in German, and a set of personal sites such as Chander Ramesh's writing, Hamel Husain's blog, and Interconnected by Matt Webb.

Three states are distinguished, and the distinction is the interesting part of the design.

Rows marked Official RSS point at a feed the blog publishes itself. The Claude Code changelog has its own feed, the Cloudflare skills repository points at its commit atom feed, the Google DeepMind blog has its own, Hamel Husain's site has an index feed, and Matt Webb's blog has one. In each case the row is a pointer, not a generated file.

The other rows point at generated XML files inside the feeds directory of the repository, and their names follow a consistent scheme, one file per site with a prefix and the site name.

And a third state is called updates paused. The README explains that such a feed keeps its last published XML but does not receive new articles, which is the honest way to handle a site whose markup changed and whose scraper has not caught up. A paused feed still works for a reader, it just goes quiet, and the catalogue says which is which.

The enabled scraped feeds are generated hourly, so the freshness of a generated feed is measured in hours rather than in minutes.

Two pipelines, and one for the pages a request will not fetch

The workflow badges at the top of the README name three pipelines, and the split between them is the clearest signal about how this kind of project actually works.

The first workflow runs the feeds. The second runs the Selenium feeds, which exist because a plain HTTP request is not always enough to see the article list. Some sites render their index with client side JavaScript, some refuse requests without a browser's headers and cookies, and some serve a different page to a headless client. Selenium with an undetected driver, which is in the dependency list, is the answer to those cases.

That split is worth dwelling on, because it is the honest shape of feed generation. The easy case is a static page with a predictable list of articles, and a requests call plus a parser handles it in a fraction of a second with no browser involved. The hard case needs a real browser, which is orders of magnitude more expensive in time and in memory, and which also breaks far more often, because a browser is a moving target and the site is not obliged to keep the same markup.

The third workflow is described as the offline feed checks, and the word offline is what makes it interesting. A pipeline that only ever fetches the live web cannot tell you whether a feed is broken until the next time someone tries to read it. A check that works against the cached or stored output can tell you immediately, on every commit, that the XML is well formed and that the entries have the fields a reader needs.

So there are two kinds of automation here, generating and validating, and the second is what stops a scraping project from decaying silently.

The dependency list, and what it says about the approach

The project manifest is short and it maps cleanly onto the approach.

The parsing side is Beautiful Soup at a recent 4.15 minimum and lxml at 6.1.1, and the lxml dependency is the interesting one, because it is a much faster parser than the built in one and it is what makes it practical to parse a few hundred sites on a schedule.

The generation side is feedgen at 1.0, which is the library that writes the XML, so the project is not hand rolling the document format.

The fetching side is requests, plus the date handling from python-dateutil and the time zones from pytz, and those two are there because feeds are a time series and a feed with the wrong timestamps is a feed a reader will deduplicate or reject.

The browser side is selenium at 4.47 and undetected-chromedriver, and the second of those exists for one reason: many sites detect automation, and a stock Chrome announces itself. The undetected driver is the standard workaround, and it is worth noting that it is a workaround against a defence rather than a feature.

The configuration side is pydantic with pydantic-settings, which means the feed definitions are validated models rather than loose dictionaries, and pyyaml for the catalogue file. The models are validated at load time, so a feed definition with a missing selector fails immediately rather than at three in the morning when the schedule runs.

The last detail is the version floor. The project requires Python 3.11 or newer, which is a modern floor, and the tool configuration targets that same version.

The Makefile as a discovery interface

The build file is the most unusual piece of engineering in the repository, and it is worth reading on its own terms.

The default goal is help, which is a deliberate choice: running make with no arguments prints the documentation rather than starting a scrape. The help output is generated by grepping the makefiles for targets that carry a double hash comment, and the comments are the documentation. It is the convention every Makefile uses and every Makefile gets right.

What is unusual is the classification. A set of patterns is defined at the top, matching the target prefixes for help, environment, feeds, development, continuous integration, cleaning, and the debug variables. The help output then walks six labelled sections, being information and discovery, environment and dependencies, development, continuous integration and delivery, cleaning, and feed generation, and within each section it greps the root makefile and every file in the makefiles directory, then sorts the result and prints it.

The reason for the pattern prefixes is that the root Makefile is a thin dispatcher over a directory of included makefiles, and the prefix convention is what makes it possible to present the flattened list by category without hard-coding a list of targets anywhere. When a new makefile is dropped into the directory, its targets appear in the right section automatically.

The debug variables target is the one to know about, because a project with this many moving parts, browser automation, schedules, and a catalogue of site definitions, needs a way to print its resolved configuration.

The formatting uses a small set of terminal variables for bold and colour, and the shell commands are a mix of grep and awk, which means the help works on any machine with a POSIX shell and no extra tooling.

The repository layout, and the schedule

The top level of the repository tells you how the pieces fit together.

The feeds directory holds the generated XML that is published, and feeds.yaml is the catalogue that drives the generation, which means adding a site is a matter of describing it rather than writing code. The feed_generators directory holds the Python, and the tool configuration points the source root at it, with the utilities and the models marked as first party for the import sorting, which tells you the code is split into a generators package and shared helpers.

There is a cache directory, which is the mechanism behind the offline checks. Cached copies of what was fetched are what lets a check run without the network, and what lets a run be retried without refetching everything.

The scheduled_tasks directory holds the scheduling definitions, and the makefiles directory holds the included makefiles that the root dispatches to. There is a tests directory, a docs directory, and a Makefile at the root.

Then there is the part of the repository that says something about how the project is worked on. There are agent instruction files for several tools at once, an AGENTS file, a Claude file, a Codex file, and a Gemini file, alongside a contributing guide and a pre-commit configuration. Multiple agent instruction files in one repository is an unusual amount of duplication, and the reason for it is straightforward: the files are read by different tools, and a rule that lives in one of them is a rule that one tool will not see.

The tooling around that is modern. The linter is Ruff with an explicit selection of rule families, including the pyupgrade, bugbear, simplify, and performance groups, and the import sorter is configured to agree with the formatter, with the source root pointing at the generator package. The pre-commit hook runs the same tools, and the dependency lock file is present, so the versions a contributor runs are the versions the schedule runs.

Editorial conclusion

rss-feeds fits a reader who follows technical blogs that never shipped a feed, and who would rather have a URL to subscribe to than a scraper to run. It does not fit a reader who needs the feed to be authoritative, because a scraped feed is only as good as the scraper, and the project is explicit that some feeds are paused. Verify first that the site you want is in the catalogue, and whether its row points at an official feed or at a generated one, because the generated feeds are regenerated on a schedule and their content lags the site. Also check whether the project can run offline, since one of the workflows exists to check the feeds without hitting the network.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. Olshansk/rss-feeds on GitHub
4. README