Substack2Markdown turns a subscription into a folder of Markdown files

A script, not a service, and the output is plain files

Substack2Markdown is a Python tool for downloading free and premium Substack posts and saving them as both Markdown and HTML files, and it includes a simple HTML interface to browse and sort through the posts. It will save paid for content as long as you're subscribed to that substack.

The output model is the first thing to understand. Once you run the script, it creates a folder named after the substack in /substack_md_files, and then begins to scrape the substack URL, converting the blog posts into Markdown files. Once all the posts have been saved, it generates an HTML file in the /substack_html_pages directory that allows you to browse the posts.

So there are two output trees, one for the Markdown and one for the generated browsing interface, and the repository carries them at the top level alongside substack_scraper.py, an author_template.html for the generated pages, a data/ directory, an assets/ directory with the interface screenshot, a tests/ directory and requirements.txt.

The target reader is someone who wants to own the text. The HTML interface exists because a folder of Markdown files is not browsable on its own, and the author addressed that with a generated page per author rather than a running web application.

The licence is MIT, the primary language is Python, and the last push was on 2026-08-10, with the newest tagged release being v2.1.0 from 2024-08-10.

The feature list is short: converts Substack posts into Markdown files, generates an HTML file to browse them, supports free and premium content with a subscription, supports scraping a single post URL directly for example /p/my-post, can download Substack-hosted images locally with --images, and the HTML interface allows sorting essays by date or likes.

Selenium against Edge, which is the part that shapes the setup

The scraping mechanism explains most of the requirements, so it is worth being direct about it. You will also need Microsoft Edge installed for the Selenium webdriver.

The dependency list confirms the approach. requirements.txt pins bs4 for parsing, html2text for turning the captured HTML into Markdown, requests at 2.31.0, selenium at 4.16.0, tqdm for progress, webdriver_manager at 4.0.1, Markdown at 3.6, and pytest at 8.3.4 for the tests.

So the flow is a real browser navigating the site, with BeautifulSoup reading the resulting DOM and html2text converting it. That is a different thing from calling a JSON API, and the difference shows up in practice: it works on whatever the page renders, including the parts of a Substack post that are only in the DOM after JavaScript has run, and it depends on a browser binary being present and a driver being matched to it.

The pinned versions are worth a second look for anyone planning to run this on a schedule or in CI. Every dependency is pinned to an exact version, which makes the build reproducible, and it also means the tool has a fixed compatibility envelope. webdriver_manager at 4.0.1 is the component that fetches a matching driver, and selenium at 4.16.0 is the version that driver has to agree with.

The repository also has a tests/ directory and pytest is pinned in the requirements, so the project has a test suite even though the README does not describe how to run it.

Installing it, and where the credentials come from

Installation is a clone and a pip install.

pip install -r requirements.txt

The README also shows an optional virtual environment, with the venv creation and activation lines commented out for Windows and for Linux, so the tool works inside or outside an environment.

Credentials are the part that needs care. For the premium scraper, set your Substack email and password with the SUBSTACK_EMAIL and SUBSTACK_PASSWORD environment variables.

export SUBSTACK_EMAIL="[email protected]"
export SUBSTACK_PASSWORD="your-password"

The alternative is a file named config.py in the root of the project containing exactly these two lines.

EMAIL = "[email protected]"
PASSWORD = "your-password"

Two things are stated plainly about that. config.py is gitignored, so your credentials will never be committed. And the environment variables take precedence over config.py if both are set.

That precedence matters in practice, because a stale environment variable silently wins over an edited config.py, and the symptom is a login failure that looks like wrong credentials rather than a configuration ordering issue.

The credential handling is also the honest limit of the whole approach. The script logs into the Substack account as you, with your subscription, and reads what that account can read. It does not crack anything and it does not extend your access, which means it will not fetch a post you have not paid for, and it means the tool inherits every policy Substack applies to your account.

The command line, and the six ways to change the output

You can either hardcode the desired Substack URL and the number of posts into the top of the file and run the script with no arguments.

python substack_scraper.py

Or you specify them. For a free site.

python substack_scraper.py --url https://example.substack.com --directory /path/to/save/posts

For a premium site the flag changes.

python substack_scraper.py --url https://example.substack.com --directory /path/to/save/posts --premium

A single post can be scraped directly.

python substack_scraper.py --url https://example.substack.com/p/my-post

Images can be pulled down locally and the Markdown links rewritten to point at them.

python substack_scraper.py --url https://example.substack.com --images

And the number of posts can be capped.

python substack_scraper.py --url https://example.substack.com --directory /path/to/save/posts --number 5

The last option is the one with downstream consequences. The default header written into each Markdown file is a `# title` followed by a `**Likes:** N` line, and the frontmatter option emits YAML frontmatter carrying title, subtitle, date, author and image instead, suitable for MDX sites.

python substack_scraper.py --url https://example.substack.com --frontmatter mdx

So the same script produces two different header conventions, and the choice depends on where the files are going next. If you are feeding a static site generator that expects frontmatter, the default output will not work without editing, and the flag exists precisely for that.

Reading the results without a Markdown renderer

There are two answers to the question of how to read what the script produced.

The README's first suggestion is to install the Markdown Viewer browser extension to read the Markdown files in your browser. But note, the author adds, that the files are also saved as HTML for easy viewing, just set the toggle to HTML on the author homepage.

That is the cheaper option. The generated /substack_html_pages directory already contains an HTML interface with the author homepage, and the toggle switches between the two views. The sorting behaviour comes from the feature list: the HTML interface allows sorting essays by date or likes.

Sorting by likes is worth a moment because it is a different axis from the one a subscription feed gives you. A feed is chronological. A sorted-by-likes index surfaces the posts that resonated, which for an archive you come back to is often the more useful ordering.

The template for those pages is author_template.html at the top level, so the generated pages are driven by a template you can edit before running the script.

There is also the online route. A web version of the tool exists at Substack Reader, built by @Firevvork, with no installation required, and it works for free Substacks only. You enter the Substack URL, then click Go to view the content or Export to download the Markdown files. The README is explicit that the online version does not support exporting premium content, and for full functionality including premium content export you should use the local script.

Where Substack2Markdown is the wrong tool

Three cases, and the first is about the architecture rather than the code.

This is a script you run. The README documents no scheduler, no watch mode, no incremental sync and no state file, so keeping an archive current means running it again, and a rerun does not obviously skip posts it has already saved. If you want a subscription archive that updates itself, you have to build that around it, and the fact that the browser has to be driven each time makes the cost of each run higher than an API call would be.

The second is premium content on someone else's account. The script logs in as you with your own subscription, so it will only fetch what you can already read. It is not a way to obtain a paid post without paying, and the README is careful to frame it as saving paid for content as long as you're subscribed to that substack.

The third is a headless environment. The README says you need Microsoft Edge installed for the Selenium webdriver. On a Linux server, a container or a CI runner with no browser, that is an extra piece of infrastructure, and browser automation is exactly the kind of thing that breaks silently when a site changes its markup.

One more thing to check before building on the output. The Markdown files are converted from the rendered page, so the conversion carries whatever the page contained. That means images are still remote references unless you use --images, links are whatever the post contained, and the default header includes a Likes count that is a snapshot rather than a live figure.

Maintenance, and the gap between the tags and the commits

The release history has three entries and the last push is more recent than all of them. v1.0.0 on 2024-01-06, v2.0.0 on 2024-01-28, and v2.1.0 on 2024-08-10, with the last push to the repository on 2026-08-10.

So the commits have continued well past the newest tag, which means work is landing on main that is not in a released version. If you install from a tag you get the 2024 state, and if you clone main you get whatever has changed since, without a version number to record. For a script this is a low-stakes difference, but it is a real one for anyone trying to reproduce a result later.

The dependency pins make that gap more consequential. bs4 at 0.0.1, html2text at 2020.1.16, selenium at 4.16.0 and webdriver_manager at 4.0.1 are all versions from a particular period, and a newer Selenium or webdriver_manager can behave differently around driver acquisition, which is the fragile part of a Selenium setup.

The licence is MIT, so there is nothing to untangle there for anyone redistributing or modifying the script.

Two documentation gaps are worth knowing about. The README does not describe how to run the test suite even though pytest is pinned in requirements.txt and a tests/ directory exists, and it does not say how the scraper detects the end of a post list or how it handles a post that changes after you have already downloaded it. Both are the kind of detail you find by reading substack_scraper.py rather than by reading the README.

Editorial conclusion

Adopt Substack2Markdown if you want to keep an offline copy of posts you are subscribed to, in plain Markdown you can grep, version and feed to a static site, and you have Microsoft Edge available for the Selenium driver. Do not adopt it if you need a scheduled sync, since the README documents a one-shot script with no scheduler, and do not expect it to fetch posts you are not subscribed to. Verify first which output format you want, because the default header is a Markdown title with a Likes line and the frontmatter option replaces that, and check the requirements.txt pins before you build a pipeline around them.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. timf34/Substack2Markdown on GitHub