CommunityScrapers: the YAML catalogue that powers Stash metadata scraping

A repository of YAML definitions, not an application

The top level of the repository makes the nature of the project clear. Alongside the README there is a scrapers directory, a templates directory, a validator directory, a site_generator directory, a lib directory, a build.ts file, a deno.json and deno.lock, a validate.js file, a ruff.toml, and a .dvmrc. The language recorded for the repository is YAML, which matches how the artifacts are distributed: individual .yml files that Stash loads as metadata providers. The build tooling is Deno based, with Python linting configured through ruff.toml, so parts of the pipeline are written in TypeScript and parts in Python even though the shipped content is YAML. The homepage at stashapp.github.io/CommunityScrapers is a generated index of sites that currently have a scraper, produced by the site_generator directory. There are no GitHub releases for this repository, which fits a model where content is pulled continuously from the default branch rather than pinned to a version. The practical consequence is that a scraper you rely on can change under you when the branch moves, and there is no release tag to pin against.

Two install routes, and the setting that decides which one is viable

The primary route is the manager. Scrapers can be installed and managed from the Settings > Metadata Providers page, and the Available Scrapers section is where installation happens. The Community (stable) source is configured by default, so a fresh install already has a source to pull from without hunting for a URL. The second route is manual. To download all scrapers at once you clone the git repository, and if you only need specific scrapers you download those .yml files individually. The individual route has a specific procedure worth following, because the naive save often breaks the format. You open the .yml file, click the Download raw file button, then save the page as a .yml file to preserve the correct format. Files then go into the Scrapers Path under Settings > System > Application Paths, which defaults to ~/.stash/scrapers, the same ~/.stash directory that holds the config and database files. After a manual update you reload the scrapers and refresh the edit scene or performer page using Scrape with... -> Reload scrapers. That reload step is the part people forget, and without it a freshly copied file appears to do nothing.

Python scrapers need a runtime and a specific dependency set

Some scrapers require external programs to function, usually Python. The README states that all scrapers are tested with the newest stable release of Python, currently 3.14.x, which gives you a version target without pinning your whole machine. The dependency install is a single command, and it is the one command most people need from this repository. Run it in a terminal window:

python -m pip install stashapp-tools requests cloudscraper beautifulsoup4 lxml

You may need to replace python with py on Windows. For Windows users the README strongly recommends installing Python using the installers from python.org rather than through the Windows Store, and installing it outside the Users folder so it is accessible to the entire system, with C:\Python314 named as a commonly used option. If Stash does not detect your installation, the Python executable path setting in Settings > System > Application Paths can point at the executable itself, and the README is explicit that it must not point only at the folder. This is where a manual install quietly breaks: a Python that a shell can run but that Stash cannot resolve will leave every Python scraper failing with no obvious cause.

Blocked user agents and Chrome CDP are the two external dependencies

Two failure modes sit outside the control of the scraper file. The first is site blocking. Some sites block content if the user agent is not valid, so if you get a blocked or denied message you configure the Scraping > Scraper User Agent setting in Stash, and valid Firefox user agent strings are pointed at the Mozilla developer documentation. The repository's guidance is that scrapers for those sites should include a comment with a tested and working user agent string, so the fix is sometimes already written in the file you installed. The second is useCDP. Scrapers with useCDP set to true require the Chrome CDP path setting to be properly configured in Stash. If you use a remote instance, the README recommends the headless Chromium Docker image from chromedp/headless-shell, and states plainly that browserless/chrome is not CDP-compatible and is not supported. That is a hard boundary. If your environment is built around browserless, a useCDP scraper will not work through it no matter how the path is set.

Manually configured scrapers are overwritten by an update

Some scrapers need extra configuration before they will work, and the README calls this unfortunate, because installing them through the web interface means any update overwrites your changes. The details are worth knowing. Python scrapers that need to talk to your Stash instance, for example to create markers or search your file system, might need to be pointed at a local Stash: by default they use http://localhost:9999/graphql with no authentication to make their queries, and if your setup requires otherwise you find py_common/config.ini and set your own values. Python scrapers that can be configured will usually create a default config.ini in their respective directories the first time you run them. Some scrapers require an API key or a cookie, and in that case the instructions are in the scraper file itself. The design trade-off is clear: the repository keeps scrapers updatable, and pays for it by handing the configuration burden to the person running it. If you edit scraper files directly, treat the manager as the source of truth and keep your local changes somewhere else.

Scraping starts with a URL, and Debug logs are where failures show up

For most scrapers you have to provide the object URL. On the stable build, which the table in the README labels as version 0.11.0 or newer, populating the URL field with an appropriate url activates the scrape URL button. Clicking it opens a popup that lets you select which fields to update, which is the part that keeps a scraper from overwriting fields you care about. Some scrapers instead support the Scrape with... function, which works with either the Title field or the filename, so the scraper needs enough data to work with. Where a scraper supports it, a Query button opens an editor for the text the scraper will use for its queries. When scraping produces errors or no results, the README points to the log section at Settings > Logs with the Log Level set to Debug. The support surface around this is worth noting too: the project keeps its own step-by-step Guide to Scraping, and the community channels are a forum on discourse.stashapp.cc, a Discord server, and a Lemmy community, with the forum described as the primary place for support, feature requests, and discussions.

Editorial conclusion

Adopt CommunityScrapers if you run Stash and want metadata from external sites without writing your own scrapers, and prefer the manager route over copying files so the stable channel keeps working for you. Do not expect it to solve scraping for sites that block unlisted user agents or that need Chrome CDP, since those require configuration the repository cannot supply on your behalf. Before installing a specific scraper, read its file for configuration notes, check whether it needs Python dependencies from the pip line above, and confirm the Stash version is recent enough for the stable build, which requires v0.11.0 or newer for the URL field to activate the scrape button.

Frequently asked questions

Official sources

1. Issues
2. License: AGPL-3.0
3. Project website
4. README
5. stashapp/CommunityScrapers on GitHub