PHPScraper: a web utility that hands PHP developers a filled-in page object

The problem is the data preparation, not the fetching

PHPScraper is described as a versatile web-utility for PHP, and its primary objective is stated in the README as streamlining the process of extracting information from websites, so that you can focus on the task instead of getting caught up with the complexities of selectors, data structure preparation and conversion.

That sentence identifies the real cost of scraping, and it is worth being precise about it. Fetching a page over HTTP is easy. What follows is not: finding the right element, reading its attributes, normalising a relative URL into an absolute one, deciding what to do when the element is missing, and turning the result into a shape your application can use. That work is repetitive, and it is the same work on every site, with small variations.

The library's answer is a single object with prepared extractors for the parts of a page that people actually want. The documented coverage includes several meta tags and other head information, social media information such as Twitter Card and Facebook Open Graph, content such as headings, outline, texts and lists, images, links, and keywords. Each of these has a page on the project website with example code, and the tests directory holds further examples.

The audience is PHP developers who need a small amount of page data inside an application, and the licence is GPL-3.0, with the licence text in the repository as `LICENSE.md`.

Every accessor works as a property or as a method

The first design choice is small and it shapes the whole API. All scraping functionality can be accessed either as a function call or as a property call, and the README shows the title being retrieved both ways:

// Prep
$web = new \Spekulatius\PHPScraper\PHPScraper;
$web->go('https://google.com');

// Returns "Google"
echo $web->title;

// Also returns "Google"
echo $web->title();

For a PHP developer this removes a small but recurring decision at every call site. When the value is used inline, the property form reads better; when you need to pass it around, test it or chain it, the method form is the one you want. Both are the same accessor underneath.

The second thing to notice is what happens when the page does not contain what you asked for. The documentation states that if there aren't any matching elements on the page, an empty array will be returned, and that if a method normally returns a string it might return `null`. That is the kind of detail that separates a scraping library that is usable in production from one that surprises you, and it is stated in the documentation rather than left to be discovered.

Configuration options such as `follow_redirects` are described as optional parameters in the same way, so the default behaviour is documented rather than folklore.

Detailed variants and the raw fallback underneath

Many of the extractors come in two forms, a simple one and a detailed one, and the README uses `linksWithDetails` to show what the detailed version buys.

The idea is that the same link can be described at two levels of fidelity. The simple form gives you the URL; the detailed form gives you a structure with the protocol, the link text, the title, the target, the rel attribute, the image URLs found inside the anchor, and a set of flags for the rel values that matter to a crawler or a reviewer, such as whether the link is `nofollow`, `ugc`, `sponsored`, `me`, `noopener` or `noreferrer`.

That is a good example of the library's value proposition, because those flags are exactly the kind of thing that is tedious to extract and easy to get wrong when written by hand.

There is a second layer of detail worth knowing about. Whenever post-processing is applied, you can fall back to the underlying `*Raw` methods. So the processing sits on top of a raw extraction layer, and when the processed shape is not what you need, you can reach the original data instead of writing a second parser.

The trade-off is that this is a fixed catalogue of extractors rather than a query interface. The library covers most of the DOM through its prepared methods, but a page element it does not anticipate needs the raw layer or a different tool. That is a sensible design for a utility library and a real limit for a general purpose scraping framework.

Feeds, assets and structured files

Beyond a single page, the library handles the things that surround one.

File downloads are covered by `fetchAsset`, which works with an absolute URL or with a relative one after a navigation, so a page can be fetched and then a file it references can be pulled with the same object:

// Absolute URL
$csvString = $web->fetchAsset('https://test-pages.phpscraper.de/test.csv');

// Relative URL after navigation
$csvString = $web
  ->go('https://test-pages.phpscraper.de/meta/lorem-ipsum.html')
  ->fetchAsset('/test.csv');

Feeds are handled as well, and the documented use is deciding what to crawl next. PHPScraper can assist in collecting feeds such as RSS feeds, `sitemap.xml` entries and static search indexes, which is useful when deciding on the next page to crawl or building up a list of pages on a website. The sitemap is returned as a set of `FeedEntry` DTOs, and the call is a chain:

(new \Spekulatius\PHPScraper\PHPScraper)
    ->go('https://phpscraper.de')
    ->sitemap

Structured files get dedicated parsers. PHPScraper comes out of the box with file and URL processing methods for CSV, XML and JSON, namely `parseJson`, `parseXml`, `parseCsv` and `parseCsvWithHeader`, the last of which generates an associative array using the first row. Each method can process both strings as well as URLs, which is the detail that makes them usable in a pipeline rather than only in a script.

The keyword extraction is where the third dependency comes in. Under the hood the library uses `donatello-za/rake-php-plus` to extract and analyze keywords.

Built on BrowserKit, not on a browser

The dependencies are listed explicitly in the README, and they define the class of problem the tool solves. It uses BrowserKit, formerly Goutte, to access the web, League/URI to process URLs, and rake-php-plus to extract and analyze keywords. The `composer.json` in the repository has the details.

The BrowserKit choice is the important one. BrowserKit is an HTTP client with a DOM, not a browser engine, which means a page is fetched and its markup is parsed, and no JavaScript in that page is executed. That is a feature for the intended use, because the tool stays light, needs no browser binary and does not have to spin up a headless Chrome to read a product page.

It is also the boundary. A site that renders its content client-side returns markup that does not contain the data, and the extractors will correctly find nothing. The repository's topic tags list headless Chrome, Puppeteer and Pyppeteer alongside BeautifulSoup and Scrapy, which is a fair description of the ecosystem this library sits in rather than of what it does itself.

So the honest comparison is with two other approaches. The first is writing the extraction by hand with a DOM library and XPath or CSS selectors, which gives full control and costs the most code, since every site needs its own selector logic and every field needs its own normalisation. The second is driving a real browser, which handles client-rendered pages at the cost of a browser dependency, much more memory and a slower run. PHPScraper sits in the middle, and the middle is exactly right for a job like reading a product page, a search results listing or a feed.

Version 3.0.0, GPL-3.0 and what the repository holds

The release history is short. Version 1.0.0 was tagged as finally stable in November 2022, 2.0.0 was a service update in June 2023, and 3.0.0 is dated 2024-07-19. The repository is not archived and the last push was on 2026-09-08, so the branch is being worked on even though the newest tag is more than two years old. If you are pinning a version, the `UPGRADING.md` file in the repository is the place to look for what changed, and the `CHANGELOG.md` is the full history.

The quality tooling in the repository is worth noting for a library that other code depends on. There is a `phpstan.neon` with a `phpstan-baseline.neon`, so static analysis runs and known issues are baselined rather than ignored, a `phpunit.xml.dist` for the test suite, `pint.json` for code style, and a `rector.php` for automated refactoring rules. The `playground.php` file at the root is a scratch file for trying things out, which is a small sign of a project that expects people to experiment with the API.

The licence is GPL-3.0, which is the one thing to check before you depend on it. A permissive licence would let you embed the library in a closed source product without conditions. GPL-3.0 does not, and while that is a statement about the licence terms rather than advice about your obligations, it is the kind of thing that has to be settled at the start of a project rather than at the end of it.

For a tool whose whole purpose is to save an afternoon of selector writing, that is the main trade on offer: less code, and a licence with conditions attached.

Editorial conclusion

PHPScraper suits a PHP project that needs to read a handful of pages, whether for a price check, a feed aggregator or a migration script, and that would rather not hand-write selectors and normalisation code. It does not suit a project that must render client-side JavaScript, because the library works at the HTTP level, and it is a poor fit for high volume crawling, where a dedicated crawler framework with its own scheduling and politeness handling is the better tool. Before adopting, check the version you are pinning, since 3.0.0 is the newest release and there is an UPGRADING.md, and confirm that the licence, GPL-3.0, suits how you intend to distribute the code that depends on it.

Frequently asked questions

Official sources

1. License: GPL-3.0
2. Project website
3. README
4. Releases
5. spekulatius/PHPScraper on GitHub