# passmark's config calls Redis required, and its quick start never mentions it

> This is a Playwright library that drives a browser from steps written in plain language, caches steps in Redis, and validates assertions with a two-model consensus and an arbiter. Following its quick start exactly leaves you without a service the shipped configuration file calls required. The licence string in the manifest is also worth reading before you adopt it, because it is not the identifier the page's open-source framing implies.

**bug0inc/passmark** — The open-source Playwright library for AI browser regression testing with intelligent caching, auto-healing, and multi-model verification.

- Repository: https://github.com/bug0inc/passmark
- Website: https://passmark.dev
- Stars: 1,276 · Forks: 188
- Language: TypeScript
- License: NOASSERTION
- Published: 2026-09-10 · Updated: 2026-09-10 · Language: en
- Canonical page: https://hysenlabs.com/projects/bug0inc-passmark

## The config file calls Redis required, and the quick start does not mention it

The shipped environment template has three sections and the first of them is a required section. Its only entry is a Redis connection URL, described as being for step caching and for global state persistence. A Redis client is a hard runtime dependency in the manifest, so this is not optional plumbing. Now read the quick start. It walks through initialising a Playwright project, installing the package, and setting two model provider keys. Redis appears nowhere in it. Following the page in order gets you a library whose own template says it will not function without a cache you have not been told to start. The caching feature is also the first of the three things the introduction promises, so this is not a minor service you can skip.

## The licence string in the manifest is not the identifier the open-source framing implies

Three artefacts describe the licence and they do not agree. The page calls the library open-source twice, once in the heading and once in the description. The manifest states a compound identifier made of a source-available licence name followed by an Apache 2.0 fallback. The licence field that the hosting API exposes is empty, which is the state a tool reading that API sees. The compound form is not one of the OSI identifiers that the word open source conventionally means, so the page's framing and the manifest's string are describing different kinds of reuse. Since this is a commercial project from a company that sells a related product, the distinction is worth ten minutes of reading the licence file before you adopt it rather than after.

## Four gateways, three resellers and one pass-through proxy

The page draws this distinction itself and it is the most operationally useful paragraph in the setup section. Three gateways are described as routing requests to multiple providers without managing individual keys, and each is a single environment variable: one for a hosted AI gateway, one for OpenRouter, one for another reseller. A fourth, on Cloudflare, is explicitly called a proxy rather than a reseller, and the consequence is spelled out: because it forwards to the upstream provider's own API, you still need the original provider keys alongside the account identifier and the gateway setting. So three routes let you hand your model traffic to somebody else, and one lets you keep your keys and add a layer for observability, caching and rate limiting. The cost and privacy consequences differ and the page does not pretend otherwise.

## Caching switches itself off in computer-use mode, and the model is locked

The default execution path uses accessibility snapshots. There is an alternative for visual, screenshot-driven work, and it changes three things at once. It requires direct access to one provider, because the computer-use tool is only exposed on that provider's own responses interface and cannot be reached through any gateway, so the gateway setting must be explicitly turned off. It pins the model, and the page says the model is currently locked and not user-configurable. And it skips Redis step caching entirely, with the reason given: coordinate-based actions are not portable across viewport sizes, so a cached click at one resolution is wrong at another. The result is that the feature the introduction leads with is silently absent in the mode you would reach for when a snapshot cannot find the element.

## A three-level override chain, and one combination that throws

The same configuration shape accepted globally is also accepted at the call and at the individual step, with a stated precedence: the step's setting wins, then the call's, then the global one. That is what makes hybrid runs possible. You can send most steps through a cheap cached gateway and put one hard step on the visual path, inside a single run, and the example on the page does exactly that with three steps where the middle one opts out and the outer two do not. The limit is narrow but real: a visual step still requires the gateway turned off, and mixing a visual step with a non-off gateway throws at the step level for the same reason it throws globally. So the chain is permissive everywhere except at one node.

## You have to patch your Playwright config before the library can read your .env

The library does not load your environment file. The page tells you to add three lines to your Playwright config, importing a dotenv helper and a path helper and calling the loader with a resolved path, and it places the snippet after the existing config import so the ordering is explicit.

```typescript
import dotenv from 'dotenv';
import path from 'path';

dotenv.config({ path: path.resolve(__dirname, '.env') });
```

It then tells you to install that helper yourself, which is a hint that it is not a dependency of the package, and indeed it is not in the manifest. That is a small amount of work for a small amount of friction, but it is friction in the setup path rather than in the test itself. The example that follows is where the page breaks: the shopping-cart test is cut off partway through a step object, mid-word, so the last steps of the worked example are not there to copy.

## A JavaScript parser and a fake-data generator are hard dependencies the page never explains

The manifest is worth reading next to the documentation. Two runtime dependencies stand out as ones you would not guess from the feature list. One is a JavaScript parser and abstract syntax tree library, a tool for reading source code, sitting alongside a model client, a data-schema validator and a request client. The other is a fake-data generator, which produces synthetic records rather than calling anything. Neither appears in the introduction, the features list, or the setup instructions in the visible documentation. The page also reveals a second export path in the package, a provider subpath dedicated to an email sink, which is a testing utility you would not expect from a browser-automation library and which the visible page never mentions either.

## Four names for one product: the repository, the manifest, the scope and the heading

The repository is named after a common English word for a benchmark mark. The manifest uses that same unscoped name and declares a version on a 1.0 line, with the description calling itself an open-source framework for regression testing. The published package is scoped to the organisation and installed under a different name, and the heading of the page uses a third name entirely for the product, with the organisation's main site as the project link. So a developer reading the page installs one name, imports a second, and searches the repository under a third. That is the ordinary consequence of scoping a package around an organisation, and it is also what keeps an unscoped repository name from colliding on a registry. The practical advice is to search on the scoped name, not the repository name.

## Conclusion

passmark fits a team that wants browser tests described in English rather than selectors, and that is willing to run a cache service and at least two model providers. Four things to settle first. Redis is required for step caching and global state and is not in the quick start, so provision it before you install. The visual computer-use mode switches caching off, locks the model, and cannot be routed through a gateway, so it is a different set of costs from the default path. Three of the four gateways are resellers and one is a pass-through proxy, which is a billing and data-handling difference rather than a preference. And read the licence string in the manifest before assuming the open-source framing means what you think.

## FAQ

### Does the passmark library need Redis?

Yes. The shipped environment template lists a Redis connection URL under a required heading, described as being for step caching and global state persistence, and a Redis client is a hard runtime dependency. The quick start on the page does not mention it.

### Which licence does the passmark package use?

The manifest states a compound identifier consisting of a source-available licence name followed by an Apache 2.0 fallback, and a licence file is present at the top of the tree. The page describes the library as open-source, and the licence field the hosting API exposes is empty.

### Can I use an AI gateway instead of separate model keys with passmark?

Yes. Three gateways are named, and each replaces the two upstream keys with a single variable. A fourth option on Cloudflare is described as a proxy rather than a reseller, so it forwards to the upstream providers and you still need the original keys alongside its account and gateway settings.

### What does computer-use mode change in passmark?

It requires direct access to one model provider because the computer-use tool is not reachable through a gateway, it locks the model so it cannot be configured, and it skips Redis step caching because coordinate actions are not portable across viewport sizes.

### Can passmark mix cached and visual steps in one test?

Yes, by overriding the setting per step. The precedence is step, then call, then global configuration. One combination throws: a visual step requires the gateway setting turned off, and pairing it with a named gateway fails at the step level for the same reason it fails globally.

## Sources

- [bug0inc/passmark on GitHub](https://github.com/bug0inc/passmark)
- [Issues](https://github.com/bug0inc/passmark/issues)
- [Project website](https://passmark.dev)
- [README](https://github.com/bug0inc/passmark/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/bug0inc-passmark