Shortest: E2E tests written in English, executed by Claude over Playwright

English sentences instead of selectors

Shortest, from the antiwork organization, approaches end-to-end testing from the writing side. A test is a sentence such as 'Login to the app using email and password', and an agent executing it through the Anthropic Claude API drives the browser underneath, since the framework is built on Playwright. Plain Playwright asks you to encode every step as code, with selectors, waits and assertions. Shortest asks you to state the intent and leaves the interpretation to the model, which means the execution path is the model's reading of your sentence rather than a script you wrote line by line. Two integrations reach beyond the page: GitHub login with two-factor authentication is supported natively, and email validation runs through Mailosaur. Every execution depends on an Anthropic API key, so runs consume that account, and the README does not document token usage or rate limits. That is the whole trade: authoring gets shorter, execution gets an external dependency.

shortest init writes four things before you do

Setup is one command:

npx @antiwork/shortest init

It installs @antiwork/shortest as a dev dependency when missing, writes a default shortest.config.ts with boilerplate, generates .env.local with placeholders such as ANTHROPIC_API_KEY unless one exists, and appends .env.local and .shortest/ to .gitignore. The config it leaves behind looks like this:

import type { ShortestConfig } from "@antiwork/shortest";

export default {
  headless: false,
  baseUrl: "http://localhost:3000",
  browser: {
    contextOptions: {
      ignoreHTTPSErrors: true
    },
  },
  testPattern: "**/*.test.ts",
  ai: {
    provider: "anthropic",
  },
} satisfies ShortestConfig;

baseUrl is where tests point, testPattern decides which files count as tests, and browser.contextOptions passes Playwright browser context options through, so anything a Playwright browser context accepts can be set here. headless: false means the generated default runs with a visible browser window, which matches how the framework expects you to watch an execution the first time. The API key is read from SHORTEST_ANTHROPIC_API_KEY or ANTHROPIC_API_KEY, and ai.config.apiKey overrides both.

The sentence carries the intent, the object carries the secrets

Tests live in files matching the pattern, app/login.test.ts for a login flow:

import { shortest } from "@antiwork/shortest";

shortest("Login to the app using email and password", {
  username: process.env.GITHUB_USERNAME,
  password: process.env.GITHUB_PASSWORD,
});

First comes the instruction the AI executes in the browser. Second comes data it can use, here credentials pulled from the environment instead of hardcoded strings. For what a sentence cannot express, a callback runs after the browser part finishes, and the README states the AI executes it at that point:

const clerkId = await page.evaluate(() => {
  return window.localStorage.getItem("clerk-user");
});

if (!clerkId) {
  throw new Error("User not found in database");
}

That fragment reads the Clerk user id from localStorage and fails the test when it is absent, before a database query checks the row. The full example continues into drizzle-orm queries against the users table, so post-run verification of application state is part of the design.

beforeEach signs in with an email code, afterAll signs out

Around individual tests sit four hooks, beforeAll, beforeEach, afterEach and afterAll, each receiving the page object. Setup work goes in beforeAll, which in the documented example prepares Clerk against the running app:

shortest.beforeAll(async ({ page }) => {
  await clerkSetup({
    frontendApiUrl:
      process.env.PLAYWRIGHT_TEST_BASE_URL ?? "http://localhost:3000",
  });
});

Per-test isolation goes in beforeEach, here signing in with an emailed code:

shortest.beforeEach(async ({ page }) => {
  await clerk.signIn({
    page,
    signInParams: {
      strategy: "email_code",
      identifier: "[email protected]",
    },
  });
});

The same example pairs these with afterEach closing the page and afterAll signing out, so the browser session is torn down the way it was built up. Anything that must exist before the browser opens, or be cleaned after it closes, belongs in these hooks rather than in the sentence itself.

Chains and API calls reuse sentences

Passing an array instead of a string chains steps into one flow:

shortest([
  "user can login with email and password",
  "user can modify their account-level refund policy",
]);

Sentences are values, so they compose. The documentation stores flows like 'login as lawyer with valid credentials' in constants and combines them with spread syntax, shortest([loginAsLawyer, ...allAppActions]), which shares one login sentence across role-specific suites. API endpoints get the same treatment: an APIRequest built with a baseURL wraps a fetch, and the assertion stays in English:

shortest(
  "Ensure the response contains only active users",
  req.fetch({
    url: "/users",
    method: "GET",
    params: new URLSearchParams({
      active: true,
    }),
  }),
);

When even that is too much structure, the request can be described inside the prompt itself, as a template literal naming the endpoint, the query parameter and the expected response. Bearer-token variants of the same pattern appear twice in the examples directory, as api-assert-bearer.test.ts and api-assert-bearer-2.test.ts, next to api-failure.test.ts showing an expectation that does not hold.

From one test on line 23 to a headless CI pipeline

Four commands cover the run side:

pnpm shortest
pnpm shortest login.test.ts
pnpm shortest login.test.ts:23
pnpm shortest --headless

The first runs everything testPattern matches, the second a single file, the third a single test by line number, and --headless drops the visible browser. CI is headless mode plus the Anthropic API key in the pipeline secrets, and the repository carries a worked example in .github/workflows/shortest.yml. Testing GitHub authentication adds a two-factor step: obtain the OTP secret from the repository's authenticator app setup, store it, then run

shortest --github-code --secret=<OTP_SECRET>

and enter the code displayed in the terminal into GitHub's setup page to complete the pairing. Your .env.local ends up holding ANTHROPIC_API_KEY plus GITHUB_TOTP_SECRET for auth tests.

A monorepo that doubles as a live web app

Cloning the repository brings more than the framework. packages/shortest is the npm package @antiwork/shortest, with its own CONTRIBUTING guide, while app/, components/, drizzle.config.ts and next.config.ts make up a Next.js web application whose .env.example enumerates Clerk, Stripe, Postgres, Mailosaur, Anthropic and GitHub credentials. Running that web app has version strings attached: React >=19.0.0 is required with Next.js 14+ or Server Actions, Next.js >=14.0.0 for Server Components and Actions, and the README warns that React 18 under Next.js 14 can produce type conflicts in Server Actions and useFormStatus. Release activity sits at v0.4.9, tagged 2025-04-01, following v0.4.8 and v0.4.7 in March 2025, with the last push to the repository on 2026-08-06. The root package.json names the tree shortest-monorepo, pins [email protected] as the package manager, and splits its scripts between the two halves: cli:build, cli:dev and cli:test work in packages/shortest, while nextjs:test:e2e aliases pnpm shortest and db:generate, db:migrate and db:studio serve the drizzle-backed web app. Example tests under examples/ cover google.test.ts, youtube.test.ts, github.test.ts and API failure cases, worth a read before writing your first sentence.

Editorial conclusion

Use Shortest when writing tests in English beats maintaining selectors and an Anthropic API account is already in the budget. Skip it for suites that must run without external AI calls or that pin every assertion in code. Before adopting, verify the CI example in .github/workflows/shortest.yml against your own pipeline, and confirm your app stack against the React 19 and Next.js 14 requirements the repository states.

Frequently asked questions

Official sources

1. antiwork/shortest on GitHub
2. License: MIT
3. Project website
4. README
5. Releases