Understudy: a local-first teachable GUI agent that watches once and replays with generalization
An understudy watches. Then performs.
At a glance
- What is it?
- Understudy is an MIT-licensed TypeScript agent runtime that drives your desktop, browser, shell and messaging apps from one instruction, and can learn a task from a single demonstration. Here is how it is put together, how to run it, and where it stops being the right tool.
- Who is it for?
- Adopt Understudy if you want a local runtime you point at your own API keys and you are willing to teach it rather than configure it, especially on macOS where the README's computer-use claims live. Do not adopt it if you need unattended production automation across a mixed fleet of operating systems, or if you cannot accept an agent holding your desktop session and your messaging accounts.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 88 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What Understudy actually solves, and who it is for
Most desktop automation breaks the moment a button moves. Coordinate-based recorders replay clicks against pixels, so a window resize or a UI redesign invalidates the recording. Understudy's pitch is different: the README says it learns the intent of a demonstrated task rather than the coordinates, which is why the project claims a taught skill survives UI redesigns, window resizing, and even switching to a different app.
The audience is narrow but real. This is for a single technical user who owns the machine the agent runs on and wants one instruction to cover a chain of work that spans a browser, a native app, a shell, and a chat client. The README's own examples are of that shape: convert a webpage to PDF and send it to a contact on Telegram, or research a topic and build a landing page in a downloads folder. It is not a hosted service and not a team platform. The package description calls it a "Local-first teachable GUI agent runtime with gateway, web UI, channels, and tools", and the topics list includes own-your-data, which tells you where the project expects your data to live.
How the runtime is put together: gateway, channels, skills, playbooks
The repository is a pnpm workspace. Top-level entries include apps/, packages/, skills/, scripts/, and tests/, with a tsconfig.base.json and separate vitest configurations for the workspace and for individual packages. That layout matters because it separates the pieces the README describes: a gateway that accepts instructions, a web UI, messaging channels, and the tools the agent calls.
The channels are the dispatch layer. According to the README, Understudy works with Telegram, Discord, Slack, WhatsApp, Signal, LINE, iMessage, and Web, which is the "8 built-in messaging channels" row in its comparison table. In the remote-dispatch demo, a message sent from a phone via Telegram arrives on a Mac and becomes a GUI automation run: convert a file to PDF, open desktop Telegram, find the contact, send it.
The teach system is the second mechanism. A demonstration produces a SKILL.md file, and the repository ships a real one under examples/published-skills/. The README describes interactive refinement after generation and invocation by natural language afterwards. On replay it says the agent generalizes: a Google Image search step becomes browser automation and a download step becomes a shell command, while native app control stays GUI-driven. The third mechanism is the playbook, which the README describes as orchestrating workers (deterministic browser and device automation) and skills (agentic subagents) as separate child sessions with their own context. That is a design decision worth noting: context isolation per stage is what makes a six-stage pipeline feasible, and it is also why a failure in one stage does not automatically explain itself in the next.
Installing Understudy and running a first instruction
The README advertises Node.js >=20.6 in its badge, and package.json declares the package as ESM with a single binary named understudy pointing at understudy.mjs. The published package name is @understudy-ai/understudy. The README does not walk through an install sequence in the text available here, so the configuration below is what the repository's own environment template contains rather than a documented tutorial.
The repository ships a .env.example at its top level. It lists four provider keys, two defaults, and an optional gateway token:
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GOOGLE_API_KEY=
MINIMAX_API_KEY=
UNDERSTUDY_DEFAULT_PROVIDER=openai-codex
UNDERSTUDY_DEFAULT_MODEL=gpt-5.4
# Optional gateway auth token for browser or CLI clients
UNDERSTUDY_GATEWAY_TOKEN=Copy that file to .env and fill in the key for the provider you intend to use. The example sets the default provider to openai-codex and the default model to gpt-5.4, so if you only supply ANTHROPIC_API_KEY you should expect to change UNDERSTUDY_DEFAULT_PROVIDER as well. UNDERSTUDY_GATEWAY_TOKEN is described in the file as an optional gateway auth token for browser or CLI clients; set it if you expose the gateway beyond your own machine.
For a first real run, follow the shape of the README's own example rather than inventing one: give the agent a single instruction that touches a browser and the local filesystem, such as researching a topic and writing a page into your downloads folder. The teach flow is entered with /teach start according to the demo description, followed by performing the task once, then refining the generated skill interactively, then invoking it in natural language. Expect the first run to be slow and the taught run to be faster, which is the project's stated premise: "Teach it once, it learns. Use it daily, it gets faster."
Where Understudy is the wrong tool
The comparison table in the README is candid about the platform boundary: the GUI and computer-use row reads "Yes (macOS)". The demo environment is macOS, and the iPhone work runs through iPhone Mirroring, which is an Apple feature. If your fleet is Linux or Windows, the computer-use half of the product is not what the README describes, and you would be left with the browser, shell and channel pieces, which are the parts other tools already cover.
There is a second, less obvious limit. A GUI agent that drives your desktop session occupies that session. The remote-dispatch demo works because the Mac is available to be driven while the user is on a phone. If you need to keep using the machine while the agent works, you are competing with it for focus and input. The README does not document a headless or virtual-display mode, and it does not document rollback of a partially completed run, so an interrupted pipeline can leave your desktop in whatever state the last action produced.
Cost is the third constraint, and it is structural rather than a bug. The README's pricing row is "Open-source runtime + own API keys", which means every replay, every generalization decision, and every agentic subagent in a playbook is billed by your model provider. A taught skill that generalizes into browser automation still needs a model to decide when to apply that generalization. The README does not publish token or cost figures for any of its demos, so the only honest statement is that the runtime is free and the inference is not.
How Understudy differs from OpenClaw and Cowork
The README's own comparison table puts OpenClaw at "Partial" for GUI and computer use, describing it as browser automation, and Cowork at "Yes" for GUI but "No" for teach by demonstration. That is the real dividing line. OpenClaw and Understudy share a distribution model (open-source runtime plus your own API keys, MIT for both), but they diverge on what the agent is allowed to touch: a browser-automation agent operates inside a tab, while Understudy's GUI layer operates the desktop itself, which is what makes the Pixelmator Pro and iPhone Mirroring steps in the demos possible.
Cowork's difference is the opposite one: the README describes it as closed source with subscription pricing, and it lists 50+ MCP connectors while noting in parentheses that this is "not a messaging inbox". Understudy's channel count is smaller (eight) but the channels are messaging apps rather than tool connectors, so the comparison is not like for like. If your integration need is a data source, connectors are the more natural fit. If your need is to receive an instruction from your phone and have a desktop act on it, that is the channel model.
The comparison table is dated in the README as a snapshot of March 26, 2026, and it states its own sourcing rule: capabilities come from official docs and narrow wording is used where a product does not clearly advertise something. Read it as the project's argument, not as an independent evaluation.
Licence, maintenance and what an upgrade costs you
Understudy is MIT licensed, and the published package includes LICENSE, CONTRIBUTING.md and CLA.md in its files list. MIT is permissive: you can use it commercially, modify it and redistribute it, provided the copyright notice and permission notice travel with it. The CLA is the part to read if you intend to contribute rather than consume, since it governs the rights you grant when you send a patch. Nothing here is legal advice; if you are embedding the runtime in a product, have counsel read the LICENSE and CLA together.
The current published version is 0.3.0, and the repository shows no retrieved releases, so versioning appears to run through package publication rather than GitHub releases. That has a practical consequence: your upgrade unit is an npm version, not a tagged release with notes. The repository does carry a CHANGELOG.md at the top level, and a pnpm-lock.yaml plus pnpm-workspace.yaml, so a workspace install is reproducible if you pin.
The maintenance picture is mixed. The repository is not archived, but the last push was on 2026-06-19, which is three months before the date of writing. That is not abandonment, and it is not the same thing as steady development either. The CI badge in the README points at a GitHub Actions workflow, and the package scripts include a synthetic playbook end-to-end mode alongside a live one, which suggests the project tests the pipeline without requiring a real desktop for every run. That is a good sign for contributors and a weak signal for users, because synthetic mode does not exercise the GUI layer you actually depend on. Before adopting, check the CHANGELOG.md for the distance between 0.3.0 and whatever is current, and check whether the skills/ directory format has changed, since a taught skill is an artifact you would have to regenerate.
Editorial conclusion
Adopt Understudy if you want a local runtime you point at your own API keys and you are willing to teach it rather than configure it, especially on macOS where the README's computer-use claims live. Do not adopt it if you need unattended production automation across a mixed fleet of operating systems, or if you cannot accept an agent holding your desktop session and your messaging accounts. Verify three things first: that your Node.js version satisfies the >=20.6 requirement, which provider key you will actually use, and whether the skill you plan to teach survives when the target app changes. The repository's last push was on 2026-06-19, so check the issue tracker before you build a workflow you cannot replace.
Frequently asked questions
What is Understudy?
It is an open-source local AI agent that operates your computer across GUI, browser, shell and messaging apps from a single instruction, according to the README. The package describes it as a local-first teachable GUI agent runtime with a gateway, web UI, channels and tools, and it is MIT licensed.
What does understudy mean in the context of work?
In this project the name is used as a metaphor rather than a job title: the README's tagline is "An understudy watches. Then performs." The agent watches you demonstrate a task once, then performs it on request, learning the intent instead of the screen coordinates.
How do I use Understudy?
Install the @understudy-ai/understudy package, put a provider key in a .env file based on .env.example, and give the agent one instruction. The teach flow is entered with /teach start, after which you demonstrate the task once, refine the generated skill, and invoke it in natural language.
What is another word for understudy?
The project does not discuss synonyms; it uses the word as the name of the agent, with the tagline "An understudy watches. Then performs." The substitute-performer sense of the word is the metaphor behind the product, not a documented feature.
Community notes