Skill Recorder: turning one screen recording into an agent procedure

The problem is the gap between showing and specifying

Most automation starts with someone writing down what they do. That write-up is where the detail dies: the window switches, the page you check, the copied identifier that ties two systems together. Skill Recorder takes the opposite route. You perform the task once, on screen, and the tool reconstructs the procedure from the evidence of the session. Its stated goal is to turn a single run into something an agent can repeat, and the README frames the payoff as generalization: recording yourself submitting one form should teach the agent to submit all of them. The audience is narrow and identifiable. You need a GitHub account with Copilot access, because the analysis step runs through the Copilot CLI, and you need to be willing to have a recorder watching your screen while you work. If neither of those is true, nothing else in the project matters.

What is actually captured, and what leaves the machine

The capture surface is wider than video. The in-app panel lists window tracking (active-app and window switches), browser URLs on macOS, screen video recorded by Chromium, short clipboard previews of copied text, and optional narration. The video is not a continuous film: low-rate snapshots are kept only when the screen changes or a heartbeat is due. That design choice is what makes a session analyzable at all, since a static hour of footage would give the model nothing to order. Narration is transcribed on-device through Whisper, in any of its 99 supported languages, with a one-time download of roughly 252 MB on first use. The privacy boundary is explicit and worth repeating because it is the sharpest constraint in the project. Recording, storage, frame extraction and transcription stay local. Nothing leaves while you record. The moment you click Analyze, the event timeline (window and document titles, URLs, clipboard previews), the extracted screen images, and the narration text go to GitHub's cloud for Copilot to process. The README warns in a callout, before every recording, not to type, paste, show, copy or narrate passwords, tokens or API keys. That is not boilerplate. Clipboard previews and window titles are exactly the fields that leak a customer name or a session token.

The pipeline: capture, control, analyze, create

The README describes four stages. Record starts a session, either from the app or from a global shortcut, Command+Shift+R on macOS and Ctrl+Shift+R on Windows. Control is a small always-on-top bar that shows capture and microphone state, so you can mute, unmute or switch microphones mid-session; a take that went wrong can be discarded with a confirmation. Analyze sends the session to Copilot, which reconstructs one overall intent and an ordered list of steps, both of which you can review and edit. Create turns an approved analysis into a Skill (a SKILL.md procedure an agent runs on demand) or an Automation (the same procedure on a schedule or trigger). The architectural decision that separates this from a macro recorder sits in the last stage: both outputs prefer the agent's native tools, such as the gh CLI or web_fetch, over replaying UI clicks. That is a deliberate bet that a described intent survives interface changes better than a recorded coordinate does. The cost of the bet is fidelity. A generated skill is a proposal about what you meant, not a transcript of what you did, and the README's own framing of generalization from one example is the assumption you are accepting.

Installing it builds from source, on purpose

Skill Recorder ships as a source release rather than a signed binary. One command downloads a pinned Node.js runtime, builds the exact release commit on your machine, and registers a Skill Recorder (Source) app you can relaunch later. Nothing is installed globally. The macOS and Ubuntu pattern is a curl of install.sh piped into bash, with the release commit pinned twice, once in the URL and once in the environment variable. Adding SKILL_RECORDER_DETACHED=1 after the pipe keeps the app running when the terminal closes. On macOS the result lands in ~/Applications; on Ubuntu it adds an application entry. Windows uses a PowerShell equivalent with install.ps1, which adds desktop and Start Menu shortcuts. Because each release pins a 40-character commit, the real command substitutes that hash into the pattern shown in the README. If you want to inspect the script before running it, or set install options, update or uninstall, the README points at INSTALL.md. For development, the requirement is Node.js 24, followed by npm ci, npm run compliance:licenses, and npm run dev, which starts Vite and launches Electron with hot reload. The README also notes a licensing boundary between local source builds and redistributable packages, detailed in INSTALL.md. That distinction matters if you plan to package the app for others rather than run it yourself.

The eval harness is the part most reviewers will skip

The repository ships a fixture-based eval suite covering the Copilot describer and the builders. Two commands matter. npm run eval scores the describer against synthetic recordings. npm run eval:builder scores skill and automation generalization. This is the only mechanism in the project for checking the claim that one recorded example generalizes to the task family, and it is the first thing a skeptical adopter should run, because the failure mode is quiet. A skill that reads plausibly but omits a conditional branch, or that substitutes a native tool call for a step that genuinely needed the UI, will not announce itself. It will fail later, on a case you did not record. The evals are described as fixture-based, so their coverage is whatever the fixtures cover; the README does not claim they are exhaustive, and you should not assume they are. Building your own fixture from a task you care about is the honest way to test the generalization claim before trusting it.

Where it is the wrong tool

Three cases. First, anything touching credentials. The README's warning is absolute, and it is not a limitation the project can engineer away, because clipboard previews and window titles are part of the analysis input. A task that requires pasting a token into a form is a task this tool should not see. Second, deterministic replay. If your requirement is that the same clicks happen in the same order every time, a generated skill that prefers native tools is the wrong output; the design intentionally abstracts away from the recorded UI. Third, platform expectations. macOS is the primary target. Windows 11 on x64 and ARM64 is supported, with a separate validation document, and Ubuntu gets an application entry from the install script. But browser URL capture is described as macOS-only, and the primary-target language suggests the macOS path is the one that gets the most attention. If your fleet is Windows-first, treat the Windows support as something to verify on your own machines rather than something the README guarantees. There is also a plain adoption cost: the analysis step requires Copilot access, so the tool is not usable in an environment where that is unavailable or disallowed.

Compared with a macro recorder, and with writing the skill by hand

The obvious alternative is a UI automation or macro recorder, which captures input events and replays them. The difference in approach is the output format. A macro recorder produces a sequence of low-level actions bound to the interface it recorded; change a button's position or a page's layout and the macro breaks. Skill Recorder produces a natural-language intent plus ordered steps, and the builders are instructed to prefer the agent's native tools such as the gh CLI or web_fetch. The trade is legibility for precision: a skill is readable, editable and portable across interface changes, but it is an interpretation, and interpretations can be wrong in ways a replay is not. The second alternative is simply writing the SKILL.md yourself. That gives you exact control and no cloud round trip, at the cost of the thing this project exists to remove, which is the effort of specifying a procedure you already know how to perform. Neither alternative is strictly better. If your task is stable and your interface is not, a hand-written skill wins. If your task is fiddly and you can demonstrate it faster than you can describe it, the recording route is the shorter path.

Maintenance, licensing and what to verify

The project is MIT licensed, which is permissive, but the README distinguishes local source builds from redistributable packages and points at INSTALL.md for that boundary, and THIRD-PARTY-NOTICES.md lists licenses for bundled dependencies. If you intend to redistribute a packaged build, read both before you do; nothing here is legal advice, and the dependency notices are the place the actual obligations live. On maintenance, the release cadence visible in the repository is three releases between early August and mid August 2026, with the last push to main in September 2026, and the latest release is v0.5.0. That is a young version number, and the source-release install model means every update rebuilds on your machine from a pinned commit, so upgrades are deliberate rather than automatic. Budget for that: an install is a build, and the Copilot CLI ships with the app rather than being something you manage separately. For a first evaluation, install a pinned release commit on one macOS machine, run npm run eval:builder to see the harness work, then record a single low-risk task with no credentials on screen and inspect the generated SKILL.md before anyone else in the team touches it.

Editorial conclusion

Adopt it if your team already has Copilot access, works mainly on macOS, and has a recurring task that is easier to demonstrate than to write up. Do not adopt it if the task touches credentials, if you need a Windows-first or Ubuntu-first workflow, or if you want a deterministic UI replay. Before rolling it out, read the in-app capture panel and the privacy note, then run npm run eval:builder against your own fixture to see whether the generalization holds for your task, because that is the assumption the whole tool rests on.

Official sources

1. Issues
2. License: MIT
3. microsoft/skill-recorder on GitHub
4. README
5. Releases