Ui.Vision RPA: a browser extension that runs Selenium IDE macros and drives the desktop through the browser

What Ui.Vision actually replaces

The project targets a specific gap: browser automation that a non-developer can record, edit and rerun without a test framework, a CI runner or a language runtime. The README describes it as "Modern Robotic Process Automation, Selenium IDE compatible" and states it is free for personal and commercial use. That compatibility claim is the load-bearing part. Selenium IDE users get an import and export path, so an existing .side style macro is not thrown away, and the command vocabulary is documented on ui.vision/rpa/docs/selenium-ide. The audience is therefore narrower than "RPA" suggests. It is people who automate a browser they can see: filling internal web forms, pulling figures out of a portal that has no API, running the same click sequence across a list of rows. If your work already fits Playwright or Puppeteer, this project is not competing for it.

Two execution models in one extension: the command table and uiv.*

Macros exist in two forms. The classic form is a command table, the Selenium IDE lineage, where each row is a command, a target and a value. The newer form is a JavaScript API, and the release notes for V10.0.31 name it directly: "AI macro creation, uiv.* API and the MCP bridge". The README lists what the API exposes: DOM and visual finders (uiv.$, uiv.findImage, uiv.ocr.findText, uiv.ai.find), input tiers (uiv.page.*, uiv.browser.*, uiv.desktop.*), plus screenshots, tabs, CSV and downloads. Two design details matter more than the method list. First, every call auto-waits and throws a real JavaScript exception on failure, so try/catch behaves as a JavaScript developer expects rather than returning a silent false. Second, the input tiers are a deliberate escalation: page-level DOM work, then browser-level control, then desktop-level input. That third tier is what lets a browser extension reach outside the browser, and it is also the tier most likely to be blocked by OS permissions or by the fact that a background window is not focused. The README does not describe the fallback behaviour when desktop input fails, so treat that as something to establish on your own machine before designing around it.

The uiv-commands.md bridge and what it admits

The repository ships uiv-commands.md, which the README says "maps every classic command-table command to its uiv.* equivalent, or to the uiv.run('command', 'target', 'value') legacy bridge where no native method exists yet." Read that sentence as documentation of an incomplete migration. Some commands have native JavaScript methods; the rest are reachable only through a stringly-typed shim that passes a command name, a target and a value. That is honest, and it is also a constraint: a macro written entirely in uiv.* still depends on the legacy command layer wherever uiv.run appears, so the two execution models are not cleanly separable. If you are porting a large command table, check uiv-commands.md for how many of your rows land on uiv.run before you commit to a rewrite.

Getting a build out of the repository

The README is explicit that building is optional: "Building the extension is not required if you 'only' want to use it." The recommended path is the Chrome, Edge or Firefox store listing, with older versions in the archive at ui.vision/rpa/archive. For developers, the stated toolchain is Node V20.11.1 and NPM V10.2.4. Install dependencies with npm i -f (the -f is in the README, and it matters because it forces past peer dependency conflicts). Then npm run build produces the Chrome and Edge bundle in dist, and npm run build-ff produces the Firefox bundle in dist_ff. For iterative work, npm start runs webpack in watch mode against dist, and npm run start-ff does the same for dist_ff. Loading the result is manual: chrome://extensions then "Load unpacked" for Chrome and Edge, or about:debugging then "Load Temporary Add-on" for Firefox. The extension declares Manifest V3, per the repository layout note for extension/manifest.json, which is worth knowing because MV3 constrains background execution and is a common source of surprise for people porting older automation.

The AI layer is a prompt file, not a black box

The most interesting architectural choice here is that the AI assistant's instructions are published. The README says the AI system prompt page contains "the complete uiv.* API reference plus the automation recipes the AI assistant works from", that it is "the same text the AI in the side panel uses", and that you can override it under Settings > AI > System Prompt. The same material is offered as a single plain-text file at ui.vision/llms-full.txt for feeding into an assistant. The page is generated from src/services/ai/macro_agent/service.ts by npm run gen:ai-prompt-page, and the README instructs maintainers to run that at release time. This is a maintainable arrangement: the reference an AI reads and the reference a human reads cannot drift apart without the build step being skipped. It also means the quality of AI-generated macros is bounded by a file you can read before trusting the output. If the assistant produces a uiv.findImage call that does not exist, the prompt is where you check.

Where it stops being the right tool

The extension model is the limitation. Ui.Vision runs as a browser extension, so it runs where a browser runs, with a profile, a window and a user session. There is no headless mode described in the supplied material, and nothing about a server-side runner or a CI integration. If your requirement is a test suite that executes on every pull request in a container, this is the wrong shape of tool, and no amount of uiv.* API surface changes that. A second constraint is the desktop input tier. Driving uiv.desktop.* implies the OS is willing to accept synthetic input into whatever window is in front, which interacts badly with screen locking, remote sessions and machines where a human is also working. A third is licence clarity: the repository's licence field reads NOASSERTION, which means GitHub could not map the file to a known identifier. The README says the software is free for personal and commercial use, but a claim in a README and an SPDX identifier are different kinds of statement. If your organisation has a licence gate, resolve that before adoption rather than after.

The alternative, and the actual difference

The obvious comparison is Selenium IDE itself, since Ui.Vision imports and exports its format. The difference is what sits around the recorder. Selenium IDE is a recorder and player for the browser; Ui.Vision adds an OCR text finder (uiv.ocr.findText), an image finder (uiv.findImage), a desktop input tier, CSV handling, and a JavaScript API with auto-waiting and thrown exceptions. In other words, Selenium IDE gives you the command table; Ui.Vision gives you the command table plus escape hatches for pages that have no usable DOM and for steps that leave the browser. The trade is that you accept an extension runtime and its constraints, where a Selenium or Playwright stack runs under your own process and can be containerised. Choose based on whether the hard part of your automation is the browser or the machine the browser sits on.

Maintenance, upgrades and the MCP bridge

Three releases in the supplied list span roughly three months: V9.6.0 in May 2026 with improved OCR commands, V9.6.1 in July with an "Unlimited hard-drive storage mode", and V10.0.31 in August with the uiv.* API and the MCP bridge. That cadence implies the extension updates through the store channel, which for a browser extension is not something you schedule. If you pin behaviour, pin it by keeping the macro definitions in version control, which the repository supports through import and export rather than through any stated snapshot mechanism. The MCP bridge lives in mcp/ with its own README, and the README describes it as letting "Claude Code and other MCP clients create, edit and run Ui.Vision macros in your browser". That is the integration to evaluate if you want an external agent driving macros; it is also a second component with its own setup, so budget for it separately from the extension itself. On the licence question, the NOASSERTION field is the fact to act on. Read the licence file in the repository and, if you are shipping commercially, get your own answer rather than relying on this article or the README's summary.

Editorial conclusion

Adopt Ui.Vision if your automation lives on a real desktop, needs a visible browser, and your team already writes Selenium IDE command tables or is willing to learn the uiv.* API from the published system prompt. Do not adopt it if your pipeline is headless CI, if you need the same macro to run inside a locked-down browser profile, or if you require a permissively named licence before shipping. Verify three things first: the exact licence terms, since the repository carries NOASSERTION rather than a recognised identifier; whether the desktop input tier (uiv.desktop.*) can reach the applications you care about on your OS; and whether the MCP bridge in mcp/README.md is the integration path you want, because that is where Claude Code and other MCP clients attach.

Official sources

1. A9T9/RPA on GitHub
2. Issues
3. Project website
4. README
5. Releases