microsoft/fara: running the Fara1.5 computer use agents from a local CLI

The gap Fara1.5 fills: browser automation without an accessibility tree

Most browser automation stacks depend on a structured view of the page. Playwright selectors, Selenium locators, accessibility trees, DOM snapshots: each of these gives the agent a parsed representation it can reason over, and each of them breaks when the page renders a canvas, a shadow DOM component, or a layout that shifts between runs. Fara1.5 takes the opposite route. The README states the models operate visually, taking actions like scrolling, typing and clicking on directly predicted coordinates, with no accessibility trees or separate parsing models in the loop. The target user is therefore an engineer who has already hit the ceiling of selector-based automation and is willing to trade determinism for generality. It is not aimed at teams that need reproducible, auditable clicks on a fixed internal form; a CSS selector is cheaper and more reliable there. The repo's own framing puts the models alongside OpenAI Operator and Gemini 2.5 Computer Use as the comparison class, which tells you the intended workload is open-ended web tasks rather than scripted regression tests.

The observe-think-act loop and what the model actually emits

The architecture described in the README is a loop, not a pipeline. On each turn the model receives a screenshot of the browser plus the conversation history. It reasons about task state and emits an action. Those actions fall into three kinds according to the documentation: mouse and keyboard inputs on directly predicted coordinates, web searches, and context management operations. Context management matters more than it sounds. A screenshot-per-step agent burns tokens fast, and without an explicit mechanism to drop or summarise earlier turns the context window becomes the binding constraint on task length. The README mentions the operation but does not spell out the policy, so how aggressively history is trimmed is something you would have to read out of the code or the paper rather than the front page. Note also what is absent: no separate parsing model sits between the screenshot and the action, which removes a failure mode (parser drift) and adds one (the model must get pixel coordinates right on the first attempt).

Three sizes, one training pipeline, and what the published numbers cover

Fara1.5 ships at 4B, 9B and 27B parameters, all built on Qwen3.5 and trained with supervised finetuning on data from the FaraGen1.5 pipeline, which the README describes as a scalable set of environments, solvers and verifiers. The release notes report 63.4% on Online-Mind2Web and 86.6% on WebVoyager for Fara1.5-9B, and 72.3% on Online-Mind2Web for Fara1.5-27B. Those are the project's own figures, published in the accompanying paper (arXiv 2606.20785), and they are browser-use benchmarks specifically. Nothing in the supplied material reports desktop application control, terminal tasks, or OS-level benchmarks, so treat the computer use label as browser-first in practice. The size spread is the practical decision axis: the 4B and 9B variants are the ones that fit latency and privacy budgets, and the README frames compact size as the reason for reduced latency and improved privacy, which only holds if you host the weights yourself rather than calling a shared endpoint.

Getting fara-cli to run: the four steps the README gives

The install path is short. Clone the repository, create a virtual environment, install the package in editable mode, and pull the Playwright browser binaries:

git clone https://github.com/microsoft/fara.git cd fara python3 -m venv .venv source .venv/bin/activate pip install -e . playwright install

The playwright install step is not optional decoration. Without it the browser the agent drives does not exist on disk. Next you need a model endpoint. The README's example deploys Fara1.5-9B from the Microsoft Foundry catalog and writes the connection details into a JSON file, azure_foundry_config.json, with three keys: model, base_url and api_key. The base_url in the sample points at an inference.ml.azure.com host. Then the run itself is a single command:

fara-cli --task "whats the weather in new york now" --endpoint_config azure_foundry_config.json

Two operational notes from the README. On Windows it recommends WSL2 rather than native execution. And if you want a sandboxed browser with auditable action logging and user prompts at critical points, the documented route is Magentic-UI, a separate repository, where you supply the same kind of endpoint but drive the agent through that UI instead of fara-cli.

The limitation the README does not hide: the repo is not the model

Every command above assumes you already have somewhere to send inference requests. The repository contains the client, the CLI and the evaluation tooling; the weights live on Hugging Face, and the hosted variants live in the Foundry catalog. There is no local inference path documented in the material supplied here. That has three consequences worth stating plainly. First, offline or air-gapped use is not covered by anything in this README. Second, cost scales with steps, because each step sends a screenshot, and a long web task is many steps. Third, the privacy argument for the small models only applies if you deploy the weights yourself; pointing fara-cli at a shared endpoint sends your screenshots to that endpoint. A second limitation is structural: coordinate prediction is brittle on pages that scroll, lazy-load, or render at unexpected zoom levels, and the absence of an accessibility tree means there is no secondary signal to recover from a misclick. The material does not document retry or recovery behaviour, so assume you will be writing that yourself.

WebTailBench and CUAVerifierBench: the part that is easy to overlook

The evaluation side of this repo is arguably more reusable than the agent side. WebTailBench is a first-class benchmark here: the loader auto-downloads tasks and rubrics from the microsoft/WebTailBench dataset on Hugging Face, and the README instructs you to use the refreshed test_v2 split, with a V1 to V2 diff published at a microsoft.github.io/fara URL. The official judge is the Universal Verifier, named MMRubricAgent in the code. The reproducibility entry points are webeval/scripts/webtailbench.py for running the benchmark and webeval/scripts/verify_trajectories.py for stand-alone re-scoring of trajectories. Separately, CUAVerifierBench is a human-annotated benchmark for the verifiers themselves, with two splits, fara7b_om2w_browserbase (106 trajectories, roughly two reviewers each) and internal (154 trajectories from a heldout aurora-v2 task suite), carrying per-judge UV-blind and UV-informed labels alongside Universal Verifier and legacy verifier outputs. If your actual problem is scoring agent trajectories rather than producing them, this is the material to read first.

How this differs from driving a browser with Playwright plus an LLM

The obvious alternative is a general-purpose model wired to Playwright through a tool-calling interface, where the model emits selectors or DOM queries and the framework executes them. The difference in approach is where the grounding lives. In the Playwright-plus-LLM pattern, grounding sits in the page's structure: the model picks an element, the framework resolves it, and failures are usually diagnosable because you can print the selector that did not match. In Fara1.5, grounding sits in the pixels: the model predicts coordinates and the action fires there. That makes the agent indifferent to whether the target is a button, a canvas, or a PDF viewer, which is exactly the case Playwright handles badly. It also removes your ability to explain a failure from a log line, because a wrong coordinate is just a number. The trade is legibility for coverage. A second alternative is a hosted computer use API from a proprietary vendor, which the README itself benchmarks against; the difference there is deployment control, since Fara1.5-4B and Fara1.5-9B can be self-hosted while a closed API cannot.

Licence, maintenance and what to check before you depend on it

The repository is MIT licensed, which is permissive for the code: the CLI, the loaders and the evaluation scripts can be reused and modified with few obligations. That licence does not automatically extend to the model weights. The Fara1.5 models are built on Qwen3.5 and distributed through Hugging Face and Microsoft Foundry, and the README does not state which terms govern the weights. Anyone planning commercial deployment should read the model card and the Foundry listing rather than assuming MIT covers the whole stack. This is a description of what the material says, not legal advice. On maintenance, the repository is active rather than archived, and the dated update log runs from April to July 2026, covering the CUAVerifierBench release, the WebTailBench promotion to a first-class benchmark with the test_v2 split, and the Fara1.5 model release. That cadence means benchmark splits can move under you, which is the concrete upgrade cost here: if you pin results to WebTailBench V1, they stop being comparable once you re-run against V2. Pin the split, record which verifier produced a score, and re-check the diff page when you bump the repository.

Editorial conclusion

Adopt the fara repo if you already have a model endpoint and want a small, MIT-licensed harness for visual browser agents, or if you need the WebTailBench and CUAVerifierBench scoring paths that live in webeval/scripts. Do not adopt it if you expected the repository to give you a working agent without a hosted model: there are no local weights in the tree, and fara-cli will not start without a valid base_url and api_key. Before committing, confirm three things in your own environment: that the Qwen3.5-derived weights carry terms you can live with for your deployment, that your target sites tolerate coordinate-based clicking without an accessibility tree, and that the test_v2 split of WebTailBench is the one you score against, since the V1 numbers in older write-ups are not comparable.

Official sources

1. Issues
2. License: MIT
3. microsoft/fara on GitHub
4. Project website
5. README