OmniAct: a lightweight multi-environment agent for Windows, browsers and the CLI

The gap OmniAct is built around: one goal that crosses browser, shell and desktop

Most agent projects pick one surface. The README makes this its opening argument: a coding agent stays inside a repository, a browser agent stays inside web pages, a desktop agent stays inside applications, while real work moves between them. OmniAct is aimed at that seam. The intended user is someone who has a task that starts in a browser (a portal, a form, a dashboard), continues in a shell (a script, a Codex session, a file), and sometimes ends in a Windows application. The README states the project is in early development and that the public repository keeps the runtime core focused on task orchestration, worker boundaries, task artefacts and a readable execution trace. That framing matters when you decide whether to try it: you are adopting an orchestration layer, not a finished product.

TeamLeader, three workers, and a task directory as the shared state

The architecture is a star, not a mesh. A user goal goes to TeamLeader, which decides whether to reason directly or dispatch a bounded subtask to a Browser Worker, a CLI Worker or a Desktop Worker. Each worker owns its own observation and action loop for its environment. The README is explicit that workers do not share internal state and do not depend on each other directly. Coordination happens through the filesystem: every hand-off is written into a task directory.

The layout the README gives is a task file, a JSONL trace, a human-readable trajectory, a per-worker folder containing request.md, result.md and an artifacts directory, and a final result.md. That is the most interesting design decision in the project. Instead of a message bus or a shared memory object, the contract between agents is a set of Markdown files on disk. It makes debugging cheap, because you can read what TeamLeader asked for and what the worker produced without instrumenting anything. It also means the README's claim that workers hand over "Markdown summaries and concrete artefacts" rather than a bare "done" is a structural property, not a stylistic promise. The cost is that anything you want to inspect has to be serialised to disk, and long-running tasks accumulate files.

Installing OmniAct on Windows and running the TeamLeader example

The README's requirements list is narrow and worth reading before you install anything: a Windows environment with Microsoft Edge installed, Conda or another Python environment manager, Python 3.11, an OpenAI-compatible endpoint that supports native tool calling for TeamLeader, a vision-capable OpenAI-compatible endpoint for the Browser Worker, and Codex CLI if you intend to use the CLI Worker. The install commands in the README are PowerShell, so the tutorial below is Windows-only by construction.

Create the environment and install the pinned dependencies. The repository ships a requirements.txt with exact versions for most packages, which is what the README's install step uses.

conda create -n omniact python=3.11
conda activate omniact
python -m pip install -r requirements.txt
Copy-Item .env.example .env

After that, .env exists in the project root and you fill in the model endpoints. The .env.example file groups variables by agent: TEAM_LEADER_API_KEY, TEAM_LEADER_BASE_URL and TEAM_LEADER_MODEL for the orchestrator, and the BROWSER_WORKER_ equivalents for the browser. Two variables decide whether the optional workers appear at all.

TEAM_LEADER_API_KEY=
TEAM_LEADER_BASE_URL=
TEAM_LEADER_MODEL=

BROWSER_WORKER_API_KEY=
BROWSER_WORKER_BASE_URL=
BROWSER_WORKER_MODEL=
BROWSER_WORKER_SESSION_MODE=inherited

DESKTOP_WORKER_ENABLED=false
CLI_WORKER_ENABLED=false

Setting DESKTOP_WORKER_ENABLED=true exposes desktop dispatch; setting CLI_WORKER_ENABLED=true exposes the CLI tools. With both false, which is the default in .env.example, you get TeamLeader plus the Browser Worker. Then run the example from the project root.

python examples\run_team_leader.py

According to the README, results land in tasks/<task-id>/ with the task description, worker requests and results, screenshots, file artefacts and a readable trace. The same README documents that when a worker pauses for human takeover, the example accepts takeover <worker-id>, resume <worker-id> and cancel <worker-id> at its prompt. That interactive loop is the part to try first, because it is where the orchestration either holds together or does not.

BROWSER_WORKER_SESSION_MODE=inherited and the Windows-only boundary

Two constraints shape what OmniAct can be used for. The first is the platform. The README lists a Windows environment with Edge as a requirement, and the dependency list backs that up: pywin32, pywinauto, uiautomation, comtypes and pyautogui are all Windows desktop automation libraries, and the Desktop Worker is described as operating Windows applications. There is no documented Linux or macOS path. If your fleet is Linux, this project is the wrong tool regardless of how well the orchestration fits your problem.

The second is session handling. BROWSER_WORKER_SESSION_MODE accepts inherited and clean. The README states that inherited copies the current Edge Profile into an isolated worker browser, reusing the login state without taking over the browser window the user is working in. That is a sensible compromise between a blank browser that cannot log in and a worker that fights the human for the same window. It also means the worker operates on a copy of a profile that contains live credentials, so the isolation boundary is the profile copy, not the account. The README does not document what happens if the profile is locked or being written at copy time, and it does not describe a rollback path for a partially completed task. Treat a failed run as something you inspect in the task directory, not something the system undoes for you.

How OmniAct differs from a single-surface browser agent

The obvious alternative is a browser-only automation agent, and the difference is not the model or the prompt, it is where the work is allowed to happen. A browser agent keeps one observation and action loop against a page, so its state is the page and its failure mode is a selector that moved. OmniAct instead keeps separate loops per environment and makes the boundary between them an explicit artefact: request.md going out, result.md and an artifacts folder coming back. You gain the ability to hand a subtask to Codex CLI in a working directory, or to drive a Windows application through UI automation, without teaching the browser agent about either. You lose the tightness of a single loop. A browser-only agent can retry a click immediately because it holds the page; OmniAct's workers do not share internal state, so a retry across a worker boundary goes back through TeamLeader and through the filesystem. For a task that never leaves the browser, the extra layer is overhead.

Licence, third-party components and what upgrading costs

OmniAct is released under the MIT License, and the README points to THIRD_PARTY_NOTICES.md for attribution and licence information on modified open source components it includes. If you redistribute the project or ship it inside a product, that notices file is the one to read, because the MIT licence on OmniAct itself does not cover the bundled components. This is not legal advice; have someone check the notices against your own distribution model.

On upgrade cost, the repository gives you two signals. There are no releases, so there is no version to pin and no changelog to read before pulling. And requirements.txt pins most dependencies to exact versions (openai==2.26.0, pydantic==2.12.5, cdp-use==1.4.5, fastmcp==2.11.3 and so on) while a handful use lower bounds (rich>=15.0.0, PyYAML>=6.0.3, pandas>=3.0.5, flask>=3.1.3, pywin32>=310). That mix means a fresh install is reproducible for the pinned packages and open-ended for the rest. The last push to the repository was on 2026-08-31. There is no documented upgrade procedure, and the README does not describe a migration path between versions, which is consistent with a project that has not cut a release yet.

Editorial conclusion

Adopt OmniAct if you already run Windows with Edge and Codex CLI, and you want the hand-off between a browser, a shell and a desktop app recorded as files you can read afterwards. Do not adopt it if you need a Linux or macOS runtime, or a supported release with a version number: the README labels the project early development and the repository has no releases. Before you commit, verify two things in a throwaway task directory: that BROWSER_WORKER_SESSION_MODE=inherited copies the Edge profile without hijacking the window you are using, and that your TeamLeader endpoint really supports native tool calling, since the README lists that as a requirement rather than a suggestion.

Frequently asked questions

Official sources

1. Issues
2. Iwtbs27/collab-agent on GitHub
3. License: MIT
4. README