Microsoft UFO³: Galaxy orchestration on top of the UFO² Windows agent

The problem Microsoft UFO solves, and the split between UFO² and UFO³

Most desktop automation breaks when the interface changes. A script that clicks a fixed coordinate fails the moment a window moves, a dialog appears, or a button is renamed. Microsoft UFO takes the other route: a language model reads the screen and the accessibility tree, decides the next action, and issues it through Windows APIs or GUI input. The unit of work is a request in natural language, not a recorded macro.

The repository now ships two things under one name. UFO² is the Desktop AgentOS: a single Windows agent that plans at the application level and executes a sequential ReAct loop, mixing GUI actions with API calls. The README calls it stable, battle-tested and the LTS line. UFO³ Galaxy is the newer layer: a ConstellationAgent decomposes a request into a DAG of TaskStars with dependencies, and a TaskOrchestrator schedules those tasks across devices, including Windows, Linux and Android according to the comparison table.

That split matters for adoption. If you only need one Windows machine to complete a multi-step task across several applications, Galaxy adds a scheduler, a protocol and a set of moving parts you do not need. If you need a laptop, a server and a phone to cooperate on one request, UFO² alone cannot do it, and the README is explicit that cross-device collaboration is not supported there.

How the Galaxy layer works: Constellation, TaskStars and the AIP protocol

The mechanism the README describes has five parts. A request is decomposed declaratively into a dynamic DAG, where each node is a TaskStar and edges encode dependencies. That graph is not fixed at planning time: the second design principle is continuous result-driven graph evolution, meaning execution feedback can trigger controlled rewrites of the graph while it runs. The third principle covers scheduling across heterogeneous devices, matching a task to a device by capability and running independent branches in parallel with locking to avoid conflicting actions.

Communication between the orchestrator and device agents runs over AIP, the Agent Interaction Protocol, described as a WebSocket-based coordination layer with fault tolerance and automatic reconnection. The repository has a top-level aip/ directory, which is where that protocol lives. Device agents themselves are template-driven and can be augmented with MCP tools, so a new platform agent is meant to be assembled from a template rather than written from scratch.

The practical consequence is that Galaxy is a distributed system, and it inherits distributed-system failure modes. A device agent that drops offline mid-task is handled by reconnection logic, but the README does not document what happens to a DAG branch whose target device never comes back, nor whether a partially executed TaskStar is retried or abandoned. That is the kind of behaviour you want to read in the source before trusting a long-running workflow to it.

Installing Microsoft UFO and running a first task

The repository is a Python project. The README badges state Python 3.10 or 3.11, and requirements.txt pins the dependency set, including openai, langchain, pywinauto and uiautomation for Windows, plus fastapi, fastmcp and websockets for the service side. Windows-only packages are marked with sys_platform markers, so the same file installs on other platforms with those entries skipped.

Clone the repository and install the pinned requirements into a virtual environment:

git clone https://github.com/microsoft/UFO.git
cd UFO
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt

The config/ directory holds the YAML configuration. The README does not print a full config example, so read the files there rather than inventing keys. The full documentation at microsoft.github.io/UFO/ is the reference for model credentials and agent settings.

For a first run, decide which half you are using. The UFO² README is at ufo/README.md, and the Galaxy quick start is at microsoft.github.io/UFO/getting_started/quick_start_galaxy/. The README also points to a recorded demo at youtube.com/watch?v=NGrVWGcJL8o, which shows cross-device orchestration before you commit to installing the Galaxy side. Expect the first task to be a single Windows request, since that path has the shortest setup.

Where Microsoft UFO is the wrong tool

Three limits are visible from the repository itself.

First, the platform story is uneven. requirements.txt gates pywin32, pywinauto, pyautogui, uiautomation and comtypes behind Windows, and the README lists Windows, Linux and Android as current device support with more coming. The deep OS integration that makes UFO² good at Windows is exactly the part that does not travel. If your fleet is macOS, the repository gives no indication that a device agent exists for it.

Second, the two halves have different support promises. The comparison table marks UFO² as LTS and Galaxy as active development, and the migration guidance tells existing UFO² users they can keep using it. A framework in active development changes; the README does not promise API stability for the Galaxy side.

Third, the task model assumes the work can be decomposed. A request that is one long conversational interaction with a single application gains nothing from a DAG and pays the cost of orchestration. Galaxy is for work that splits into dependent steps across machines. If your automation is a single app on a single desktop, the sequential ReAct loop in UFO² is the simpler and better-supported fit.

How Galaxy differs from a plain RPA or scripting stack

The obvious alternative is conventional RPA: record or script deterministic UI actions and schedule them. The approaches differ at the point of failure. An RPA script encodes the steps a human decided on in advance, so it is fast and predictable until the interface drifts, at which point it fails and stays failed until someone edits it. Microsoft UFO decides the steps at runtime from the current screen state, so it can absorb a moved button, and in exchange it is slower per action and its behaviour depends on the model behind it.

Galaxy pushes that difference one level up. An RPA orchestrator usually schedules fixed jobs across machines; Galaxy schedules a graph that the system itself can rewrite in response to results. That is a real architectural difference, not a marketing one, and it is also the source of the uncertainty: a graph that edits itself is harder to audit than a job list. If you need a signed-off, repeatable sequence for compliance reasons, a deterministic script plus a scheduler is the more defensible choice. If you need an agent that copes with screens nobody has scripted, UFO is aimed at you.

The README also positions UFO² as usable as a Galaxy device agent, so the two are not competing options. You can run the stable Windows agent today and add the orchestration layer later.

Maintenance, licence and upgrade cost

The repository is not archived, and the last push was on 2026-09-09, so it is being worked on. Releases are frequent: v3.0.8 on 2026-08-10, 3.0.7 on 2026-06-12 and 3.0.6 on 2026-06-06. For a project of this scope, that cadence means you should expect to track it rather than pin once and forget.

The upgrade cost is concentrated in requirements.txt, which pins exact versions across a wide surface: langchain and langchain_community, openai, faiss-cpu, sentence-transformers, numpy, pandas, fastapi, fastmcp, websockets and more. Several of those are fast-moving libraries, and the pins are tight enough that a single bump can cascade. Anyone embedding UFO in a larger Python application should expect dependency conflicts and should isolate it in its own environment.

The licence is MIT, which is permissive and places few obligations on how you redistribute or modify the code. This is not legal advice, and the repository also carries a DISCLAIMER.md and a SECURITY.md; read both before pointing an agent at production systems, since an agent that drives a desktop has a broad blast radius by design.

Editorial conclusion

Adopt UFO³ Galaxy if you already run UFO² on Windows and need several machines to cooperate on one request, and you accept a moderate setup and a framework the README labels as active development. Stay on UFO² if your work is single-device Windows automation, since it is the long-term support line and Galaxy can call it as a device agent anyway. Before committing, verify three things in the repository: that the Python version you have matches the 3.10 or 3.11 badge, that your target platforms are covered by the Windows, Linux and Android support the README lists, and that the Galaxy quick start page at microsoft.github.io/UFO/getting_started/quick_start_galaxy/ still matches the code in the galaxy/ directory, because that is where the multi-device path is documented.

Frequently asked questions

Official sources

1. License: MIT
2. microsoft/UFO on GitHub
3. Project website
4. README
5. Releases