OpenCUA: an open stack of computer-use models, data and annotation tooling

Four pieces that are usually bought separately

OpenCUA is presented as a framework rather than a single model, and the interesting part is what it bundles. The README lists four components.

AgentNet is the dataset, described as the first large-scale computer-use task dataset spanning 3 operating systems and 200+ applications and websites. AgentNetTool is annotation infrastructure that captures human computer-use demonstrations. AgentNetBench is an offline evaluator that benchmarks model-predicted low-level actions against ground-truth trajectories. The OpenCUA Models are end-to-end computer-use foundation models that produce executable actions in computer environments.

That combination is the real differentiator for most readers. A dataset without a way to collect more data forces you to crawl for annotations. An evaluator that needs a live desktop environment to judge a single action is slow to iterate against. An offline evaluator that compares predicted low-level actions with recorded trajectories lets you rank checkpoints without booting an OS.

The project describes itself as a NeurIPS 2025 Spotlight, and the paper and project page are linked from the README.

The model weights are modified Qwen checkpoints, not stock ones

This is the section to read twice. The README places a warning box for the Qwen-based models, covering OpenCUA-7B, OpenCUA-32B and OpenCUA-72B, and states that to align with the training infrastructure the model was modified in two places. Multimodal Rotary Position Embedding, M-RoPE, has been replaced with 1D RoPE, and the same tokenizer and chat template as Kimi-VL are used.

The consequence is stated without softening: do not use the default transformers and vllm classes to load the model. Tokenizer and chat template should be aligned if training the models.

In practice this means a checkpoint that looks like a standard Hugging Face causal LM will not behave like one. If you load it with the default classes, the positional encoding assumptions and the tokenisation of image regions and action tokens will not match what the weights were trained on, and you get degraded output that is hard to diagnose because nothing raises an error. Anyone planning to fine-tune these models has to reproduce the tokenizer and template configuration exactly, which is a real integration cost.

The model collection is hosted on Hugging Face, and the README links a demo space alongside it.

Getting the weights and running a server

The installation path uses conda with Python 3.10 and a requirements file named requirement.txt:

conda create -n opencua python=3.10
conda activate opencua
pip install -r requirement.txt

The weights are pulled with the Hugging Face hub client:

from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="xlangai/OpenCUA-7B",
    local_dir="OpenCUA-7B",                
    local_dir_use_symlinks=False  
)

For deployment the README recommends vLLM and states the requirement of vllm>=0.12.0 with --trust-remote-code. The single GPU form for the 7B model looks like this:

# OpenCUA-7B (single GPU)
vllm serve xlangai/OpenCUA-7B \
  --trust-remote-code \
  --served-model-name opencua-7b \
  --host 0.0.0.0 \
  --port 8000

The 32B model is served with tensor parallelism across four GPUs, and the 72B model is shown with a tensor parallel size of 2 and a data parallel size of 4 for four instances on eight GPUs, with --gpu-memory-utilization 0.85. The README notes that the parallel and memory flags should be adjusted to the hardware at hand.

vLLM support was contributed upstream by the Meituan EvoCUA team, and the README points to the vLLM Serve section of the model README for further usage notes.

Grounding runs against a server or straight from transformers

The GUI grounding example is the quickest way to see whether the model does anything useful on your own screenshots. The README starts the server first, using OpenCUA-7B as the example, and then runs the grounding script:

cd ./model/inference/
python vllm_inference.py

There is also a path that avoids a server entirely:

python huggingface_inference.py

Both live under model/inference/, alongside vllm_inference.py which the README cites for more examples and inference code. The model directory also holds a README covering vLLM serve usage.

Grounding is the capability the README quotes specific figures for. The October 2025 update announcing OpenCUA-72B states it ranks first on the OSWorld-Verified leaderboard, has strong grounding ability with 37.3% on UI-Vision, and 60.8% on ScreenSpot-Pro. The introduction quotes an average success rate of 45.0% on OSWorld-Verified for OpenCUA-72B, presented as a new state of the art among open-source models.

Treat those as the authors' reported numbers on their model card and paper rather than as independent measurements.

AgentNet trajectories, a data viewer and metadata

The dataset side has grown tooling around it. A December 2025 update records that AgentNet dataset trajectories can be viewed online through the AgentNet Data Viewer, and that the code in data/vis/ can visualise your own trajectory data, with usage instructions in the vis README. The same update points to a metadata json file on the Hugging Face dataset repository, described as a summary of the metadata of AgentNet.

The AgentNet dataset lives on Hugging Face under xlangai/AgentNet, and the AgentNetTool project has its own site. AgentNetBench lives in the AgentNetBench directory of this repository.

The repository layout supports the four-part structure: assets, data, evaluation, model and tool at the top level, alongside a CLAUDE.md file that carries agent instructions for contributors. Evaluation code is separate from the model code, which is the layout you want if you are running offline action-level evaluation rather than full end-to-end episodes.

One thing the README does not spell out is the licence on the dataset itself. The repository carries the MIT licence and states that all content and source code for the package are subject to it, but the dataset is distributed through a separate Hugging Face repository, and you should read the terms there before redistributing the data or the recorded demonstrations.

Where this stack stops being the right tool

The clearest limit is the loading path. A model that requires a custom rope configuration and a foreign tokenizer and chat template is fine inside the project's own scripts, and wrong the moment your inference stack assumes standard Hugging Face conventions. Teams with an existing vLLM or transformers deployment for other models will find that OpenCUA weights are the odd one out, and the README does not document a compatibility shim for the default classes.

Second, the reported numbers come from the model authors on a leaderboard. The 45.0% average on OSWorld-Verified, 37.3% on UI-Vision and 60.8% on ScreenSpot-Pro describe one checkpoint family on tasks those benchmarks define. They say little about how the models behave in a workflow with unusual screens, non-English interfaces, or applications the dataset never covered.

Third, AgentNetBench is an offline evaluator working at the level of low-level actions against recorded trajectories. It is a ranking tool for candidate models, not a substitute for running the agent end to end in a real environment, so it will not tell you whether a task completed.

Fourth, the footprint. A 72B checkpoint served with tensor parallel size 2 and data parallel size 4 across eight GPUs is a serving bill, not a hobby. The 7B model with an exllamav2 quantization produced by a third party lowers that bar, but the official path still assumes accelerator hardware.

Maintenance pace and what the release history shows

The repository is not archived. The last push was on 2026-05-25, and the single tagged release is v1.0.0, named the AgentNet-Tool Release, dated 2025-08-11. Release tags are therefore less informative than the updates list, which records dated entries through 2026: vLLM gaining full support for the 7B, 32B and 72B models on 2026-01-17, the AgentNet data viewer and visualisation code on 2025-12-17, and upstream vLLM support for OpenCUA-7B on 2025-11-28.

Other entries show community packaging: a third party exllamav2 quantization of OpenCUA-7B for lower VRAM use, dated 2025-10-12, and the OpenCUA-72B model announcement on 2025-10-03. The paper and project page were released on 2025-08-13, days after the v1.0.0 tag.

Licensing is MIT, which is permissive and places no copyleft requirement on code you derive from the framework. The MIT terms are the ones in the LICENSE file at the repository root, and they are separate from any terms attached to the dataset on Hugging Face.

Work continues on the repository, but the gap between the last push and today means you should pin a specific commit and read CHANGELOG entries in the repositories you actually consume.

Editorial conclusion

OpenCUA fits teams that need a complete computer-use stack rather than a single model: a labelled dataset to train on, a way to capture new demonstrations, an offline evaluator, and checkpoints they can serve under their own licence. It does not fit work that must load the checkpoints with stock transformers or default vLLM classes, because the 1D RoPE change and the Kimi-VL tokenizer and chat template make the stock path wrong for these weights. Before building on it, verify that the model revision you pick exposes the tokenizer and chat template your training code expects, check the AgentNetBench directory for the action space your task uses, and read the last push dated 2026-05-25 to see what has changed since the v1.0.0 release of 2025-08-11.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. xlang-ai/OpenCUA on GitHub