ClawGUI: GUI agent training, evaluation, and deployment

ClawGUI separates the GUI-agent lifecycle into five modules

ClawGUI is a research framework for GUI agents, covering training, evaluation, deployment, reusable skills, and phone-local execution. It is intended for engineers and researchers who need more than a model wrapper. The repository organizes the work into five named modules, allowing a team to focus on one stage without installing the entire system.

ClawGUI-RL trains policies with online reinforcement learning. ClawGUI-Eval measures GUI grounding behavior on standardized benchmarks. ClawGUI-Agent connects natural-language instructions to mobile-device control. ClawGUI-Skills stores and revises reusable procedures. ClawGUI-APP runs the full brain and GUI-agent stack directly on one Android phone. This division reflects different operational concerns, such as GPU training, benchmark execution, chat integrations, procedure management, and Android deployment.

Each module has its own environment. That design can make a narrow project easier to scope, but it also means the repository is not a single Python package with one universal setup. The repository is not archived. Its last push was on 2026-06-03. The listed application releases are `clawgui-app-v0.2.0`, dated 2026-05-18, `clawgui-app-v0.3.0`, dated 2026-05-19, and `clawgui-app-v0.3.1`, dated 2026-05-21.

ClawGUI-RL trains policies in Android environments

ClawGUI-RL is the training component. According to the README, it can use parallel Docker-based Android emulators or physical Android devices, including cloud phones. The same API is offered for these environments. This gives an engineering team two operating models: containerized emulators for parallel experimentation or real devices when device-specific behavior matters.

The module uses `GiGPO+PRM` instead of standard `GRPO`. The README describes this as a fine-grained, step-level reward mechanism. That design is relevant to GUI agents because an episode contains many intermediate actions rather than only a final outcome. A reward can therefore be connected to individual steps, but the available top-level documentation does not specify reward definitions, dataset format, or configuration files.

The documented training infrastructure also includes automatic spare-server rotation and episode recording with replay. The README presents server rotation as a failover mechanism for training continuity. Replay gives an operator a way to inspect trajectories after an episode. The repository also includes `ClawGUI-2B`, described as a 2B model trained end to end with ClawGUI-RL and `GiGPO`. The README reports 17.1 MobileWorld SR against an 11.1 baseline. This is a project-reported result, and the README does not provide enough detail to reproduce it independently.

ClawGUI-Eval standardizes inference, judging, and metrics

ClawGUI-Eval organizes evaluation into three named stages: `Infer`, `Judge`, and `Metric`. Inference runs a model against a GUI benchmark. The judging stage evaluates the generated predictions, and the metric stage produces the reported result. This separation makes the evaluation path more explicit than a single benchmark script, particularly when several model providers and benchmarks are involved.

The README lists six benchmarks: `ScreenSpot-Pro`, `ScreenSpot-V2`, `UIVision`, `MMBench-GUI`, `OSWorld-G`, and `AndroidControl`. It also lists support for more than 11 models, including `Qwen3-VL`, `Qwen2.5-VL`, `UI-TARS`, `MAI-UI`, `GUI-G2`, `UI-Venus`, `Gemini`, and `Seed 1.8`.

Inference can use a local GPU through `transformers` or a remote OpenAI-compatible API. Multi-GPU and multi-thread execution are documented, with automatic resume for interrupted work. The README reports 95.8% reproduction against official results. That figure describes the project's stated evaluation fidelity, but the README does not identify the exact model settings, benchmark subsets, or decoding configuration behind it. Those details must be checked in `clawgui-eval/README.md` before comparing results across papers.

ClawGUI-Agent and ClawGUI-Skills extend deployment

ClawGUI-Agent is the deployment component for natural-language control of mobile devices. The README states that it supports more than 12 chat platforms and includes one-command evaluation. Its role is therefore different from ClawGUI-Eval: it places an agent in a conversational interface and connects that interface to device actions.

ClawGUI-Skills adds a procedure layer. The README describes structured skill packages, retrieval, failure diagnosis, restricted revision, and reuse. The described flow is specific. A task can be associated with a stored skill, a failure can be diagnosed, the skill can be revised within the documented restrictions, and the resulting package can be used again. The repository presents this as training-free skill evolution proposed and validated in the paper.

ClawGUI-APP changes the deployment boundary again. According to the overview, the full brain and GUI-agent stack can run directly on one Android phone without a desktop coordinator, using `Shizuku`. The visible documentation does not state supported Android versions, model packaging requirements, network requirements, or permission details. Therefore, the module is relevant to on-device work, but device compatibility cannot be inferred from the top-level overview.

The initial ClawGUI setup begins with module selection

The available top-level README provides a clone operation but does not provide a complete environment installation command. The repository then directs users to the documentation for the module they intend to use.

git clone https://github.com/ZJU-REAL/ClawGUI.git
cd ClawGUI

After the command completes, the working directory should contain the `ClawGUI` repository and its top-level module directories, including `clawgui-rl`, `clawgui-eval`, `clawgui-agent`, `clawgui-skills`, and `clawgui-app`.

A first real use then depends on the module. For online training, start with `clawgui-rl/README.md`. For benchmark execution, start with `clawgui-eval/README.md`. For natural-language mobile-device control, use `clawgui-agent/README.md`. For skill retrieval and revision, use `clawgui-skills/README.md`. For phone-local execution, use `clawgui-app/README.md`.

No additional install command is reproduced here because the top-level README does not state the module dependencies or setup commands. A Python 3.12 badge identifies an expected interpreter version, but it does not install GPU packages, Docker support, Android tooling, model weights, or remote API configuration. This is a concrete limitation for a new user: the repository announces independent module environments without presenting their full installation requirements in the visible top-level guide.

ClawGUI differs from a narrow agent integration layer

A practical alternative is to combine separate libraries for policy training, benchmark execution, and device control, then write adapters between them. That approach may be better for a small application that needs only one function. It avoids taking on a framework that includes Docker environments, Android devices, GPU inference, remote APIs, chat platforms, skills, and phone-local execution.

The cost of that alternative is integration work. The team must define how model outputs, screenshots, actions, benchmark results, and device-control calls move between components. ClawGUI's approach is to keep those stages in related modules and name them according to their roles. ClawGUI-RL handles training, ClawGUI-Eval handles measurement, ClawGUI-Agent handles natural-language device interaction, ClawGUI-Skills handles reusable procedures, and ClawGUI-APP handles phone-local execution.

This breadth also creates a boundary. ClawGUI should not be treated as a small, stable runtime with one dependency surface or a complete production service. The README does not promise a compatibility contract between modules, a unified release version, or a single deployment command. A team that needs a minimal prompt-and-action library may find the dedicated agent layer more suitable than the repository as a whole.

Apache-2.0 and release history affect adoption planning

ClawGUI uses the `Apache-2.0` license. The repository includes a `LICENSE` file, and the README identifies Apache 2.0. License interpretation for a particular deployment requires separate review, but the project license itself is clear.

The maintenance record is recent. The project is not archived and the last push was on 2026-06-03. The listed `clawgui-app` releases are `clawgui-app-v0.2.0` on 2026-05-18, `clawgui-app-v0.3.0` on 2026-05-19, and `clawgui-app-v0.3.1` on 2026-05-21. These dates show activity around the application module, but they do not establish the release schedule of the training, evaluation, agent, or skills modules.

Upgrade planning also depends on external artifacts and services. The README links to the `ClawGUI-2B` model on HuggingFace and ModelScope. ClawGUI-Eval can use local GPU inference or a remote OpenAI-compatible API. The README does not document artifact compatibility across module releases, model-version pinning, or a migration guide. The concrete first checks are therefore the selected module README, the linked model requirements, the Android device prerequisites, and the local-versus-remote evaluator configuration.

Editorial conclusion

Adopt ClawGUI when you need related tooling for GUI-agent reinforcement learning, benchmark execution, natural-language device control, and Android deployment. Do not assume the repository is a single turnkey installation. First verify the selected module's README, model requirements, Android prerequisites, and dependency setup, because the available top-level instructions only show how to clone the repository.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. ZJU-REAL/ClawGUI on GitHub