ClickClickClick: a planner/finder split for Android and desktop automation, with LLM choice left to you
Autonomous Android and computer use using any LLM (local or remote)
At a glance
- What is it?
- InstaVM's MIT-licensed Python framework separates task planning from on-screen element finding, then drives Android over adb or macOS directly. The README calls the code highly experimental and names Gemini 3.1 Flash-Lite as the current best pairing for both roles.
- Who is it for?
- Adopt it if you want to experiment with swapping planner and finder models independently, or if you need a self-hosted path through Ollama, and you can tolerate a README that labels the code highly experimental. Do not adopt it as production infrastructure for unattended automation, and do not expect Ollama to carry the finder role reliably.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Activity is slowing. The repository last received commits 6 months ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem ClickClickClick is aimed at
Most computer-use agents are sold as a single model doing everything: read the screen, decide the next step, decide where to click. ClickClickClick splits that into named parts. The README's own to-do list names three components: planner, finder, executor. The planner turns a task prompt into a plan. The finder locates the UI element to interact with. The executor performs the action on the device. That split is the whole point of the project, and it is what makes the model recommendations interesting rather than cosmetic.
Who it is for: someone with a Python environment, an Android device reachable over adb or a macOS machine, and an opinion about which model should do which job. The README's demo tasks are ordinary desktop chores, drafting a Gmail message, reading Google Maps for bus stops in a named town, starting a specific time-control game on lichess. These are the tasks the framework is being pointed at. It is not a scraping library and not a test runner. It is a way to hand a sentence to a model and have something click on your behalf.
How the planner, finder and executor fit together
The data flow visible in the material is a loop over a screen image. The finder consumes a screenshot and returns an element to act on. The image is passed at a configurable quality, and the README is explicit that lower values reduce image size for faster processing. The --image-quality flag takes a percentage from 1 to 100 and defaults to 100. That single knob is the main latency lever the documentation exposes, which tells you the pipeline is image-bound: every step of the finder costs a vision inference over the current screen.
The planner and finder are separately selectable. The CLI exposes --planner-model and --finder-model, each accepting openai, gemini or ollama, and the REST endpoint takes planner_model and finder_model in the request body with the same three values. That means you can plan with a local model and find with a hosted one, or the reverse, without touching code. The README states the best current result comes from using Gemini 3.1 Flash-Lite as both planner and finder, which is a pairing recommendation rather than a claim about the architecture.
Platform selection is a separate axis. --platform accepts android or osx, defaulting to android. The REST body mirrors this with a platform field and the same two supported values. Anything outside those values is documented to return 400 with a detail string naming the unsupported platform or model.
Getting it running: setup, run and the API
Installation follows the standard Python path in the README. Clone the repository, create a virtual environment with python3 -m venv venv, activate it, then pip install -r requirements.txt. Model settings go in config/models.yaml, and the README instructs you to export the keys specified in that yaml file. There is no separate secrets mechanism described.
Before any task runs, the README says you must configure planner and finder models using the setup command: python main.py setup. You are prompted to choose the planner and finder and to supply any necessary API keys. Tasks then run with python main.py run "<task-prompt>". A full example from the README combines the flags: python main.py run "Open Google news" --platform=android --planner-model=openai --finder-model=gemini. The local variant shown is python main.py run "Open Reddit" --platform=android --planner-model=ollama --finder-model=gemini --image-quality=45.
There is also a packaged CLI path. The README shows pip install <repo-whl> followed by ./click3 run open google.com in browser. Note that the wheel placeholder is unresolved in the README as published, so you cannot copy that install line literally. The REST service starts with uvicorn api:app --reload and accepts POST /execute. The documented curl example posts a task_prompt of "Open uber app" with platform android, planner_model gemini and finder_model openai to http://localhost:8000/execute. The other curl example in the README omits the model fields entirely and posts only a task_prompt of "Open Safari", which relies on the documented defaults of android, openai and gemini. The prerequisites section states that adb must be installed on the machine running the code.
Where the model split stops working
The README is unusually direct about a failure case. Ollama models are described as follows: qwen3.5:4b works as planner, described as slow with basic navigation, but is not reliable as finder because of poor UI element detection. The recommended mitigation is to pair an Ollama planner with --image-quality=45. So the local path is documented as viable for planning and not viable for finding. If your reason for choosing this project is fully offline operation, the finder half of that goal is not met by the models the README names.
The project also labels itself. The README states the current code is highly experimental and will evolve in future commits, and asks you to use it at your own risk. The release history is consistent with that: v0.1.0-alpha and v0.2.0-alpha both landed on 2024-12-17, and v0.3.0 added Molmo support via mlx on 2024-12-26. Three releases inside ten days, then the most recent push recorded in the repository metadata is 2026-03-17. The README still contains an unresolved wheel placeholder in the install instructions and a project structure section with no content under it. Those are documentation gaps, not code defects, but they are the kind of gap that costs an afternoon when you are trying to reproduce a demo.
One more constraint worth naming: the README lists GPT 4o among supported remote models, and the finder defaults to gemini while the planner defaults to openai. Those defaults are not the pairing the README recommends. If you run with no flags, you are not running the configuration the project says works best.
The alternative worth comparing, and the actual difference
The obvious comparison is a single-model computer-use agent that takes a screenshot and emits an action in one call, with no separate finder stage. The difference is not model quality, it is where the failure surfaces. In a single-model design, a wrong click and a wrong plan look the same in the logs: one bad output. In ClickClickClick, they are separable. If the planner produces a sensible step and the finder clicks the wrong control, you can change --finder-model and leave the plan alone. That is a real debugging advantage, and it is the reason the Ollama limitation above is stated so precisely: the README can say the model is fine as planner and bad as finder because the roles are distinct objects.
The cost of that separation is an extra inference per step. The finder sees an image every time it needs to locate something, and the planner sees the task state. A single-model agent pays for one call where this pays for two, which is why --image-quality exists as a knob at all. If your tasks are short and the screen is simple, the split buys you little and costs you latency. If your tasks are long, or you are trying to run part of the stack locally for cost or privacy reasons, the split is the reason you would pick this over a monolithic agent.
Maintenance, licensing and what the MIT grant covers
The project is MIT licensed, and the README points to the LICENSE file for details. MIT is permissive: you can use, modify and redistribute the code, including in closed products, provided the copyright notice and permission notice are preserved. That is the standard shape of the grant and it is not legal advice; if you are shipping this inside a commercial product, read the LICENSE file in the repository rather than this paragraph.
What MIT does not cover is the models you point it at. ClickClickClick calls OpenAI, Gemini and Ollama endpoints, and each of those carries its own terms, rate limits and data handling rules. The framework's licence tells you nothing about whether you may send a screenshot of a customer's screen to a hosted vision model. That question is answered by your agreement with the model provider and by whatever the screen contains.
Upgrade cost is hard to estimate from the material. The version history shows one feature release, Molmo support via mlx, arriving nine days after the first alpha, then no further releases in the supplied list despite a much later last push. Whether that means development moved to unreleased commits or slowed down cannot be determined from what is here. The config surface is small (config/models.yaml plus CLI flags), which limits how much a version bump can break, but the README's own warning that the code will evolve in future commits is the honest signal to plan around.
Who should pick this up, and what to check before committing
Take it if you are building or evaluating agentic UI control and you want the planner and finder to be independently swappable, or if you specifically need a documented path to a local planner through Ollama. The three-role structure is the substance here, and the CLI flags make the experiment cheap to run.
Leave it if you need unattended reliability on a schedule, if you need the finder to run locally today, or if you need documentation you can follow without filling in blanks. The README's unresolved wheel placeholder and empty project structure section are the concrete evidence for that last point.
Before you invest time, confirm three things in this order. First, adb is installed and your Android target is reachable, since the README lists adb as a prerequisite and android is the default platform. Second, config/models.yaml exists and the keys it names are exported in your shell, because setup will prompt for them and the run command depends on them. Third, decide your planner and finder pair deliberately rather than accepting the defaults, since the documented defaults (openai planner, gemini finder) differ from the pairing the README says currently performs best.
Editorial conclusion
Adopt it if you want to experiment with swapping planner and finder models independently, or if you need a self-hosted path through Ollama, and you can tolerate a README that labels the code highly experimental. Do not adopt it as production infrastructure for unattended automation, and do not expect Ollama to carry the finder role reliably. Verify first that adb is installed and that your target device is reachable, that your chosen planner and finder are configured in config/models.yaml with the keys exported, and that image quality is low enough for your latency budget, since the README pairs Ollama planner use with --image-quality=45.
Community notes