A phone agent that reads your screen, with everything that implies

An agent that drives your phone by looking at it

This is an Android automation agent built with Flutter. You give it an instruction in plain language, and it operates your phone on your behalf, tapping, typing and scrolling through whatever applications are installed, by reading the screen and deciding what to do next.

The mechanism is the interesting part and the part that determines everything else about it. The agent uses Android's Accessibility Services, the operating system facility built so that screen readers and switch-access tools can perceive and control the interface for people who cannot do it directly. This project uses that same facility to let a language model perceive and control the interface instead.

That is a legitimate and clever use of the platform. Accessibility Services exist precisely so that one program can read another's layout and act on it, which is normally impossible on Android for security reasons. Building automation on top of it means the agent works across any application without that application's cooperation, because it is operating at the level the accessibility layer sees rather than through any per-app interface.

The audience is someone who wants to automate multi-step tasks on their phone that no single app exposes an interface for, and who is comfortable with what granting that access involves. This analysis spends most of its length on that second clause, because it is where the real decision sits.

The loop, and the choice to use coordinates

The README describes a continuous feedback loop of six steps, and it is worth following because the design is sound.

A command arrives by voice, text or a remote channel. The agent captures the current screen hierarchy and computes the spatial coordinates of every interactive element. That layout, the task, and the result of the previous action go to the AI provider. The provider returns the next action, such as tapping a coordinate, entering text or scrolling. The native Android layer executes it. The loop repeats until the task is marked complete.

The detail that shows real experience is the emphasis on coordinates. Instead of trying to identify elements by their text or accessibility labels, which are frequently missing, wrong, or absent on icons, the agent works from the geometry of where things are on screen and simulates physical taps at those points. The README states this explicitly as a way to handle elements with missing labels or inaccessible icons.

That is the correct engineering call. Anyone who has tried to automate a real Android application knows that half the buttons have no usable label, and a system that depends on labels breaks constantly. Falling back to spatial coordinates holds up far better, and choosing it deliberately signals the author hit the label problem in practice and solved it rather than wishing it away.

Passing the previous action's result back into the next decision is the other good choice, because it lets the agent notice when a tap did nothing and try something else, rather than marching blindly through a plan that stopped working at step two.

What granting accessibility control actually means

Here is the part that deserves to be read slowly, and it is not a criticism of the project so much as a description of what using it involves.

An accessibility service can see everything on your screen and act anywhere on it. That is what the permission is for, and it cannot be scoped down. Whatever is visible while the agent runs, the agent reads: the contents of your messages, your email, your banking app, a one-time passcode when it appears. And it can act anywhere, which is to say it can send, delete, pay and confirm exactly as a person holding the phone could.

Now add the second fact. In this design, the screen layout is transmitted to an external AI provider at every step of the loop. That means the structure and text of whatever is on screen leaves the device and goes to a third party each time the agent decides what to do next. Which provider, and what that provider does with the data, is determined by the key you configure, and the README's free-usage path points at a hosted service where the free tier's data handling is the provider's to define.

None of this is hidden and none of it is unusual for this category of tool. It is the unavoidable shape of an agent that reads your screen and thinks with a cloud model. The honest way to state it is that running this grants total on-device control to an automated system and streams your screen contents to a remote service while it runs, and both of those are the price of the capability rather than flaws to be fixed.

The practical consequence is that the machine you run this on, and the tasks you point it at, should be chosen accordingly. A spare device, or a careful separation from anything sensitive, is the cautious approach, and the sensitivity of what is on screen while it works is the thing to keep in mind.

The restricted-settings step is the OS trying to protect you

The setup instructions include a step that rewards attention, because the README is transparent about something many projects would gloss over.

Modern Android blocks applications installed from an APK, rather than from the official store, from receiving accessibility access. The user has to open the app's settings, choose allow restricted settings from a menu, and confirm, before the accessibility service can be enabled at all. The README documents this and provides shortcuts to the relevant screens during onboarding.

It is worth being clear about what that step is. It is not a bug or an annoyance the project is helping you past for no reason. Android added exactly this friction because accessibility access granted to a sideloaded application is one of the most powerful things a user can hand over, and malicious applications have historically abused it to do precisely what this agent does legitimately: read the screen and act on it. The prompt is the operating system asking you to confirm you meant to grant that.

The project's honesty in documenting the step plainly is to its credit. The corresponding responsibility on the user's side is to recognise that clicking through it is a real decision, not a formality, and to have decided you trust this specific application before you do. The friction exists to make you pause, and pausing is the correct response rather than something to be optimised away.

Remote control, and providers you can swap

Two further features extend both the usefulness and the trust surface, and they deserve to be weighed together.

The agent integrates with a messaging bot platform, maintaining a background polling connection so a user can issue commands and watch task progress from elsewhere. That is genuinely useful for automation you want to trigger while away from the phone. It also means the device can now be driven remotely by whoever holds the bot token, which is one more credential whose compromise would hand control to someone else. Voice control through native speech-to-text is offered for hands-free operation, which raises no comparable concern.

On the model side, the project is provider-flexible rather than tied to one service. The README's free path uses a hosted aggregator's free models, with a named model string to paste in, and the description mentions a major model provider's interface. The release notes for version 1.0.2, published on 2026-07-17, describe adding support for a hardware vendor's inference service, discovery of its free chat models, and making one of them the default, alongside fixes to streaming response parsing.

Provider flexibility is a real strength, because it means the agent is not hostage to one company's pricing or availability, and a user who prefers a particular provider's data handling can choose it. It also means the answer to where your screen data goes depends on your own configuration, which puts that decision, and its consequences, in your hands rather than the project's.

A capable tool with an unfinished licence and a clear caller

The most concrete gap is licensing. The README states the project is open source and available for modification, but the repository carries no licence file and its metadata reports no licence. A sentence in a README is not a licence grant, so despite the stated intent, the terms under which this may actually be used, modified or redistributed are not established. Anyone planning to build on it should ask the author to add a real licence file, because right now the invitation and the legal reality do not match. This is not legal advice.

The repository is a normal Flutter project in layout, with a release published the same day as the last push on 2026-07-17, so it is a young project rather than an abandoned one. It reports 328 stars, 141 forks and eight open issues, and the fork count relative to stars suggests a fair number of people are building their own variants. Distribution is by APK from the releases page, with a universal build recommended and per-architecture builds otherwise, supporting Android 8.0 and newer, and the notes mention checking recent Android versions' native-library alignment requirement.

Before running it, work through four things in order. Decide which device this goes on, treating total accessibility control and streamed screen contents as the baseline rather than an edge case, and prefer a phone without your most sensitive accounts on it. Choose your AI provider deliberately, since that choice decides where your screen data travels. Treat the restricted-settings prompt as the real decision it is. And if you intend to modify or redistribute, get a licence file in place first, because the current one-line invitation does not give you the permission it implies.

Editorial conclusion

PrivateAgent is a capable and honestly documented tool for automating multi-step tasks across Android apps that expose no interface of their own, and its choice to act on screen coordinates rather than unreliable labels shows the author solved the real problem rather than the easy one. Approach it understanding that an accessibility service grants total control of the device and that this design streams your screen layout to an external provider at every step, so choose the device and the provider deliberately and prefer to keep it away from your most sensitive accounts. Treat the restricted-settings prompt as the deliberate safety pause Android intends rather than an obstacle, and if you plan to modify or redistribute, ask for a real licence file first, because the README's one-line invitation is not the grant it sounds like.

Frequently asked questions

Official sources

1. Issues
2. orailnoor/private-agent on GitHub
3. README
4. Releases