Taskhomie computer agent: a local AI on your desktop

A local agent that drives the screen

The repository is described as a desktop app to control your computer with AI using your terminal, browser, mouse and keyboard, and the readme calls the project Taskhomie, the Computer Use AI Agent.

The short version is that you give it an instruction in natural language and it takes screenshots, moves your mouse, clicks, types and runs terminal commands. The stack is Tauri for the desktop shell, React for the interface and Rust for the backend, and the models are the Anthropic family, with Haiku, Sonnet and Opus selectable in the user interface.

Calling it local is about the application rather than the model, and it is worth being precise about that, because the distinction decides what leaves your machine. The application runs on your desktop, the screenshots it takes are of your screen, and the text it sends to the model is whatever the task requires, which for a browser task can be the contents of a page. The API key is an Anthropic one, so the model calls go to Anthropic's API from your machine.

The readme also notes that the project is from the author of a demo in which the agent autonomously read and replied to tweets, and the author's own comment is that it is purely for demonstration and research and that you should not attempt the same. That is a sensible warning from someone who has seen what a mis-scoped agent does on an account you care about.

The licence field on the repository points at a file the hosting platform cannot classify, while the readme says Apache License 2.0, and the last push was on 2026-08-13.

Two modes, and the split is about control

The mode design is the most interesting part of the project, and it is a split on one axis, how much of your machine the agent is allowed to touch.

Computer Use Mode takes over your screen. It sees what you see through screenshots and controls your cursor and keyboard directly. The readme says to use it when the task spans multiple applications or needs full desktop access, and the expectation is that you step away while it works.

Background Mode runs asynchronously while you do other things. It uses the Chrome DevTools Protocol for web automation and the terminal for everything else, and it does not touch your mouse or your keyboard. The readme describes it as faster and more reliable for web and terminal tasks.

That second claim is the interesting one, and it is a technical claim rather than a preference. Driving a browser through its debugging protocol means the agent acts on the page the way the page expects to be acted on, with a DOM rather than with a rectangle of pixels, and it means the agent can be working while you keep typing in another window. Taking over the screen is the fallback for the cases the protocol cannot reach, and it is inherently the slower and more fragile of the two, because every action is a coordinate and a click.

So the modes are a capability ladder. Background Mode is the precise, fast, non-intrusive path. Computer Use Mode is the general, slow, intrusive one, and the readme's guidance is to reach for the second only when the first cannot do the job.

Push to talk, and four shortcuts

The interaction model is voice driven, and the shortcut table is the entire interface surface.

The primary shortcut is a push to talk combination that switches into computer use mode, and a second one that switches into background mode. A third opens a help mode, described as a screenshot plus a quick prompt, and a fourth stops the agent.

The push to talk behaviour matters more than it sounds. A desktop agent that is always listening is a desktop agent that is always recording, and binding the microphone to a modifier means the microphone is only open while you hold the combination. The difference between that and a permanently hot key is the difference between an agent you invoke and a system that watches you.

The stop shortcut is the other one to know. An agent that is midway through a task with your cursor under its control needs a way to be interrupted that does not depend on the agent deciding to stop, and a global shortcut is the only version of that which works when the agent has taken over the screen.

The help mode is a smaller detail and a useful one, since a screenshot plus a prompt box is a way to point the agent at something without describing it in words, which is often faster than typing a description of a button on the screen.

Setup is a Tauri dev command and one API key

The installation is short, and the requirements are the three you would expect for a Tauri application backed by a hosted model.

You need Rust and Cargo, Node.js and npm, and an Anthropic API key. Then the three commands are:

# install deps
npm install

# add your api key
echo "ANTHROPIC_API_KEY=your-key-here" > .env

# run dev
npm run tauri dev

For a production build, the readme gives npm run tauri build, and the package manifest has the matching scripts, with a plain Vite dev server for the front end alone, a build that runs the TypeScript compiler before Vite, and a preview script.

The one platform-specific step is on macOS, where the application needs accessibility permissions when prompted, under System Settings, Privacy and Security, Accessibility. That is the operating system asking for the right to synthesise input events, and it is the same permission a remote control tool or an automation utility needs.

Writing the key into a .env file rather than exporting it in the shell is the tidier of the two, since the file is in the project directory and does not end up in the shell history.

The manifest shows a desktop app with a graphics layer

The package manifest is short, and it tells you more about the application than the readme does.

The package is named taskhomie at version 0.1.0, private, and an ES module. The Tauri packages are the API and the shell plugin, which is the piece that lets the Rust side run processes, and that is what a terminal-capable agent needs. The front end is React 19 with react-dom, Zustand for state, Framer Motion for animation and lucide-react for the icons.

Three dependencies are worth a second look because they are not what a dashboard would use. Three is at 0.182, so the application has a 3D layer, and combined with the screenshot in the readme that suggests the interface renders something spatial rather than being a flat chat window. Streamdown is a Markdown renderer, so the model's output is rendered as formatted text rather than as raw characters. And simplex-noise is a noise generator, which is what you use for a procedural background or for a procedural cursor trail.

The build tooling is Vite 6 with the React plugin, Tailwind 3 with the typography plugin, PostCSS and autoprefixer, and TypeScript 5. The type checker runs as part of the build, since the build script is tsc followed by vite build, so a type error fails the build rather than shipping.

The repository layout matches a Tauri project, with the Rust side in src-tauri, the React side in src, the static files in public, and a Vite configuration, a Tailwind configuration, a PostCSS configuration and two TypeScript configurations for the app and for the tooling.

Editorial conclusion

The Taskhomie computer agent fits a developer who wants to hand a repetitive desktop task to a model and walk away, because the background mode drives the browser through the Chrome DevTools Protocol and the terminal for everything else, which is both faster and safer than letting a model move your cursor. It is the wrong choice if you need it to drive applications that only exist as pixels, because that is what the screen takeover mode is for and it is the mode the author himself demonstrates only as research, with a warning not to try it on a social account. It is also the wrong choice if you need a maintained project, because the version is 0.1.0, there are no releases, and the package is named differently from the repository. Before you run it, decide which mode you want, because the shortcuts switch between them mid-session, and set the API key through the environment file rather than into your shell history.

Frequently asked questions

Official sources

1. Issues
2. README
3. suitedaces/computer-agent on GitHub