Handy: A Local-First Speech-to-Text App Built to Be Forked

The Problem: Offline Transcription Without a Paywall

Most speech-to-text tools either send audio to the cloud or charge a subscription. Accessibility tools, the README argues, belong in everyone's hands. Handy targets that gap: a desktop app that transcribes speech locally, pastes the text into any field, and costs nothing. It is aimed at users who value privacy and at developers who want to modify the tool. The README states Handy is not trying to be the best speech-to-text app, but the most forkable one. That positioning shapes every design decision, from the MIT license to the modular Rust libraries.

How It Works: Shortcut, VAD, Whisper or Parakeet, Paste

The flow is simple. You press a configurable keyboard shortcut to start recording, speak, and release. Handy filters silence using Silero VAD, then transcribes with one of two model families. Whisper models (Small, Medium, Turbo, Large) use GPU acceleration when available, while Parakeet V3 is CPU-optimized with automatic language detection. The resulting text is pasted into whatever application is focused. All processing happens locally. The architecture is a Tauri app: a React and TypeScript frontend for settings, and Rust for audio capture, resampling with rubato, and ML inference. Core libraries include transcribe-cpp for Whisper-family GGML models and transcribe-rs for Parakeet. This split matters because it lets users pick a model based on their hardware: GPU for Whisper, CPU for Parakeet.

Getting It Running: Installers, Homebrew, and winget

Installation is straightforward for end users. Download a release from GitHub or handy.computer. macOS users can run `brew install --cask handy`; Windows users can run `winget install cjpais.Handy`. The README notes these packages are not maintained by the Handy developers, so expect possible delays. After install, grant microphone and accessibility permissions, then configure shortcuts in Settings. Developers building from source should consult BUILD.md for platform-specific requirements. The app also exposes CLI flags for remote control of a running instance: `handy --toggle-transcription`, `handy --cancel`, and startup flags like `--start-hidden` and `--no-tray`. On macOS, invoke the binary directly at `/Applications/Handy.app/Contents/MacOS/Handy`. These flags enable autostart scenarios, which is a practical touch for a tool meant to be always available.

The Hard Limits: Bluetooth, fn Keys, and Whisper Crashes

The README lists three significant limitations. On macOS, using a Bluetooth headset microphone can temporarily degrade playback quality because Bluetooth switches to bidirectional audio. The workaround is to use a separate microphone. Shortcuts that include the fn or Globe key only work on Apple keyboards. This is a hardware limitation: third-party keyboards handle fn entirely in firmware and send no event to the OS. The README advises using standard modifiers instead. More seriously, Whisper models crash on certain Windows and Linux configurations. The issue is configuration-dependent and the project asks developers to help fix it. These are not edge cases; they are the kind of problems that plague a young open source project. If you rely on a Bluetooth headset or a non-Apple keyboard on macOS, Handy may not work as expected.

Linux and Wayland: A Partial Story

Linux support exists but has caveats. For X11, Handy recommends `xdotool`; for Wayland, it requires `wtype` or `dotool` to inject text. The README says Wayland support is limited. This is a real constraint because Wayland is the default on many modern distributions. The dependency on external tools for text input adds friction. On X11, you must install xdotool separately. On Wayland, you need wtype or dotool, and even then support is not guaranteed. If you are a Linux user, check your display server before installing. The README's Linux Notes table is truncated, so exact installation commands for each tool are not fully listed. That incompleteness is itself a signal: the project is still filling in platform gaps.

Extensibility: The Forkable Core

Handy's main differentiator is its extensibility. The README invites users to extend it and contribute. The architecture supports this: separate Rust crates for transcription (transcribe-cpp, transcribe-rs), VAD, and audio I/O. There is already a Raycast extension that controls recording, browses history, manages a dictionary, and switches models and languages. The CLI flags also open scripting possibilities. For developers, this means you can build your own workflow on top of Handy without touching the core. The MIT license allows commercial use and modification. However, extensibility has a cost: the project is not a finished product. The known issues list is long, and the README asks for help. If you adopt Handy, you are implicitly signing up for a project that expects its users to be contributors.

Alternatives: What Else Is Out There

The obvious alternative is a commercial service like OpenAI's Whisper API or Google Speech-to-Text. Those are cloud-based, so they offer high accuracy and low latency but send audio off-device. For offline use, there is also whisper.cpp, a standalone C++ implementation of Whisper that runs locally. Handy builds on Whisper via transcribe-cpp, so the difference is the application layer: Handy provides the keyboard shortcut, VAD, and paste integration. Another alternative is Parakeet models directly through transcribe-rs, but that requires building your own interface. The key difference is that Handy is a ready-made app, while whisper.cpp is a library. If you want a turnkey tool, Handy is closer. If you want maximum control, you might use whisper.cpp directly. The README's emphasis on forkability suggests the author would not mind if you did.

Maintenance and Licensing: What You Need to Know

The project is under the MIT license, which is permissive: you can use, modify, and distribute it freely, even commercially. The last push was August 2026, with releases every few weeks (v0.9.4, v0.9.5, v0.9.6). That cadence suggests active development, but the README also lists major issues that need help. Maintenance cost for adopters is non-trivial: you may need to recompile or update when the project changes. The Homebrew and winget packages are not maintained by the developers, so they may lag behind. The debug mode (Cmd+Shift+D on macOS, Ctrl+Shift+D elsewhere) and CLI flags help troubleshoot, but you will likely need to read the source or file issues. For a project this young, expect breaking changes between releases. Verify the current release notes before upgrading.

Editorial conclusion

Adopt Handy if you need a free, offline, keyboard-driven transcription tool and are comfortable with a project that is still maturing. It suits developers who can debug crashes, Linux users on X11 (or willing to install wtype/dotool for Wayland), and anyone who values a forkable codebase over a polished product. Avoid it if you require guaranteed stability on all hardware, rely on Bluetooth headsets on macOS, or need a non-technical support path. Before adopting, verify your GPU acceleration works with Whisper models, test your keyboard shortcuts on Apple vs. third-party keyboards, and check the known issues list for your exact OS configuration. The project's transparency about its limitations is a strength, but it is not a drop-in replacement for a commercial service.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes