Lipflow types what you mouth, using a webcam and a lip reader

The pipeline, and where each stage runs

The README draws the architecture as a diagram, and it is worth reading because the whole design is in that picture. You hold the key, and a webcam feed goes to face landmarks, live, which produce mouth crops at 25 fps. Those crops go into a visual speech recognition encoder running on the Apple GPU. From there, a beam search plus a language model turns the visual features into words, and an LLM cleanup pass corrects what the lip reading got wrong. The result is pasted at your cursor. There is a parallel branch for feedback: while you talk, a live preview runs greedy CTC every 0.45 seconds so you can see the reading forming as you mouth the words. That preview is a usability detail rather than a flourish, because silent speech is hard to calibrate without immediate feedback. The core claim is that no microphone and no sound is involved, just the webcam, which is what makes it usable in a shared office or on a call.

Installation is a script and about 1.2 GB of models

On macOS the whole setup is a clone, a setup script and a launch:

git clone https://github.com/amywork777/lipflow.git ~/code/lipflow
cd ~/code/lipflow && ./setup.sh     # installs uv deps, ~1.2 GB of models, builds /Applications/Lipflow.app
open /Applications/Lipflow.app

The script installs the uv dependencies, downloads roughly 1.2 GB of models and builds a Lipflow.app bundle in Applications. The requirements are an Apple Silicon Mac on macOS 13 or later, with macOS 26 called out for the Liquid Glass look, and about 2 GB of disk. Linux and Windows need about 3 GB each, the extra coming from PyTorch rather than from the models. The models are not fetched at first launch but at setup time, which means the install step is where the download happens and the first launch is where the training happens. Those are two different waits and people conflate them, so it is worth knowing which one you are in. The package itself is a Python project declared in pyproject.toml, with the lipflow entry point mapped to lipflow.__main__:main and a build backend of hatchling.

First launch is eight minutes of practice and training

The first launch opens a setup flow that takes about eight minutes, and it is the part of the product that actually determines whether the output is usable. It begins with permissions. Camera, Input Monitoring and Accessibility are each requested once for Lipflow. The reason is architectural and the README explains it: the app is a small signed program that loads this checkout in-process, so the permissions belong to Lipflow rather than to Python. That is also why a toggle can be on in System Settings while setup still shows it off, in which case you click Restart Lipflow under Continue. Second, your words: if you use Wispr Flow, it imports your dictation history, read locally and never uploaded, to learn your phrasing and the names you say. Third, practice, in which you silently mouth 24 sentences, taken from your own history when available. Fourth, training, which fine tunes the language model on your phrasing and the lip reader on your face, on this Mac's GPU. Six practice sentences are held out, and the face model is only kept if it reads them better than the stock model, with the before and after score shown to you. That held-out check is the honest part of the design, since it means the tool tells you whether your training was worth keeping rather than silently accepting it.

Where your data lives and how to erase it

The privacy story is mostly a question of where files sit on disk, and the README is specific. Everything personal, meaning clips, phrases and trained models, lives in ~/Library/Application Support/Lipflow/, and deleting that folder starts you over. Setup can be re-run at any time from the menu, via Set up / train on my face, and more practice clips help more. To start Lipflow at login the instructions are System Settings, General, Login Items, add Lipflow. Logs are in ~/Library/Logs/Lipflow.log. Running from a terminal with uv run lipflow also works, but then macOS asks for the permissions in the terminal's name, which is a practical detail worth knowing before you conclude the permissions are broken. The import of Wispr Flow history is read locally and never uploaded, and the same locality applies to the webcam frames, since the whole pipeline is described as running locally on your Mac, Windows PC or Linux. The one network call in the chain is the optional LLM cleanup pass, and that is the stage where the choice of provider is yours.

Windows and Linux run the same model on weaker hardware

The two other platforms work, but the README is explicit about the differences, and they are mostly about speed and about which model does the cleanup. On Windows you need 64-bit Windows 10 or 11, and the PowerShell equivalent of the setup is a clone and a script:

git clone https://github.com/amywork777/lipflow.git $HOME\code\lipflow
cd $HOME\code\lipflow
powershell -ExecutionPolicy Bypass -File setup.ps1   # uv deps, ~1.2 GB of models, Start menu shortcut

The speed difference is quantified. Without an NVIDIA GPU the lip reader runs on the processor, so expect a second or two more per sentence than on a Mac, and the README gives the figures for an M4 Pro where the encoder takes 1.7 s on CPU against 0.16 s on the GPU. Training on your face takes several times longer, and with an NVIDIA card you install the CUDA build of PyTorch, with the command given as uv pip install torch --index-url https://download.pytorch.org/whl/cu126, re-run after uv sync because sync puts the CPU build back. The cleanup stage is the other difference, and it is a hard one. The in-process local model is Mac-only because it is MLX, so on Windows and Linux you use Claude, the Codex CLI or Ollama, or you get the offline rules instead. Names from context on Windows come from the window title only, and learning from your corrections is Mac-only for now.

Linux needs a compositor toolkit before it works

The Linux path needs a 64-bit distro, a webcam, Python 3.11 to 3.12 and about 3 GB of disk, and the setup differs from the Mac in that the models are downloaded by a separate script:

git clone https://github.com/amywork777/lipflow.git ~/code/lipflow
cd ~/code/lipflow
curl -LsSf https://astral.sh/uv/install.sh | sh   # if you don't have uv yet
uv sync
./scripts/download-models.sh                        # ~1.2 GB of models
uv run lipflow doctor
uv run lipflow                                      # system tray (pink mouth icon)

The doctor command checks the models and camera before you start. What makes Linux fiddly is the paste step, because a compositor does not let a program type into another window without a helper tool. Wayland needs wl-clipboard and one key tool, wtype on Hyprland and Sway or ydotool with ydotold running on GNOME, KDE and other compositors, while X11 needs xdotool and xclip, and the --copy-only flag leaves the text on the clipboard if you would rather paste it yourself. The tray icon needs PyGObject and an AppIndicator typelib, and on Ubuntu the package named is gir1.2-ayatanaappindicator3-0.1. The camera comes from PipeWire or v4l2 at /dev/video0. And the global hotkey on Wayland is read from /dev/input, so your user has to be in the input group, with a log out and back in needed for the group change to take effect. Context names come from the active window title, via hyprctl on Hyprland, swaymsg on Sway, kdotool on KDE and xdotool on X11, and GNOME has no stable title API.

Editorial conclusion

Adopt Lipflow if you need to type without making sound, on a machine you control, and if you are prepared to spend the first launch on about eight minutes of practice and training, since the accuracy gain comes from that step rather than from the shipped model. Do not adopt it if you need the cleanup step to run on a private machine, because the in-process model is Apple Silicon only and Windows and Linux users have to point it at Claude, the Codex CLI or Ollama or fall back to the offline rules. Verify first that your platform is supported, that the machine meets the disk and hardware requirements for the model download, and that the key you press does not collide with an existing shortcut, since the hotkey is read from the keyboard device rather than from the window focus.

Frequently asked questions

Official sources

1. amywork777/lipflow on GitHub
2. Issues
3. License: MIT
4. README