ghostwriter: a vision-LLM interface for the reMarkable 2 and Paper Pro

What ghostwriter is for, and who should care

The README describes ghostwriter as "an experiment for the reMarkable that watches what you write and, when prompted either with a gesture or some on-screen content, can write back to the screen." The problem it addresses is narrow: a reMarkable is an excellent input surface and a poor place to run a model. ghostwriter sits on the tablet itself, captures what is on the screen, sends it to a hosted vision model, and renders the reply back into the page. The author frames it as "an exploration of various interactions through this handwriting+screen medium," so the target audience is people who want to feel out that interaction rather than ship it. If you want a drawing assistant that answers a handwritten math prompt or sketches line art in response to a written request, that is the intended use. If you want a note-taking app with a plugin system, this is not it.

The loop: screenshot, model call, render back

The mechanism visible in the material is a loop. The program runs on the tablet, watches for a touch in a configured corner, takes a screenshot of the current page, and submits that image to a vision model. The README states that while processing "you should see some dots drawn during processing and then a typewritten or drawn response." Two output paths exist. One is an SVG drawing tool that the model can invoke, which ghostwriter rasterises and then draws as individual dots on the screen. The other is a virtual keyboard, used by the text-assist mode, which types into the page's text layer. The author distinguishes the sources: text comes from the machine, handwriting from the human. The journal entry for 2024-10-10 notes that the reMarkable's text layer is "basically one large textarea for each page with some very basic formatting," which explains why a synthetic keyboard is needed at all. The SVG path is the more fragile of the two, and the author says so directly: drawing back "doesn't work super well" because rasterised SVG becomes many individual dots, and a screen-filling black square causes the tablet to struggle and the operation to not complete.

Getting the binary onto the tablet

There is no package manager path. You download a prebuilt binary on your laptop and copy it over. For the reMarkable 2: wget -O ghostwriter https://github.com/awwaiid/ghostwriter/releases/latest/download/ghostwriter-rm2, then scp ghostwriter root@192.168.1.117: with your own tablet address. For the reMarkable Paper Pro the asset is ghostwriter-rmpp. On the tablet you run chmod +x ./ghostwriter and then ./ghostwriter --help to confirm it executes. Credentials come from environment variables, and the README suggests adding them to ~/.bashrc on the device: OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY. The engine is auto-detected from the model name, and you can override it with --engine openai, --engine anthropic or --engine google, or point at a different endpoint with --engine-base-url. The default model is claude-sonnet-4-0; ./ghostwriter --model gpt-4o-mini switches to ChatGPT. To keep it alive after you close the SSH session, the README gives nohup ./ghostwriter --model gpt-4o-mini & and then admits in parentheses: "TODO: figure out how to run it on boot!"

Tuning behaviour without rebuilding

The CLI exposes more knobs than the quick start suggests, and they are the practical way to work around the drawing limitations. --trigger-corner takes UR, UL, LR or LL, so you can move the tap target away from wherever you habitually rest your hand. --no-svg disables the SVG drawing tool entirely, which is the obvious response if a model keeps returning images that the tablet cannot render; --no-keyboard disables text output for the opposite preference. --prompt selects a prompt file, defaulting to general.json, so the behaviour is editable without touching Rust. For debugging there is --log-level with info, debug and trace, plus --no-loop to run once and exit, --input-png to feed a file instead of a screenshot, and --save-screenshot and --save-bitmap to capture what the program saw and what it rendered. --no-submit and --no-draw split the pipeline so you can test capture and rendering independently. --apply-segmentation adds image segmentation "for spatial awareness," which is the author's attempt to give the model a better sense of where things sit on the page. --thinking and --web-search are marked as Anthropic features, so they are unavailable on the other engines.

Where it breaks, and when it is the wrong tool

The failure modes are documented by the author rather than discovered by users. Rasterised SVG is drawn as a large number of individual dots, and the journal records that when the whole screen becomes a giant black square the tablet "freaks out a bit" and does not complete. So image-heavy responses are unreliable by design, not by accident. The status indicator is an X drawn in the corner that gains further crosses as processing continues, and the README notes plainly: "You have to erase it yourself though." There is no daemon, no service unit, and no boot integration; a reboot ends the session. The whole thing runs over SSH with root access to the tablet, which is a meaningful trust decision given that the device is sending screenshots of whatever is on the page to a third-party API. The README does not discuss what is captured beyond the page or how long providers retain it. Cross-compilation is also a real cost: you need cross-rs, the armv7-unknown-linux-gnueabihf and aarch64-unknown-linux-gnu targets, and either gcc-arm-linux-gnueabihf on Ubuntu or arm-linux-gnueabihf-binutils on macOS. That is a development environment, not a download.

Alternatives and the difference in approach

The nearest alternative is the reMarkable's own cloud sync plus a desktop client, where you export a page as PDF or PNG and paste it into a chat interface by hand. That approach keeps the model off the device, requires no cross-compilation, and survives reboots because nothing is running on the tablet. The difference is latency and friction: you break the writing flow, move to another machine, and lose the sense that the page itself is the interface. ghostwriter trades that friction for a persistent process on the device and a dependency on the tablet's input and drawing stack. A second alternative is writing against the reMarkable's own file formats and sync tooling directly, which gives you durable storage but no interaction loop. ghostwriter's contribution is the loop, and the loop is exactly the part that is still experimental.

Building from source and the release cadence

The development path is documented and short. Install Docker for cross-compiling, install Rust (rustup, asdf, apt or brew are all mentioned), install cross from git with cargo install cross --git https://github.com/cross-rs/cross, and add the two targets with rustup target add armv7-unknown-linux-gnueabihf aarch64-unknown-linux-gnu. Then cross build --release --target=armv7-unknown-linux-gnueabihf for the rm2 or --target=aarch64-unknown-linux-gnu for the rmpp, followed by scp of the binary from target/<triple>/release/ghostwriter. The author wrapped this into build.sh, so ./build.sh targets the rm2 and ./build.sh rmpp targets the Paper Pro. The working rhythm described is: Ctrl-C the running ghostwriter in one SSH session, run the build script on the laptop, then restart the binary on the tablet. Releases are cut by tagging main, for example v2026.09.21-01, which triggers a GitHub Action that publishes the latest release. The three most recent tags are v2025.09.27-01, v2025.09.21-03 and v2025.09.21-02, so updates arrive as tagged binaries rather than through an in-place upgrade mechanism. There is no migration path to consider because there is no persistent state beyond the prompt files and the page contents.

Licence and what to check before you commit

The repository is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. Note that the licence covers ghostwriter's own code, not the hosted model APIs it calls; your use of OpenAI, Anthropic or Google endpoints is governed by those providers' terms, and the API keys you export into ~/.bashrc on the tablet are the credentials that matter. This is not legal advice. The practical checks are narrower: confirm your device is a reMarkable 2 or Paper Pro so the correct binary applies, confirm the environment variable name matches the engine you intend to use, and decide which corner you want as the trigger before you start writing, because the X indicator stays on the page until you erase it. If those three things line up, the setup is a wget, an scp, a chmod and a single command.

Editorial conclusion

Adopt ghostwriter if you already own a reMarkable 2 or Paper Pro, are comfortable with SSH and cross-compilation, and want a tangible way to explore handwriting plus vision-model interaction; skip it if you need a supported product, a stable API, or a workflow that survives a reboot. Before installing, verify that your tablet matches one of the two release binaries, that you have set the relevant API key environment variable, and that the touch trigger corner you configure does not collide with your normal handwriting.

Official sources

1. awwaiid/ghostwriter on GitHub
2. Issues
3. License: MIT
4. README
5. Releases