claude-desktop-buddy: an ESP32 desk pet for Claude Cowork and Claude Code

What claude-desktop-buddy actually solves

Claude for macOS and Windows can talk to maker hardware over BLE, and this repository is the worked example of that API. The problem it addresses is small but real: approval prompts and session activity live inside a desktop window, so a developer watching a long agent run has to keep switching back to it. The buddy is a physical object that shows the prompt and lets you answer it with a button.

The README is explicit about the audience. It is written for makers and developers, and it states the API "isn't an officially supported product feature." That sentence should shape every decision you make about this code. You are not adopting a product. You are reading a reference implementation, one that happens to be complete enough to flash and use.

If you are building your own device, the README says you do not need any of the firmware here. REFERENCE.md carries the wire protocol: Nordic UART Service UUIDs, JSON schemas, and the folder push transport. That split is the most useful thing in the repository. The protocol is the contract; the ESP32 desk pet is one client of it.

The BLE bridge, the state machine, and the seven states

The firmware is a C++ Arduino sketch. The project layout lists the pieces: main.cpp holds the loop, state machine and UI screens; ble_bridge.cpp implements the Nordic UART service with line-buffered TX and RX; character.cpp decodes and renders GIFs; data.h holds the wire protocol and JSON parsing; xfer.h receives folder pushes; stats.h keeps NVS-backed stats, settings, owner and species choice.

State is derived from the bridge, not chosen by the user. The README's table maps each state to a trigger: sleep when the bridge is not connected, idle when connected with nothing urgent, busy when sessions are actively running, attention when an approval is pending, celebrate on a level up every 50K tokens, dizzy when you shake the stick, and heart when you approved in under 5 seconds. That last one is the design telling you what it wants you to do.

The approval screen is where the hardware earns its place. In approval mode, button A approves and button B denies. The screen stays powered while a prompt is up, overriding the 30-second auto-power-off that applies otherwise. An LED blinks in the attention state, which matters because the stick may be sitting face-down or across the desk.

Everything else is texture: eighteen ASCII pets with seven animations each, a shake gesture, a face-down nap that refills energy, and a screen that wakes on any button press. The pet is the demo, and the permission flow is the mechanism.

Flashing the firmware with PlatformIO

The firmware targets ESP32 with the Arduino framework, and as written it depends on the M5StickCPlus library for its display, IMU and button drivers. You need that board, or a fork that swaps those drivers for your own pin layout. Install PlatformIO Core first, then build and upload from the repository root:

pio run -t upload

If the device was flashed before, the README recommends wiping it first. The erase target runs before the upload in the same command:

pio run -t erase && pio run -t upload

There is also an on-device path: hold A, go to settings, reset, factory reset, and tap twice. That is useful when the stick is already mounted and you do not want to open a terminal.

Pairing happens on the desktop side. Enable developer mode through Help, Troubleshooting, Enable Developer Mode. Then open Developer, Open Hardware Buddy, click Connect, and pick the device from the list. macOS prompts for Bluetooth permission on first connect. Once paired, the README says the bridge auto-reconnects whenever both sides are awake.

If discovery finds nothing, the README gives two checks: make sure the stick is awake with any button press, and confirm bluetooth is on in the stick's settings menu.

Custom GIF characters and the 1.8MB ceiling

ASCII pets are built in. GIF characters are pushed at runtime. You drag a character pack folder onto the drop target in the Hardware Buddy window, the app streams it over BLE, and the stick switches to GIF mode live. Settings, delete char reverts to ASCII.

A pack is a folder with manifest.json and 96px-wide GIFs. State values can be a single filename or an array, and arrays rotate so each loop-end advances to the next GIF. That is how you build an idle carousel instead of one clip on repeat:

{
  "name": "bufo",
  "colors": { "body": "#6B8E23", "bg": "#000000", "text": "#FFFFFF", "textDim": "#808080", "ink": "#000000" },
  "states": {
    "sleep": "sleep.gif",
    "idle": ["idle_0.gif", "idle_1.gif", "idle_2.gif"],
    "busy": "busy.gif",
    "attention": "attention.gif",
    "celebrate": "celebrate.gif",
    "dizzy": "dizzy.gif",
    "heart": "heart.gif"
  }
}

The constraint that bites is size. The whole folder must fit under 1.8MB, and the README suggests `gifsicle --lossy=80 -O3 --colors 64`, which it says typically cuts 40 to 60 percent. Width is fixed at 96px; height up to about 140px stays on a 135x240 portrait screen. Crop tight, because transparent margins waste screen and shrink the sprite. `tools/prep_character.py` resizes source GIFs at any size into a 96px-wide set where the character is the same scale in every state.

If you are iterating on a character, the BLE round-trip is slow. `tools/flash_character.py characters/bufo` stages the pack into `data/` and runs `pio run -t uploadfs` over USB instead. `characters/bufo/` is the working example to copy.

Where this firmware is the wrong tool

The board lock is the first limitation, and it is not incidental. The README states the firmware depends on the M5StickCPlus library for display, IMU and button drivers. Nothing in the repository abstracts those away. Porting to a different ESP32 board means writing your own drivers for the screen, the accelerometer and the buttons, then reworking the UI assumptions in main.cpp. That is a fork, not a configuration change.

The size ceiling is the second. A folder over 1.8MB will not push, and the manifest format gives you no streaming or lazy-load path around it. A character with many long animations runs out of room quickly, and the documented answer is lossy compression, which changes how the art looks.

The developer mode requirement is the third, and it is a hard gate. The BLE API is only available when the desktop apps are in developer mode. If you are distributing a device to people who will not enable developer mode, the bridge will not be there for them. Anthropic's own framing, that this is not an officially supported product feature, means no compatibility promise across desktop releases. The README does not document a versioning scheme for the wire protocol, and it does not document rollback if a firmware update goes wrong beyond the erase and factory reset paths.

Building your own device instead of flashing this one

The most direct alternative is not another project. It is REFERENCE.md. If you have hardware that is not an M5StickC Plus, or you want a different form factor entirely, the README points you there for the wire protocol: Nordic UART Service UUIDs, JSON schemas, and the folder push transport. You implement a BLE peripheral that speaks that protocol and skip the firmware in this repository completely.

The difference in approach is concrete. Flashing this firmware gives you a working desk pet on one specific board, with eighteen ASCII species, seven animations each, NVS-persisted settings and a GIF pipeline, in exchange for accepting the board dependency and the M5StickCPlus driver stack. Implementing against REFERENCE.md gives you freedom in hardware and display at the cost of writing the state machine, the renderer and the character handling yourself. The protocol is the smaller half of the work; the UI and animation code is the larger half, and it is the half you would be giving up.

A middle path exists in the repository itself: fork the firmware and swap the driver calls. The README names this as the expected route for a different board. The state machine and the BLE bridge are worth keeping; the display, IMU and button layers are what you replace. How much of main.cpp survives that swap is not something the README describes, so read the file before committing to the fork.

Editorial conclusion

Adopt it if you already own an M5StickC Plus, run PlatformIO, and want a worked example of the Nordic UART JSON protocol before writing your own firmware. Skip it if you need a supported product feature, a board other than the M5StickC Plus without forking the drivers, or a character pack larger than 1.8MB. Before flashing, read REFERENCE.md and confirm the BLE API appears only after enabling developer mode via Help, Troubleshooting, Enable Developer Mode.

Frequently asked questions

Official sources

1. anthropics/claude-desktop-buddy on GitHub
2. Issues
3. README