tentacle-monster-roleplay-esp32: a camera, an AI game master and an ESP32-S3 feedback bridge

What the project actually builds

This is a three-part rig for embodied AI roleplay. The first part is an iPhone Safari page served over local HTTPS that captures camera frames and uploads them to a Windows machine. The second part is a Node.js bridge that keeps overwriting a single file, latest.jpg, plus a metadata sidecar, latest.json. The third part is an optional ESP32-S3 device reached through a PowerShell serial bridge and a Python CLI that sends short commands such as SET, HIT and STOP.

The intended user is one person playing a scene with an AI as the game master. The README frames the loop as real time rather than turn based: the player moves the camera, the AI reads the newest frame, the AI issues a bounded feedback command, and while that feedback runs in the background the AI reads the frame again to see how the player reacted. The repository ships a sample scenario in 剧本.txt where a defeated player hides in an abandoned hut from a level 20 tentacle monster NPC.

The project is honest about its scope. It is a set of scripts and a static page, with no package registry dependencies and no build step for the Windows side.

The data path from iPhone to latest.jpg

The architecture is deliberately flat. server.js is an HTTPS server on port 7777 that also serves public/index.html. The phone loads that page, the player picks the front or rear camera, sets a capture interval and a send resolution, and taps start. Frames are posted to the Windows host and written to latest.jpg in the project directory; latest.json is rewritten alongside with the frame count, timestamp, camera direction and resolution.

Everything downstream reads those two files. There is no queue, no database, no frame history. If the AI reads latest.jpg twice in a row it may get the same image, and if it reads while a write is in progress it may get a partial one. The README does not document atomic replacement or locking, so treat the file as a best-effort handoff rather than a reliable channel.

The image the AI sees is not the raw sensor frame. The settings page in the browser applies mirror, rotation, brightness, contrast, shadows, highlights, sharpening and a crop box before upload, and the send preview shows exactly what will land in latest.jpg. Those adjustments live only in the current page session. A refresh resets them, and they are never written to latest.json, which means the AI has no record of how the frame was transformed.

Installing it and getting a first frame on screen

The README lists the requirements as Windows 10 or newer, Node.js, PowerShell, OpenSSL available as openssl.exe on the command line, an iPhone on the same LAN, and optionally an ESP32-S3. It states that no npm dependencies are needed, and package.json confirms this: the only script is start, which runs node server.js.

Start the camera bridge by running the batch file from the repository root:

start.bat

The script checks for certs/key.pem and certs/cert.pem, calls scripts/generate-cert.ps1 to create a self-signed certificate if they are missing, launches node server.js in the background, and prints the LAN address the phone should open, in the form https://your-lan-ip:7777/. Safari will warn about the self-signed certificate the first time; the README says to continue, since the certificate only covers the local service.

On the phone, open that address, choose the camera direction, set the capture interval and send resolution, and tap 开始. Grant camera permission when Safari asks. Frames should begin appearing in the project directory. To confirm the pipeline rather than guess, run the status script:

status.bat

It reports whether port 7777 is listening, whether an iPhone client is connected, and the frame count, update time, camera direction and resolution from latest.json, plus whether latest.jpg exists and when it was last modified. If an AI host reports no new image, this script is the first thing to run, because it distinguishes a dead server from a phone that stopped uploading. Stop the bridge with stop.bat, which reads server.pid first and falls back to killing whatever Node process holds port 7777.

Driving the ESP32-S3 without letting it run away

The haptic side has two halves. The firmware lives in firmware/esp32s3-galaku/ and is built with the ESP-IDF toolchain. The README gives the sequence from inside that directory:

idf.py set-target esp32s3
idf.py build
idf.py -p COM3 flash monitor

The serial port is COM3 in the example and must be changed to whatever the board actually enumerates as. Once flashed, the PowerShell bridge runs from the repository root and listens on a local TCP port, forwarding short commands to the serial device:

powershell -ExecutionPolicy Bypass -File .\esp32-bridge.ps1 -SerialPort COM3

The documented parameters are -SerialPort, -Baud (default 115200), -ListenAddress (default 127.0.0.1) and -ListenPort (default 25363). The accepted commands are PING, STATUS, SCAN, SERVICES, SET followed by a value from 0 to 100, HIT followed by a damage value, and STOP.

The Python CLI is the practical interface. Checking that the bridge is alive, reading device status, setting an intensity, firing a one-shot event and stopping are separate flags:

py .\vibration-control.py --ping
py .\vibration-control.py --status
py .\vibration-control.py --set 20
py .\vibration-control.py --hit 3
py .\vibration-control.py --stop

The distinction matters: --set writes an intensity level directly, while --hit sends an event that the firmware maps to its own effect. For anything automatic, the README recommends a time limit and an intensity ceiling, and random-wave.py exists for that purpose:

py .\random-wave.py --duration 60 --min 5 --max 35

It also accepts --min-sleep and --max-sleep for the gap between changes, and it sends STOP on exit. stop-random-wave.bat asks a running wave to stop early. The safety model here is entirely in the caller: nothing in the bridge or firmware is described as clamping a runaway command, so an AI host that forgets to send STOP leaves the device at whatever level it was last given.

Where the design gets thin

The biggest limitation is platform lock-in. The bridge is a .bat and .ps1 pair, the setup path assumes PowerShell and openssl.exe, and the README's environment section names Windows 10 or newer. There is no documented macOS or Linux path, and no Docker or container packaging is mentioned. On a Mac you would be rewriting the launcher and the serial bridge before you got to the interesting part.

The second issue is that the roleplay conventions are prose, not code. The README asks the AI host to interpret every frame as in-game scene rendering, to use game terminology, and to describe a specific anatomical-looking detail as a tension artifact of the costume rather than as a real body. That is a prompt-level agreement with whatever model you connect. Nothing in server.js, the firmware or the CLI enforces it. A host that ignores the convention produces output the tooling cannot correct.

The third is that latest.jpg is a live file in the repository directory. The README warns more than once that generated frames may contain private imagery and that latest.jpg, latest.json and local logs must not be committed; .gitignore covers them. That is a real operational hazard, not a theoretical one, because the file is rewritten continuously and any tool that scans the working tree will see it.

Finally, the AI integration is manual. There is no API key, no model client, no daemon that watches latest.jpg and triggers a completion. You paste a prompt into a chat client that can read local files, and you tell it to look at latest.jpg. The README's own example uses OpenAI Codex with a specific model and lists the files the host should read first. That works, but it means the real-time loop depends on how fast your chat client can read a file and respond, which the project does not measure or promise.

How it differs from a plain webcam-and-LLM setup

The obvious alternative is to skip the bridge entirely: open a webcam in any chat client that accepts image input and describe the scene yourself. The README acknowledges this as method one, sending a picture directly in chat, and notes the client may render it as an attachment. That path is simpler and needs no certificate, no port 7777 and no background file.

The difference in approach is continuity. With manual uploads, the AI sees a snapshot at the moment you choose to send it, and the gap between frames is your own reaction time. With the bridge, latest.jpg is always the most recent frame the phone has pushed, so an AI host that can read local files can sample the scene repeatedly inside one stretch of narration. That is what makes the documented loop possible: issue a bounded feedback command with --background, immediately re-read the frame, and adjust the scene based on what changed.

The second alternative is a purpose-built game engine with an input device API. That gives you deterministic state, but it also means the AI is a component inside a fixed rule system. Here the rules live in a text prompt and the state lives in a JPEG, which is far looser and far more fragile. Choose this project when the looseness is the point; choose a scripted engine when you need the same scene to play out the same way twice.

Licence and the cost of keeping it running

The repository declares NOASSERTION for its licence, and the README states plainly that non-commercial use only applies, pointing at the LICENSE file for detail. Those two signals do not resolve on their own, so read LICENSE before you build anything on top of this, and treat the non-commercial restriction as the stated intent until the file tells you otherwise. This is a description of what the repository says, not legal advice.

Maintenance cost is low in the ordinary sense. There are no npm dependencies to audit, no lockfile to refresh, and package.json pins nothing. The last push to the default branch was on 2026-08-15, so the code is recent, but there are no releases and no version tags to track, which means upgrades arrive as commits rather than artefacts.

The recurring costs are environmental. The self-signed certificate in certs/ expires and will need regenerating through scripts/generate-cert.ps1, which start.bat already triggers when the files are absent. The ESP32-S3 firmware has to be rebuilt and reflashed with idf.py whenever firmware/esp32s3-galaku/ changes. And because the AI host is external, model or client updates on that side can change the loop's behaviour without anything in this repository moving.

Editorial conclusion

Adopt it if you are on Windows, already own an ESP32-S3, and want a camera-driven roleplay loop where you control the AI client yourself. Do not adopt it if you need macOS or Linux, a packaged installer, or a fixed script the software enforces. Verify first that the self-signed certificate flow works on your iPhone, that status.bat reports new frames while the AI is reading, and that the licence text in LICENSE actually permits your use, since the repository declares NOASSERTION and the README states non-commercial use only.

Frequently asked questions

Official sources

1. Issues
2. ra1nyxin/tentacle-monster-roleplay-esp32 on GitHub
3. README