Hermes3D: a self-hosted 3D office for watching AI agents work

What Hermes3D actually is, and the problem it picks

Agent systems produce logs, terminal output and dashboards. Hermes3D's README frames that as the problem: work that is "still usually hidden behind logs, terminal output, and dashboards", and a goal of making agent systems visible through "space, motion, and presence". The deliverable is a live 3D office at /office where agents appear as workers moving between desks and rooms, plus an /office/builder surface for editing and publishing layouts.

The intended user is someone who already runs an agent backend and wants to observe it. The README is blunt about the boundary: this repository "does not build the upstream runtimes themselves. It is the frontend, Studio, and adapter/proxy layer that connects to a runtime speaking the Hermes3D gateway protocol." If you have no runtime, the built-in demo gateway exists so you can walk around the office without one.

That framing matters for evaluation. Hermes3D competes with observability UIs, not with game engines, even though it is built on three.js and carries game-engine topics. The interesting question is not whether the renderer is fast; it is whether a spatial view of agent activity tells you something a table of runs does not.

Gateway-first architecture: where state lives

The architecture described in the README is gateway-first. Runtime state stays in the connected backend, while Studio stores local UI preferences: gateway connection details, focused-agent preferences, desk assignments and office state. The office is therefore a projection of backend events, not a simulation with its own truth.

The browser does not talk to the upstream gateway directly. A custom same-origin WebSocket proxy sits in the middle: the browser talks to Studio, and Studio talks to the upstream gateway. That is a deliberate trade. It keeps the gateway token out of the client and gives Studio a place to normalize events, at the cost of one more process that can fail between the two ends. The package layout confirms the shape: a custom server in server/index.js, a Hermes gateway adapter script, and a separate demo gateway adapter script, all started through npm scripts rather than through Next.js alone.

Runtime selection is a seam rather than a hardcoded backend. The README lists Hermes through the bundled WebSocket gateway adapter, a direct HTTP custom runtime provider for orchestrator-backed stacks, and the demo gateway. Studio persists the selected mode (Hermes, Demo, Local, Hermes3D or Custom) and displays the active backend reported by the connected gateway. The README describes the runtime seam as backend-neutral so additional providers can be integrated without rewriting the UI, which is a claim about the intended extension path rather than a documented API contract.

Installing Hermes3D from source and opening the office

The README recommends Node.js 20+ and npm 10+. Note the disagreement with the Dockerfile, which states that Node 22 is required because the runner ships next.config.ts without the typescript devDependency, so Next auto-installs it at startup and pulls transitive dependencies that need Node >=22; on Node 20 that install fails and the app crash-loops before binding to a port. For a container build, treat Node 22 as the floor.

Run from source as the README shows. The clone line uses a placeholder repository URL in the README itself, so substitute the real one.

git clone <your-public-repo-url> hermes3d
cd hermes3d
npm install
cp .env.example .env
npm run dev

Then open http://localhost:3000 and configure the gateway URL and token in Studio. The dev script is not plain next dev: package.json maps dev to node server/index.js --dev, which is why the custom proxy exists in development too.

The environment file distinguishes two gateway variables. NEXT_PUBLIC_GATEWAY_URL is baked in at build time, so changing it requires a rebuild. HERMES3D_GATEWAY_URL takes effect on restart without a rebuild, and the example file recommends it when you want to move the endpoint. HERMES3D_GATEWAY_ADAPTER_TYPE accepts hermes, hermes-agent, demo or custom, and tells Studio which backend the gateway URL represents.

For a first run with no agent framework at all, start the bundled demo gateway in one terminal and the app in another:

npm run demo-gateway

The example file sets DEMO_ADAPTER_PORT=18789 and notes that if HERMES3D_GATEWAY_URL is unset, Studio can still detect this local adapter port. The default NEXT_PUBLIC_GATEWAY_URL in .env.example is ws://localhost:18789, so a fresh checkout with a copied .env should find the demo gateway without further configuration. What you should see is the retro office with agents moving through it.

For a real backend, the adapter path is the alternative: run npm run hermes-adapter, then connect Hermes3D to ws://localhost:18789, with HERMES_API_URL, HERMES_API_KEY and HERMES_MODEL set in the environment. The README also points to TUTORIAL.md for a cross-machine setup with Hermes and Tailscale, and to docs/hermes-agent-tailscale.md for connecting to a remote hermes-agent backend with no adapter process. There is a doctor script, npm run doctor, if the wiring does not come up cleanly.

The 2D fallback and what it says about the 3D default

Hermes3D ships a second renderer. A Gather-style 2D pixel office shows the same live agents for low-power machines, reachable from Settings under Office renderer or from the 2D button in the office toolbar. The dependency list backs this up: phaser sits alongside three, @react-three/fiber and @react-three/drei, so the project carries two rendering stacks rather than one.

Shipping a low-power mode is an admission that the 3D office is expensive. A scene with desks, rooms, navigation, animations and event-driven activity cues, rendered through react-three-fiber with postprocessing, will not run comfortably on a thin laptop or a small cloud VM. If your only goal is to see which agent is blocked on an approval, the 2D view is the cheaper surface and probably the better default on shared hardware. The README does not publish frame-rate targets or hardware requirements for either renderer, so the practical way to judge is to open both on the machine you intend to use.

Where Hermes3D is the wrong tool

Two failure modes are visible in the repository itself. First, Hermes3D cannot run without a backend unless you use the demo gateway. The README states the prerequisite plainly: the project does not install or build Hermes for you, and your chosen runtime must already be running with a known gateway URL and token. A team with no Hermes deployment and no orchestrator speaking the gateway protocol gets a walkable office and nothing else.

Second, the runtime seam is described in prose, not specified as a public contract. The README says the seam exists so additional providers can be integrated without rewriting the UI, but the protocol details a custom backend must implement are not laid out in the README; they live, if anywhere, in the linked docs and the adapter source. Budget integration time accordingly, and do not assume a generic REST or WebSocket agent service will work unmodified.

The project is also explicitly unofficial. The README calls it an independent, community-driven frontend that is not affiliated with or endorsed by the teams behind the agent backends it connects to. If you need vendor support for the whole stack, this is not that. Maintenance signals are reasonable rather than settled: version 1.0.0 was released on 2026-08-20 and the last push to main was on 2026-08-29, so the codebase is recent, but a 1.0.0 release on a project this young is a version number, not a stability guarantee.

Hermes3D compared with a plain three.js application

The nearest alternative is not another agent dashboard. It is writing the visualization yourself on three.js, which is already a dependency here. The difference is what each one gives you out of the box. A bare three.js project gives you a renderer and nothing else: you build the scene graph, the camera controls, the asset pipeline, the office layout and the event plumbing that turns backend messages into movement.

Hermes3D gives you the office, the desks and rooms, the navigation, the animations, the activity cues, the layout editor at /office/builder, and the WebSocket proxy that carries gateway events into the scene. You trade control for that: your agents appear as workers in someone else's office, and changing the fundamental look means working inside the react-three-fiber component tree rather than from a blank canvas. If you want a bespoke visual identity for your agents, three.js directly is the shorter path. If you want the office to exist today and care more about what it shows than how it looks, Hermes3D is the head start.

Licence, upgrade cost and what to check before adopting

The licence is MIT, declared in package.json and in the LICENSE file, and the Dockerfile labels the image with org.opencontainers.image.licenses="MIT". That permits commercial use and modification, with the usual requirement to keep the licence notice; it is not legal advice, and if you redistribute the container image you should read the LICENSE file yourself.

Upgrade cost is dominated by the Node version and the two-stage environment split. The Dockerfile pins node:22-slim across all three stages and explains why Node 20 fails, so a container upgrade path that moves off Node 22 will break. The build-time versus runtime gateway distinction is the other recurring cost: NEXT_PUBLIC_GATEWAY_URL requires npm run build after every change, which is exactly the friction HERMES3D_GATEWAY_URL exists to remove. Use the runtime variable for anything beyond a first local run.

The project publishes a CHANGELOG.md, a ROADMAP.md and a SECURITY.md, and the example environment file lists STUDIO_ACCESS_TOKEN as required only for public or remote deployments. If you expose Studio beyond localhost, set it. Verify three things first: that your gateway implements the Hermes3D gateway protocol, that your Node version matches the Dockerfile's Node 22 requirement, and that the demo gateway renders on your target hardware before you invest in the adapter path.

Editorial conclusion

Adopt Hermes3D if you already run a Hermes-compatible gateway and want a spatial view of approvals, standups and PR review instead of another log pane, and if you accept that the office is a frontend with no runtime of its own. Skip it if you have no gateway to point at, if you need a general-purpose game engine, or if you expect the project to build Hermes for you. Verify first that your gateway speaks the Hermes3D gateway protocol, that your Node version matches the Dockerfile's Node 22 requirement, and that the demo gateway renders on your machine before you wire in a real backend.

Frequently asked questions

Official sources

1. iamlukethedev/Hermes3D on GitHub
2. Issues
3. License: MIT
4. README
5. Releases