LLM Space 4: a desktop workbench for tracing and replaying agent runs

What LLM Space 4 is for

Agent development has a habit of turning into guesswork. You write a prompt, wire up a few tools, run the loop, and when the output is wrong you read a wall of text and try to remember which call produced it. LLM Space targets that gap. The README describes it as a desktop app for agent builders: prototype an idea, inspect every step of harness execution, debug failures, and evaluate performance in one place.

The audience is narrow and specific. This is for people who already write prompts, tools and model settings and want to version them, not for someone looking for a chat client. The README also notes that the project is a sister project of DeerFlow and that every DeerFlow version is built and debugged with LLM Space. That is a strong signal about intended use: the tool exists to serve its own maintainers first, and the feature set reflects the work they actually do. The project started in March 2023, and v4 is the fourth major iteration.

The harness loop: build, trace, replay, evaluate

The mechanism is a local runtime wrapped in a desktop shell. According to the README, the desktop app is an Electrobun shell with a React UI, and the agent framework underneath is Pi Agent Core. The monorepo splits the work: packages/core holds shared domain types, clients, storage and generators; packages/runtime holds the local runtime, models, tools, skills, MCP and Plugins; packages/ui holds the shared React design system and Thread Playground; apps/desktop is the shipped app.

That split matters when you are debugging. Tracing is not a separate observability service bolted on after the fact; it is part of the runtime that executes the loop, which is why the README can promise that you see every model call and tool run inside the agent loop as it happens. Replay works from history, so a run you already finished can be stepped through again. Evaluation sits on top of the same history.

Storage is the other half of the design. Threads are kept as files on your own machine, and the README states that your files and API keys stay on your local computer. The app does collect a small amount of anonymous usage data, and TELEMETRY.md documents exactly what is collected and how to opt out. Read that file before you assume the app is fully offline.

Installing LLM Space from a DMG or from source

The fastest path is the prebuilt app. The README points to the latest release and says to grab a DMG for macOS, Apple Silicon and Intel. Two editions ship: LLM Space uses the system WebView and is around 27 MB, while LLM Space Performance embeds its own rendering engine at around 130 MB. The README says you can install either or both, that they share the same ~/.llm-space data, and that both update themselves in place. If you switch editions, your threads and settings come with you.

Building from source needs Bun first. The README recommends the official install guide, then this command from the repo root:

bun install

If you want the exact toolchain CI uses, the README says to install mise and run the setup task instead, which installs the locked Bun version from mise.lock plus JS dependencies in one step:

mise run setup

To start the desktop app for local development, the README gives one command:

mise run dev

The package.json shows a second route without mise: `bun run dev` changes into apps/desktop and runs dev:hmr. There is also a canary build, `mise run build:canary`, and a web target through `bun run dev:web` and `bun run build:web`. Expect the first launch to ask for a model provider; the README only names BytePlus's Coding Plan as a recommended default, and does not walk through configuring other providers.

Plugins, Skills and the Atlas example

Extension points are where LLM Space tries to be more than a viewer. The README lists Skills, MCP servers, model providers, Plugin Tools, Commands and Thread Storages as things a Plugin can contribute, along with a multi-field Settings schema. The repository ships examples/atlas-plugin, described as a complete Plugin example covering every Extension type, with two Skills, MCP servers, model providers, Plugin Tools, Commands and Thread Storages. If you want to know what the extension surface really allows, that directory is more informative than the README, which stays at the level of a feature list.

The generate feature is the other unusual piece. The README says the app can let AI write your prompt and tools for you, and can turn any thread into a runnable LangGraph agent. The package.json includes a gen:langgraph-tools script, which suggests the LangGraph export path is maintained rather than incidental. Treat generated agents as a starting point: you still need to read what was produced before you trust it in a loop.

Where LLM Space is the wrong tool

Contribution is the first constraint. The README states plainly that only pull requests from DeerFlow core team members are merged, and that everyone else should open an issue instead. This is a source-available project in practice even though the licence is MIT. If your team needs to fork and carry patches, the licence permits it, but you will be maintaining that fork yourself, and the upstream will not take your fixes back.

Platform is the second. The download section describes DMG files for macOS only. Nothing in the README promises a Windows or Linux build. If your engineers run Windows, the desktop app as described is not for them, and the web target in package.json is a development path, not a documented deployment.

Scale is the third. Threads are files under ~/.llm-space, which suits one person on one machine. There is no mention of a shared server, team accounts or hosted history. The README describes the project as local-first and cloud-ready for managed agents, but the cloud half is not spelled out, so do not plan around it. Finally, if your problem is aggregate production monitoring across many services, a tracing backend built for that job will fit better than a desktop app that reads a local thread history.

How it differs from LangSmith and plain LangGraph

The obvious comparison is a hosted tracing platform such as LangSmith. The difference is where the data lives and what the tool is shaped like. A hosted platform collects traces from your services into a shared backend and gives you a web UI for querying across them. LLM Space runs on your machine, keeps threads as files, and is built for the loop of one developer iterating on one agent. The trade-off is real in both directions: you get local keys and file-based history, and you give up cross-service aggregation and any shared view for a team.

A second comparison is LangGraph itself. LangGraph is a framework for expressing agent graphs; LLM Space is a workbench that can export a thread into a runnable LangGraph agent, according to the README. They are not substitutes. If you already have a LangGraph service in production, LLM Space does not replace it or attach to it as a tracer; it is a separate place where you prototype and inspect runs before the code goes out.

Maintenance, licence and upgrade cost

The repository is not archived, and the last push was on 2026-08-29, the same day as the v4.15.2 release. The release list shows v4.15.0 on 2026-08-27, v4.15.1 and v4.15.2 on 2026-08-29, so the project ships frequently and in small increments. That cadence cuts both ways: fixes arrive quickly, and the surface you depend on can shift between minor versions. The CHANGELOG.md and .versionrc.json at the repo root are the files to read before upgrading.

Upgrading the packaged app is close to free, since the README says both editions update themselves in place and share the same data directory. Upgrading a source build means matching the locked Bun version in mise.lock and re-running the install, and if you maintain a Plugin, re-checking it against the current Extension types. The typecheck script in package.json runs tsc across every workspace including examples/atlas-plugin, which is a useful signal that the example is kept compiling.

The licence is MIT, which permits commercial use, modification and redistribution with the copyright notice and permission notice retained. That is a permission grant, not legal advice; if you ship a modified build, have your own counsel review the notice requirements and the telemetry behaviour described in TELEMETRY.md.

Editorial conclusion

Adopt LLM Space if you build agents and want your threads stored as files under ~/.llm-space while you watch each model call and tool run, or if you want to replay a failed run step by step instead of guessing from logs. Skip it if you need a browser-based team service, or if you expect to land a patch: the README states that only pull requests from the DeerFlow core team are merged, so outside contributions go through issues. Before you commit, open docs/core-concepts.md and docs/plugins.md, then check that your provider works through the runtime package, since the README only names the BytePlus Coding Plan as a recommended provider.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes