Surf: Wiring OpenAI's Computer Use Model to an E2B Desktop Sandbox

The gap Surf fills: a streamed desktop that a model can actually drive

Running an agent that clicks through a graphical interface requires two things that are awkward to combine: a machine the agent cannot damage and a way to watch what it does. Surf puts those together in one repository. The README describes it as a Next.js application that integrates E2B's desktop sandbox with OpenAI's API so an agent can perform tasks on a virtual computer through natural language instructions. The target reader is a developer evaluating computer use agents, not an end user. There is no hosted signup flow described, no user accounts, and no persistence layer mentioned. You clone it, add two API keys, and run it locally. That framing matters when reading the rest of the README, because several sections describe behaviour in demo terms: a timer, example prompts, a dark and light theme. The scope is deliberately small.

How a chat message becomes a click inside the sandbox

The flow documented in the README has seven steps and one endpoint. A user starts a sandbox, E2B creates a virtual desktop and returns a URL for streaming, the user sends an instruction through the chat interface, and the backend processes that instruction with OpenAI's API. The model then generates actions such as clicks and typing, those actions execute on the sandbox, and the results stream back to the frontend. The loop repeats. On the server side the README names a single API route, /api/chat, which handles chat messages and streams AI responses and actions. Three server actions are listed separately: createSandbox, increaseTimeout, and stopSandboxAction. That split is the clearest architectural signal in the material: sandbox lifecycle is managed through Next.js server actions, while the agent loop runs through a streaming route. The README states that Server-Sent Events carry the stream. It does not describe the event schema, how partial action results are ordered against model output, or what happens to an in-flight action when the sandbox stops mid-step. Anyone building on this should expect to read the route handler rather than the README for those answers.

Setup is two keys and one command, and that is the whole story

The README's setup path is short. Clone the repository, run npm install, create a .env.local file based on the provided .env.example, and start the development server with npm run dev. The environment file needs exactly two values: E2B_API_KEY and OPENAI_API_KEY. The app then serves on http://localhost:3000. Prerequisites are Node.js at a version specified in package.json, npm, an E2B API key, and an OpenAI API key. Note what is absent. There is no configuration key for choosing a model, no timeout value exposed as an environment variable, and no base URL override for routing OpenAI calls through a gateway or proxy. If you need any of those, they are code changes, not configuration. The README also does not state the Node version directly, only pointing at package.json, so check that file before assuming your local runtime matches. The troubleshooting section covers three cases and all three are credential or state problems: a wrong E2B key, an invalid OpenAI key or a key without access to the required models, and actions failing because the sandbox is not running or the instructions are unclear.

Sandbox lifetime is the part the README handles least clearly

Under usage, the README says a timer shows the remaining time for the sandbox instance, that you can stop it with a Stop button, and that the sandbox will automatically extend its time when it is about to expire. The increaseTimeout server action is listed among the server actions, which suggests the extension is a real call rather than a client-side illusion. What the README never states is the initial timeout, the extension increment, or whether there is a ceiling on how many times a session can extend itself. That matters for cost and for failure modes. A long agent run against a paid desktop sandbox is a metered resource, and the README gives no per-minute or per-session figure. The same silence applies to concurrency: nothing in the material describes what happens if two browser tabs each start a sandbox, or whether a stopped sandbox releases its resources immediately. Treat the timer as a UI affordance whose server-side policy you have to read out of the code. This is the single largest gap between what the README promises and what an operator needs to know.

Where Surf is the wrong tool

Surf is a browser-driven agent. Every action travels through a streamed desktop image, which means the agent's feedback loop depends on screen state rather than structured output. For tasks that have an API, a shell, or a file format, that is the expensive path: you pay for a desktop session and you pay for vision-capable model calls, and you accept the latency of rendering a frame before the model can decide what to do next. The README's own example instruction, opening Firefox and navigating to google.com, is a good illustration of the intended shape of work. If your task is reading a CSV, calling an HTTP endpoint, or running a build, a plain tool-calling agent on the host is cheaper and more reliable. There is a second boundary. Surf does not appear to be packaged for deployment. The README describes a development server and a localhost URL, with no build, container, or hosting instructions. Running it in production would mean designing the sandbox lifecycle, session isolation, and key handling yourself, and none of that is documented here.

The alternative: browser-only agents and the difference in approach

The closest alternative in this space is a browser automation agent built on Playwright or a similar driver, where the agent manipulates a DOM through a headless browser rather than a full desktop. The difference is not cosmetic. A browser agent receives structured signals such as element selectors and page events, so it can act without a screenshot round trip and can retry a failed click deterministically. Surf instead operates at the pixel and input level of a whole Linux desktop, which is why it can drive native applications that never touch a browser. That is the trade: broader reach into arbitrary GUI software, in exchange for a slower and more expensive perception loop and no structured view of what is on screen. If your target is a web app, the browser agent is the better fit. If your target is a desktop application with no scripting interface, Surf's approach is the one that generalises. The README does not benchmark either path, and it does not claim Surf is faster.

Licence, maintenance, and what you inherit

Surf is Apache-2.0, so you can use, modify, and redistribute it, including in commercial settings, subject to the terms of that licence. Read LICENSE for the actual text rather than relying on the summary here. The repository is not archived and the last push recorded is 2026-09-09, which indicates the project was still receiving commits at that point. No releases were retrieved, so there is no tagged version to pin against. That shapes the upgrade story. Without releases you are tracking a branch, and the surface you depend on includes a Next.js app router, a streaming API route, and three server actions, all of which are application code rather than a published library. Upgrading means diffing your fork against main. The dependency set named in the README is Next.js, @e2b/desktop, the OpenAI SDK, Tailwind CSS, and Framer Motion, and the README defers the complete list to package.json. Two of those move quickly and independently: the OpenAI SDK and the E2B desktop SDK. Expect the maintenance cost to sit mostly in keeping those two aligned with the code that calls them, not in the UI layer.

Editorial conclusion

Adopt Surf if you want a working scaffold for computer use agents and are comfortable reading the Next.js source to find the pieces the README does not spell out. Do not adopt it if you need a supported product, a documented model list, or a clear answer on sandbox billing. Before writing real code against it, verify which OpenAI computer use model the code calls, how the sandbox timeout is extended, and what an E2B desktop session costs per minute.

Official sources

1. e2b-dev/surf on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README