This Minecraft agent plans with one model and acts with another, and keeps the evidence

Two models with different jobs, not one model doing everything

The project description is precise about the architecture. GPT-6 Astra or GPT-5.6 Sol plans and JEV selects player actions in Minecraft Java 1.16.5, using the official vanilla server and Mineflayer. The models are given a role split, and the split is the design.

The planner sets the current objective, the item targets and a travel waypoint. JEV then selects one available action from the current game observations. The available actions include travel, mining one block, collecting a drop, crafting, opening a chest, eating, sleeping and combat interactions. Mineflayer handles the movement path and the game protocol underneath.

The model identifiers are listed in the documentation. The planner runs as openai/gpt-6-astra by default, or openai/gpt-5.6-sol through the PLANNER_MODEL variable, both through /api/v1/chat/completions. The action model is typesafe/jev-1.13, through /api/alpha/decisions.

What makes this interesting is the framing in the documentation: this is structured state control, not control from screenshots or individual key presses. The models can see blocks in loaded chunks, and known seed coordinates are supplied to them. The action space is a small set of verbs rather than a stream of mouse moves, which is why a planner can stay abstract and an action model can stay small.

Each action is then sent to the game through the normal player protocol, and the project states that it does not change game rules or entity state.

A run is only a result once the evidence files agree

The verification story is what separates this from a demo script, and the documentation treats it as the main deliverable.

Each run writes runs/<run>/events.jsonl, which stores the requests, the model responses, the selected actions, the game results and the completion events, plus a status.json holding current progress. Then there is victory.json, and the rule for writing it is strict. It requires dragon death evidence and the exit portal event. Death evidence is either the kill advancement or a server supplied zero health with the dragon dying phase, and the Java 1.16.5 exit event uses numeric reason 4. A final review must also check the world DragonFight state and the video.

The last clause is the interesting one, and the documentation explains why. Bed kills can omit the kill advancement, so the advancement is not a sufficient signal on its own. A run that ends with the right video but without the DragonFight state checked is not verified, and a run that has the event but no review is not either.

The run also has to clear a set of checks. The documentation for the latest run states that all 17 run checks, eight route, camera and screen checks, and 29 local tests passed, and that the full earlier video and action log were reviewed before the local tests and the new recording.

So the definition of success in this project is a set of files, a set of checks and a review, not a claim.

The recorded run, in the numbers the documentation gives

The documentation states a latest verified result, and it is worth reading as a data point rather than a boast.

The latest run, named nether-final-08, completed in 8 minutes 43.300 seconds. The previous video took 14 minutes 31.800 seconds, so the new run was 40 percent shorter. The End combat stage took 152 seconds instead of 332. The documentation is careful to add that dragon flight and landing times can vary between runs.

The run started with an empty inventory in a fresh Survival and Peaceful world. Astra planned and JEV selected the actions. It used the Nether for travel, killed the dragon with six bed explosions on the first landing, and reached the exit with full health and no deaths. There was no restart, source change or operator repair during the run.

It used 131 JEV decisions and 35 Astra calls, which is a ratio of roughly four action selections for every plan. That number is the clearest single indication of the architecture: the planner is consulted occasionally to set a waypoint, and the action model does the stepping.

The comparison against the earlier run is also the honest part of the story, because the earlier one was recorded on a different stack. The documentation says the older recorded-06 video used Prismarine Viewer and a custom HUD, while the new recording uses the official Minecraft Java 1.16.5 client with a read-only local protocol mirror sending game state to it.

Two renderers, one of them the real client

The visual output is handled by a native client, and the reason for it is fidelity.

A read-only local protocol mirror sends game state to the official client, which renders the game with native entity models, textures, movement, digging and hand animations, and shows the hotbar, the held item, health, hunger, air and dragon health. The capture reads the game frame buffer, not the desktop, the window is hidden, and focus and cursor control are disabled.

The camera control is rate limited, with a limit of 240 degrees per second on each axis and an acceleration of 960 degrees per second squared, and it uses continuous turns. A hidden native Minecraft client renders the game without desktop input.

The video is 960 by 540 at 20 frames per second without audio. A translucent full-width top banner in Menlo shows the model names, the elapsed time in milliseconds, the XYZ coordinates, the planner objective and the selected action, and the documentation notes that these labels come from recorded responses and game events, so the overlay is derived from the same evidence files as the run rather than drawn on top by hand.

The older approach, Prismarine Viewer with a custom HUD, is still a dependency in the package manifest, alongside canvas, minecraft-assets, mineflayer, mineflayer-pathfinder and vec3, which is what the older recorded-06 video was made with.

A read-only sensor reports the dragon head position

The one piece of information the game does not naturally expose is the dragon's head position, and the project gets it from a Java sensor.

A read-only Java sensor can report the exact dragon head position, and the agent waits for that sensor before the first action. It is described as read-only, and the project states that it does not change game rules or entity state, so it observes without writing.

The sensor runs in its own process, with the DRAGON_SENSOR_URL variable pointing at it, and there is a separate observer/ directory in the repository tree alongside the combat-lab/ directory.

The combat-lab directory deserves a separate note because the documentation is careful about what it does not prove. It is a separate test server, and it can use prepared items and positions to test the combat code. A combat-lab result does not qualify as a full Survival run. That distinction is the point: the lab is where you develop and check the bed explosion and melee logic, and the full run is where the verification rules apply.

The route itself is also documented as surveyed. The documentation says the route was surveyed in a separate test world, and that final-run placements and movement use normal player interactions. The seed 8398967436125155523 is described as a speedrun seed with a village, three supply chests with 21 obsidian and a naturally active End portal, and the documentation is careful to say it is a useful speedrun seed, not proof of the easiest possible seed.

Starting a run means configuring secrets and local infrastructure

The setup is concrete, and it is also where the project's rough edges are.

Dependencies go in with npm install. The server is a jar, server/server.jar, started from the server directory with a compatible Java runtime, and the documentation notes that the installed Minecraft Java runtime works for this server. The main server runs on 127.0.0.1:25576 with the read-only sensor, using ./start-server.sh.

The agent itself is started with a few environment variables.

node optimization/nether/freeze-run.mjs NEW_RUN
PLANNER_MODEL=openai/gpt-6-astra RUN_ID=NEW_RUN \
NATIVE_VIEW=1 WAIT_NATIVE=1 NATIVE_RECORD=1 \
DRAGON_SENSOR_URL=http://127.0.0.1:3093 node nether-agent.mjs

A fresh recording needs a new world name chosen in server/server.properties and a new log directory with the same name. When the local mirror is listening on port 25578 you start python3 native-client/launch.py, and the agent waits for the hidden client and the recorder before it takes the first action. The native client uses an isolated game directory and does not change the user's Minecraft settings. The recorder stops after the exit portal event, and to stop a test at an action boundary you create runs/NEW_RUN/stop.

The credentials are handled through Google Secret Manager, with the key kept in process memory. The general OPENROUTER_API_KEY environment variable is ignored, on purpose, to prevent the use of an unrelated key, and the key is not written to run logs.

Two caveats are stated plainly. The scripts refer to the original deployment's project and secret name, and those references have to be configured for your own project in model-relay.mjs and models.mjs. The relay is started with MODEL_RELAY=http://127.0.0.1:3099, and you authenticate with Google application default credentials and grant access to the secret. And the native renderer and launch scripts were developed on macOS, so runtime paths and local server settings must be configured for another machine.

What stays out of the repository, and what the licence is

The repository is arranged around evidence that is deliberately excluded. The documentation states that recordings and generated evidence remain local and are excluded from Git. The local video for the latest run is runs/nether-final-08/full-playthrough-banner.mp4, and its verification, timing comparison and action log sit in the same run directory. The repository itself contains the code, the tests, the build scripts and the route data, and it does not include Minecraft binaries, downloaded runtimes, saved worlds or credentials.

That is a sensible arrangement for a project with large video artefacts and third party game content, and it also means a clone of the repository has no recording to look at. The evidence exists only for whoever ran the agent.

The licence is the gap. The project metadata lists the licence as unknown, and there is no licence statement in the documentation, so there is nothing in the repository telling you what you may do with the code. That is a real constraint for anyone considering building on it, and it is worth resolving with the author before the code goes into anything of yours.

The maintenance picture is otherwise clear. The repository is not archived, the last push was on 2026-09-20, and there are no GitHub releases, so there is no version to pin. The package manifest declares the project as private, version 0.1.0, with no test, probe and postinstall scripts beyond the standard ones, and a postinstall that runs patch-pathfinder.mjs and prepare-viewer.mjs.

The tree is flat and large, with a long list of top level modules named after the situations they handle, such as night-safety, ground-safety, corner-safety, breath-reflex, covered-shaft, fortress-platform and end-combat, which is a readable map of the run.

Editorial conclusion

minecraft-agent fits a developer who wants to study multi model game agents and to keep the run evidence rather than a screenshot, since the planner, the action selector and the verification layer are separated and every run writes events, status and victory files that a review can check. It does not fit a team that needs a licensed, supported product, since the project declares no licence and its own documentation says the credentials and runtime paths must be reconfigured for another machine. Before adopting it, set MODEL_RELAY and the secret references in model-relay.mjs and models.mjs for your own project, expect to build the native client and server on macOS first, and read the run checks and the victory file rules, since those are what turn a successful kill into a verified result.

Frequently asked questions

Official sources

1. Issues
2. README
3. rmalde/minecraft-agent on GitHub