Robonix: a language model writes the program a robot runs

The model writes a program, and the program is the artifact

Robonix uses large language models and vision-language models to convert a natural-language task into a Robot Task Description Language program, which is then executed. Everything else in the design follows from making that program a real, named thing rather than a stream of tool calls nobody can inspect afterwards.

The third of the three stated advantages says it outright: the model's program is explicitly represented in RTDL so its structure can be inspected, the system performs program validation prior to execution, and it can verify specified outcomes when the run completes. A robot that can be asked why it did something, and checked against a declared goal, is a different proposition from one that simply acts.

The second advantage is about generation rather than verification. The program is written for the specific task, the capabilities that robot actually has, the skills registered on it, the environmental conditions and the body state, which is why the same prompt produces different RTDL on different hardware.

Four layers sit between a goal and a motor command

The vocabulary is four words, and the definitions matter more than the marketing around them. A primitive is a single hardware function, sensing or actuation, behind a software interface. A service is reusable software implementing a Robonix interface, such as planning, execution, interaction or scene management. A skill is a learned model or an algorithmic procedure packaged as a reusable executable unit. A task is a goal, its constraints, and the conditions that count as completion.

The mechanism that makes this more than a diagram is contract registration. Every primitive, service and skill declares what it offers in a versioned contract and registers with Atlas, the capability catalog, at startup. Callers bind to that contract rather than to the implementation underneath.

Two consequences follow. A task written against one body can transfer to another, and an implementation can be swapped out without modifying any caller. That is the hardware and software decoupling the project leads with, expressed as a registry and a versioned interface rather than as an abstraction layer you are trusted to use correctly.

Soma is where a robot's body is declared

At the bottom of the architecture sits Soma, the body abstraction, and it is the part that decides whether a new robot is a project or an afternoon.

The body is described twice. A URDF file gives the geometry and the coordinate frames. A YAML body description gives the component tree, so base, arm, gripper and camera, and declares which primitive capabilities each component provides. Everything above that layer, services, skills and tasks, then binds to the contracts rather than to a specific arm.

Where Robonix meets Soma is also where the vendor's own preset skills sit, described as things like a dance or a backflip. Those ship with the body, which is a useful reminder that a body description is not only a kinematic statement: it is also where a manufacturer attaches what the robot already knows how to do.

Skills, as opposed to presets, live in the library beside Soma, and that is where the simulation story starts.

Twelve components are described, seven are Rust crates

The architecture is a set of named components with distinct jobs. Liaison is the human and machine interaction layer and verifies the user with Keystone before delegating the task. Pilot handles planning, decision, memory and the world model, and it is the component that writes and checks the RTDL program. Executor does capability orchestration, with Sentinel monitoring it. Scene keeps the semantic map of objects and their relations alongside the spatial map of the world.

The supporting set is Atlas for the capability catalog, Nexus for messaging, Chronos for time, Scribe for logging and Vitals for health. Beside Scene sit the custom services from the package catalog: navigation, SLAM mapping, object detection, grasp-pose estimation, motion planning and memory.

The Cargo workspace at the root has nine members, seven of them under system/ plus the rbnx CLI and a codegen tool under tools/, while a comment in the same file says twelve system components live one per directory there. The same file says Executor embeds Sentinel for 0.1, while the architecture description treats Sentinel as a component of its own. Two documentation drift points, both in comments, both worth resolving before you rely on either.

Python gets a workspace and a venv per package

The Python side explains itself unusually well in comments. The root pyproject.toml turns the repository into a uv workspace, where every listed member shares one resolution and one uv.lock while each package keeps its own venv under rbnx-build/venv/.

The reason given is dependency conflict rather than preference. The packages carry conflicting heavy dependencies, memsearch with onnx, PyAudio for audio and torch for a future vision-language-action skill, and a shared venv would block on the worst of the versions. The workspace also lets a package reference shared in-repo libraries with workspace = true instead of hard-coding relative paths.

The members are the shared libraries under pylib/, the Scene component, and the memory, memsearch, speech and voiceprint services plus the verifiers, with services/verifiers explicitly excluded from the workspace. So there are two build systems over one tree: cargo for the Rust crates and uv for the Python half, coordinated by the same Makefile.

Three terminals and one VLM endpoint

Robonix does not install its own prerequisites, and the list is worth reading before anything else: Rust stable, uv, Docker for the Webots simulator stack, and system packages, installed with sudo apt install build-essential pkg-config libssl-dev git curl alsa-utils. On a machine with an NVIDIA GPU you also need the NVIDIA Container Toolkit, because Scene runs its detector in a container with --gpus all and that flag fails without it.

Then the build, which must be a recursive clone because the repository carries git submodules:

git clone --recursive https://github.com/syswonder/robonix.git
cd robonix
make install

make install builds the Rust components and the rbnx command line interface into ~/.cargo/bin and registers the repository through rbnx setup. Three terminals follow. One starts the simulator with DISPLAY set and bash examples/webots/sim/start.sh. Another boots the system:

export RMW_IMPLEMENTATION=rmw_zenoh_cpp
export VLM_BASE_URL=https://api.openai.com/v1
export VLM_API_KEY=sk-...
export VLM_MODEL=your-model-name

cd examples/webots
rbnx build
rbnx boot

The third runs rbnx chat, where the documented first attempts are go to room 101, what can you see? and explore the office. Any OpenAI-compatible VLM endpoint will do, which is the one piece of external dependency the whole loop has.

Beta version, dev branch, and a simulator-first workflow

Version 1.1.0-beta.1 appears in the Cargo workspace and 1.1.0b1 in the Python root, with tags v0.1-rc1 from 2026-05-15, v1.0.0 from 2026-08-14 and v1.1.0-beta.1 from 2026-10-01. The last push to the repository is dated 2026-10-01 and it is not archived.

The default branch is dev, while the licence badge in the README points at a blob/main/LICENSE path. If you clone the default branch you get dev, and if you follow the README's links you get main, so know which one you are reading before comparing a file.

Skills are trained away from hardware. The library holds picking, handing over an item, pouring and folding, many of them vision-language-action models, and they can be learned in Webots, MuJoCo or Isaac Sim and then deployed to real robots through Robonix. Simulation first is not a convenience here, it is the only documented path from a skill to a body.

The licence is MulanPSL-2.0, declared in the Cargo workspace, in the SPDX header of the Python root and in the Makefile, and the tree also carries AGENTS.md, CLAUDE.md, MAINTAINERS.md, CHANGELOG.md, pyrightconfig.json, capabilities/ and testing/.

Editorial conclusion

Robonix fits a robotics team that wants the generated plan to be an inspectable artifact and that already has a body abstraction in mind, whether that is a simulated arm or a real one. It does not fit someone who wants a working robot this afternoon, because the quick start needs a VLM endpoint, a simulator and per-package Python environments. Before you build, check the default branch, which is dev while the documentation links point at main.

Frequently asked questions

Official sources

1. License: MulanPSL-2.0
2. Project website
3. README
4. Releases
5. syswonder/robonix on GitHub