URLab: MuJoCo physics inside the Unreal Engine 5 editor

Two simulators, one robot, on two threads

UnrealRoboticsLab, shortened to URLab, is an Unreal Engine 5 plugin that embeds the MuJoCo physics engine directly into the editor and runtime. The README's framing is that you simulate robots with MuJoCo's accurate contact dynamics and render them with Unreal's lighting, materials and cameras.

That combination is the whole idea, and it exists because the two engines are good at different things. Unreal has a rendering pipeline that produces images people and perception models can be trained on, with real materials, real lighting and real camera models. MuJoCo has contact dynamics that most game physics engines approximate badly, which matters enormously for manipulation and locomotion, where the outcome often turns on whether a foot actually grips the ground.

The architectural consequence is stated in the feature list: physics runs on its own thread while Unreal renders, so you get accurate contacts and photorealistic output together. This is the design decision that makes URLab more than a wrapper, and it is also the source of its main constraint, which is that a physics step and a render frame have to be reconciled in time.

The project description calls it a high-fidelity, open-source robotics simulator integrating Unreal Engine's photorealistic rendering with MuJoCo's precision physics. The primary language is C++, the licence is Apache-2.0, the default branch is main, and the documentation site is urlab-sim.github.io/UnrealRoboticsLab. There is an ICRA 2026 paper with the arXiv identifier 2504.14135, by Embley-Riches, Liu, Julier and Kanoulas, and the README asks that it be cited if URLab is used in research.

The repository tree confirms the shape of an Unreal plugin: a UnrealRoboticsLab.uplugin descriptor at the root, then Source/ for the C++ plugin code, Config/, Content/ and Resources/ for Unreal assets, Scripts/, third_party/ for the bundled dependencies, and docs/ with a mkdocs.yml for the documentation site.

Importing MJCF models and what the Articulation Blueprint gives you

The model import path is where most of the practical value sits, and it is described concretely.

You drag an MJCF .xml into the Content Browser and URLab builds a ready-to-simulate Articulation Blueprint, with support for joints, actuators, tendons, muscles and flexcomp. That is a full translation of MJCF's feature set into Unreal's asset system, not just geometry. Joints become constraints, actuators become force or position driven elements, and tendons and muscles become deformable couplings rather than rigid links.

flexcomp is the one worth calling out, because it is the soft-body path in MuJoCo, and having it survive the import means a deformable object in the simulation is not approximated by an Unreal skeletal mesh hack. That matters for contact-rich tasks where the deformation is the contact.

The alternative entry point is lower friction. You can turn any static mesh into a physics body with a single component, which is how you get environment geometry, props and obstacles into the simulation without authoring them in MJCF.

Two tools make the result legible. The debug feature list includes body-island and segmentation overlays, tendon and muscle rendering, collision overlays, and mouse-driven body perturbation. Body-island overlays in particular are the difference between debugging a simulation and guessing at one, because contact islands show which bodies are currently interacting, which is usually the question you have when a robot is not doing what you expected.

Mouse-driven body perturbation is a debugging tool that also has a use in data generation: you can push a robot around and record what happens, which is a cheap way to produce perturbed trajectories for robustness training.

The documentation index lists the guides as covering importing, articulations, sensors, cameras, controllers and debug, so each of these areas has its own page in the mkdocs site.

Controlling actuators from Blueprints, Python and ROS 2

The control surface is deliberately wide, and the choice tells you what the plugin is for.

The README says you control robots however you like, driving actuators from Blueprints, the in-editor dashboard, or Python and ROS 2 over the network, with PD controllers and live gain tuning. Three paths, for three different users: a technical artist tuning a scene in the editor, a researcher iterating on a policy in Python, and a robotics stack that wants ROS 2 topics.

Live gain tuning is the detail that makes the in-editor path viable. Changing a PD controller's gains while watching the robot respond in the viewport is a tight loop, and it is how you find out whether a controller is stabilising the robot or merely not destabilising it yet.

The Python and ROS 2 path runs over ZMQ, with a companion package urlab_bridge in a separate repository providing Python middleware for remote control, policy deployment and ROS 2 bridging. Separating the bridge from the plugin is the right split, because the plugin has to stay a C++ Unreal plugin while the bridge is an ordinary Python package you install into your policy training environment. The documentation page for Python and External Control covers driving URLab from Python and running and evaluating policies.

The network design has a consequence worth stating plainly. Because control comes in over ZMQ rather than through an Unreal-native mechanism, the simulation and the policy can run on different machines, or the same machine, and the bridge is a network component in the middle. That is what makes policy evaluation in the loop possible, and it is also why the plugin needs libzmq bundled as a submodule.

The Python requirement is listed as optional, for urlab_bridge, and the version floor is Python 3.11.

Sensors, cameras, and deterministic record and replay

Sensor data is captured from two places at once, which is what makes the rendered images usable for perception work.

The README describes rendering RGB, depth and segmentation from robot cameras with real camera intrinsics, alongside MuJoCo's full sensor suite. The distinction matters. Unreal gives you images that look like photographs, with a depth buffer and a segmentation mask derived from the scene's actual geometry, and the camera intrinsics are the ones a real camera of that field of view would have, which is what you need if a policy is going to be trained on those images and later run on a physical camera. MuJoCo's own sensors give you the physically meaningful quantities, contact forces and joint and actuator states, which the renderer cannot produce because the renderer does not know them.

Record and replay is the other capability, and it is described as capturing entire simulation episodes and playing them back deterministically. Deterministic replay is what makes a simulator usable as a test fixture. If a policy evaluation produced a surprising result, being able to replay the exact episode and get the same result is what lets you debug it rather than re-running and hoping for the same trajectory. Given that the physics runs on its own thread, reproducibility is a deliberate property rather than a lucky accident.

Together these cover the loop most robotics learning pipelines need: render realistic observations, read ground-truth physics, actuate through a controller, record the episode, replay it exactly.

Installing it into an Unreal project

The install is a plugin clone with submodules plus one dependency build, and the constraint that shapes it is that URLab is a C++ plugin.

Your project must be a C++ project. Clone it into your project's Plugins folder with submodules, build the native dependencies once, then build your project.

cd "YourProject/Plugins"
git clone --recurse-submodules https://github.com/URLab-Sim/UnrealRoboticsLab.git
cd UnrealRoboticsLab/third_party
./build_all.sh        # or .\build_all.ps1 on Windows

The build_all script is where the heavy lifting happens, because MuJoCo, CoACD and libzmq are bundled as submodules and built from source. CoACD is there for convex decomposition, which is how you turn an arbitrary mesh into a set of convex pieces that a physics engine can collide against efficiently, and libzmq is the transport for the Python and ROS 2 control path.

The requirements list is where you find the real constraints.

Unreal Engine 5.7 is the only tested and supported version, and the README tells you to avoid 5.8 because of Vulkan driver regressions on Linux. That is a precise and useful warning: it means the plugin was validated against one engine version and a known breakage exists in the next one. Windows is Win64 with Visual Studio 2022 or 2025, and Linux is x86_64 with Unreal's bundled clang. CMake 3.24 or newer is needed to build the dependencies.

Since Unreal projects cannot use standard CI, the contributing docs ask that each pull request include proof of a local build and passing tests, which tells you something about what a contribution to this repository actually costs.

The documentation site has a full Windows and Linux walkthrough on its Installation page, and a Quickstart page for importing a robot and running a simulation.

Where URLab is the wrong choice

There are three limits, and the first is the engine version lock.

The first is that Unreal Engine 5.7 is the only supported version and 5.8 is called out as problematic on Linux. For a plugin, being tied to one engine version is normal, but being told to avoid the next one means your upgrade path is blocked until the project says otherwise. Combined with the requirement for a C++ project, a Visual Studio 2022 or 2025 install on Windows, and a CMake toolchain to build the bundled dependencies, the entry cost is high. This is not a plugin you drop into a blank project on a laptop in an afternoon.

The second is the two-engine boundary. You have Unreal's renderer and MuJoCo's dynamics, and the seam between them is where the compromises live. MuJoCo simulates its own scene graph, and Unreal renders a scene graph, and the plugin has to keep them consistent. When a robot behaves oddly, the usual question is which engine is responsible, and that question is not always cheap to answer. There is also no headless rendering story described here: Unreal is a heavyweight editor and runtime, and the README's requirements say nothing about a build server or a farm, so anyone planning to render thousands of episodes in a cluster has work to do that is not documented.

The third is that the Python side is a separate repository. urlab_bridge lives at URLab-Sim/urlab_bridge, so a policy workflow means installing two things and matching their versions, and the plugin's own Python support stops at the bridge.

The honest alternative depends on which half you need. If the problem is photorealistic images for perception training and the physics does not need to be accurate, Isaac Sim from NVIDIA does both natively in one engine with its own GPU-accelerated physics, has a mature Python API, and runs headless on a render farm. The difference in approach is that Isaac Sim owns the whole stack, whereas URLab deliberately bolts MuJoCo's solver onto Unreal's renderer, which is why you get MuJoCo's contacts but also Unreal's constraints. If your task is manipulation where contact fidelity decides success, that trade is worth it. If your task is driving a car through a city and the pixels matter more than the tyre model, Isaac Sim is the simpler answer.

If you need MuJoCo specifically, the plain MuJoCo itself with its own viewer and offscreen rendering is the alternative, and it is dramatically simpler: one install, no Unreal, no plugin. What you give up is Unreal's rendering quality, its camera model and its materials, which is exactly the thing URLab was built to give you.

Editorial conclusion

Adopt URLab if you need photorealistic rendering of robots whose contact behaviour you care about, since MuJoCo's dynamics running on their own thread is the point of the plugin. Do not adopt it if you need Unreal 5.8 or a headless build on a platform other than Windows x64 or Linux x86_64, since those are outside what the README says is supported. Before you start, confirm your engine version, run build_all.sh in third_party, and note the last push was on 2026-10-11 with no tagged releases.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. URLab-Sim/UnrealRoboticsLab on GitHub