threepp: the three.js API in C++20, with a Vulkan backend and simulated sensors

A three.js shaped API with a native scene graph

threepp describes itself as a cross-platform C++20 3D library with the high-level API of three.js, and the two backends behind one scene graph are portable OpenGL and a deferred Vulkan renderer with ray-traced shadows, reflections, AO and GI. That framing is the whole design in one sentence: the mental model, the object names, and the material system come from the browser library, and the difference is that here they run natively, on a desktop GPU, inside a simulation loop.

Who this is for is fairly specific. Roboticists and simulation engineers, people building synthetic datasets for perception models, and developers who already know three.js but need the same scene description in C++ or Python. The topic list gives it away in a good way: physics-simulation, gaussian-splatting, physx-5, warp, emscripten, usd, fbx, and python-library all appear alongside the graphics topics.

The project also says who it is not for. The project status section states that threepp is built for research, prototyping and teaching, not as a production engine, and that APIs change, so a tag or commit should be pinned if reproducibility matters. That sentence should decide your adoption before any feature list does.

The OpenGL path tracks three.js r129 and borrows from r186

The three.js parity layer is the part most users touch, and its scope is wide. The API follows three.js r129, with selected features from r186 on the OpenGL path: shadow filtering, environment and background controls, material fixes, and the newer post-processing passes. The result is a library that looks like three.js to anyone who has written a scene in the browser, with Mesh, InstancedMesh, Line, Points, Sprites and 2D and 3D text, the usual geometries, lights including RectArea, cameras, raycasting, animation, morph targets and bones.

Material work is taken seriously enough to include physical materials with transmission, alpha hash and alpha-to-coverage, PMREM environment maps and RoomEnvironment. Effects are covered by an EffectComposer with bloom, depth of field, GTAO, outline, FXAA and output passes, and the control set includes Orbit, Trackball, Fly, Drag and Transform.

The loaders are where a research project usually shows its real value, because asset pipelines are what break first. threepp reads glTF and GLB with meshopt, OBJ and MTL, STL, COLLADA, SVG, URDF and xacro with no ROS needed, PNG, JPEG, DDS, WebP, HDR and EXR, with USD, FBX and Assimp available as opt-in features. URDF import without a ROS dependency is a deliberate choice and it is the one that matters most for robotics work. Scenes serialize as three.js Object JSON or as a single file .tpz archive, and three.js editor documents load as they are.

The Vulkan backend is a deferred renderer with real ray tracing

Behind the OpenGL layer sits a deferred Vulkan renderer with a raster G-buffer, ray-traced shadows and reflections, ambient occlusion and probe global illumination, volumetric fog, denoising and temporal antialiasing, DLSS and FSR 3.1 upscaling, automatic mesh level of detail and GPU occlusion culling. Two specialised pieces stand out. The Ocean component is a three-cascade FFT water surface with foam, wind, JONSWAP fetch and an underwater view. ParticleField does GPU particles for fire, smoke, snow, rain and physics-coupled grains, drawn as meshes or billboards, or marched as participating media.

Gaussian splatting is supported through SplatCloud, which renders .ply and SOG scans on both backends and can bake a scan into a triangle surface so that physics and sensors can see it. That bake step is not optional decoration. The project status section states that Gaussian splats cast no shadows and are invisible to the ray-traced sensors unless they are baked into a surface, so a scan that looks correct on screen can still be missing from every synthetic label you generate.

The hard constraint is stated just as plainly: the Vulkan backend needs the Vulkan ray-tracing extensions, which in practice means an RT-capable GPU, and it does not run on macOS because MoltenVK has none. On macOS, and as the conservative choice elsewhere, the instruction is to use GLRenderer.

Ground truth labels come from the G-buffer, not from a renderer plugin

The perception features are the reason a lot of teams look at this library. Ground truth labels are produced from the G-buffer on the Vulkan path, giving metric depth, stable instance ids, semantic classes, normals and motion, and an addView call that renders a whole camera rig from a single scene build. That last point matters more than it first appears, because a multi-camera rig that shares one scene build is the difference between one render and a dozen.

The sensor suite is simulated rather than emulated by post-processing. There is lidar with VLP-16, HDL-32E, OS1-64 and OS0-128 patterns, imaging sonar, depth, colour and event cameras, IMU, joint encoders, contact and force and torque sensing. The physical camera model is explicit: exposure derives from aperture, shutter and ISO, OpenCV intrinsics are supported, lens distortion is applied to both the image and the labels, and sensor noise is added.

Two properties make this usable for reproducible work. The sensors are seeded and stamped with a simulation clock, so driving setSimTime from that same clock replays a frame bit for bit. And PhysX adds rigid bodies, articulations built from URDF and xacro with PD drives, tendons and cables, soft bodies, vehicles, character controllers, V-HACD convex decomposition and a damped-least-squares IK solver. Python drives soft and granular ground through NVIDIA Warp, including Bekker-Wong mud and snow and MLS-MPM grains that carry a wheel.

Installing the Python package and building a wheel

The Python side is published on PyPI, and the packaging comment in pyproject.toml explains how it is assembled. The build uses scikit-build-core, which drives the existing CMake build and produces a self-contained wheel through the pybind11 bindings for the GL backend. Installing from source is one command:

pip install .

To produce a redistributable wheel rather than install in place:

pip wheel . --no-deps -w dist

The comment also records what is in the wheel: the GL renderer, CPU PhysX physics with PhysxWorld, articulations and the proprioceptive sensors, and the Vulkan deferred renderer with its G-buffer AOVs. The extras are found quietly and optionally at configure time, so a machine with neither SDK still builds, GL only. The CI wheels provide the SDKs through CMAKE_ARGS hints in the .github/workflows/wheels.yml file.

Two behaviours are worth knowing before you install. The Vulkan loader is delay-loaded on Windows and auditwheel-vendored on Linux, so import threepp works on a machine with no Vulkan runtime, threepp.vulkan_available() reports the truth, and VulkanRenderer raises a clean RuntimeError where Vulkan is missing. And the scene editor ships prebuilt for Windows:

pip install threepp-editor

The editor gives a viewport, a hierarchy, an inspector, undo and redo, a PhysX-backed Play mode and Python behaviour scripts. Scenes save as plain three.js JSON with editor data in userData, so the same file also runs in an ordinary threepp program. A headless threepp_player exists for CI and batch episodes.

The GPU physics runtime is deliberately left out of the wheel

The single most useful limitation to know is written in the packaging comments as a deliberate exclusion. The wheel does not include the GPU physics runtime, and the reason is a file size limit: PhysXGpu_64.dll is 76 MB zipped against a PyPI cap of 100 MiB per file. The library is dlopened rather than linked, so a user who needs it can drop PhysXGpu_64.dll and PhysXDevice64.dll next to the installed module and use PhysxWorld with gpu_dynamics=True for direct-GPU reinforcement learning work.

The catch is that soft bodies run only on the GPU solver, so add_soft_body, bound since 2026.9.6, needs that same pair of files. A pip installation on a machine without them gives you rigid body physics and no soft bodies, with no compiler error to explain why.

The other limits are stated in the project status section. The editor is tested on OpenGL and its Vulkan viewport is best-effort. CI has no GPU, so the Vulkan backend runs there on lavapipe under the validation layers, which the project is careful to say gates spec correctness, not pixels. The Python package version is 0.1.0, and the releases are date named, with 2026-09-27, 2026-09-28 and 2026-10-11 in the recent list and a last push on 2026-10-19.

When three.js in a browser is still the better answer

The honest comparison is not with another C++ 3D library, it is with the library threepp is modelled on. If you want a three.js scene in a browser, three.js is the natural choice. threepp earns its keep in three situations: when the scene has to run natively rather than in a page, when a headless render has to happen in CI or in a batch loop with a Python driver, and when you need labels that only a real renderer can produce, such as the G-buffer depth, instance ids and semantic classes described above.

Within the library itself there is a second axis worth holding on to. GLRenderer is the conservative choice and the one that matches three.js semantics most closely, and it is what the editor is tested against. The Vulkan path buys deferred shading, ray-traced shadows and reflections, probe GI, TAA, denoising, upscalers, ocean and particle work, at the cost of requiring an RT-capable GPU, excluding macOS, and moving at a much faster pace than the port layer.

For build integration, the repository ships CMakeLists.txt, CMakePresets.json, a vcpkg.json manifest, a.clang-format file, examples/ as the de facto reference, tests/, doc/ with a getting started guide that covers the scene graph, ownership and lifetimes, geometry, materials, lights, the frame loop, loaders and the two backends, plus apps/ and the editor-core and editor-wheel directories. The MIT licence keeps the commercial question simple for anyone embedding it, and CITATION.cff exists for academic use.

Editorial conclusion

Adopt threepp if you want three.js shaped code in a native or Python program, need a headless renderer for CI, or are building synthetic data where you need ground truth labels, and accept that you are tracking a research project whose APIs change. Do not adopt it for a production game engine, and do not plan a macOS Vulkan workflow, because the README states the Vulkan backend needs ray tracing extensions that MoltenVK does not provide, so use GLRenderer there. Before you start, pin a tag or commit as the project status section instructs, check that PhysX and the Vulkan SDK are available in your build environment, and read doc/getting_started.md for the ownership and lifetime rules the library expects.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. markaren/threepp on GitHub
4. README
5. Releases