# SAPIEN: a physics simulator built for part-level manipulation research > SAPIEN is a physics-rich simulated environment for articulated objects, distributed on PyPI, with a renderer that runs headless on a GPU server and a version 3 that replaced actors with entities. The changelog is the best documentation of the project, because it is where the physics, the renderer and the camera system all moved. **haosulab/SAPIEN** — SAPIEN Embodied AI Platform - Repository: https://github.com/haosulab/SAPIEN - Website: https://sapien.ucsd.edu/ - Stars: 859 · Forks: 81 - Language: C++ - License: NOASSERTION - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/haosulab-sapien ## Part-level articulation is the point, not a feature SAPIEN is described as a realistic and physics-rich simulated environment that hosts a large-scale set of articulated objects, and it enables robotic vision and interaction tasks that require detailed part-level understanding. That phrase, part-level understanding, is the differentiator. Plenty of simulators offer rigid bodies and articulated joints. What a manipulation researcher needs is a scene where a drawer has a handle, a cabinet has a door, and a bottle has a cap, with each of those pieces modelled and movable. That is the requirement the project names, and it explains the rest of the design. The data side follows from it. The README describes SAPIEN as a collaborative effort between researchers at UCSD, Stanford and SFU, and the dataset as a continuation of ShapeNet and PartNet, the two well known sources of annotated 3D shape and part data. So the environment and the asset library were built together, and the assets are the point as much as the physics are. There is a practical note about where to get those assets. The README says the primary way to access PartNet-Mobility is now via Hugging Face, with the dataset hosted under the sapien-sim organisation. That is a change worth knowing about if you have an old pipeline that fetches from the original distribution, because the old path is no longer the primary one. The project homepage is sapien.ucsd.edu, the documentation is at sapien-sim.github.io/docs, and the licence is recorded in the LICENSE file at the root, with the repository listing noting a custom licence that GitHub cannot classify. ## One pip command, and a GPU is not optional The installation is genuinely short, and the hardware requirement is the part that trips people up. SAPIEN is distributed via PyPI, and the README says the installation is just: ```bash pip install sapien ``` It requires Linux with an NVIDIA, AMD or Intel GPU to run, and the README gives a verification step, which is a module invocation rather than a separate script: ```bash python -m sapien.exapmle.hello_world ``` The documentation then points at the tutorial, and the note that the example module name contains a typo is worth having, because that is the string you will type when you are checking a fresh install and it is easy to conclude the package is broken when the command fails. So the supported platforms, as stated, are Linux with a GPU from any of the three major vendors. The project is a simulator, and a simulator needs a graphics device to render with, so the requirement is structural rather than a preference. The Windows and macOS situation is not described in the README, which is worth checking in the documentation before you plan a pipeline around one of them. The library itself is C++, with the Python package as the interface. The repository shows a full C++ project with src/ and include/ directories, a python/ directory for the bindings, vendored third-party code in 3rd_party/, a vulkan_library/ and vulkan_shader/ directory, a pinocchio/ directory, and toolchains/ for the build. The Python distribution is produced by a setup.py that builds native extensions, so a pip install is a compilation, not a wheel download. ## Headless rendering needs two libraries, a display needs five Running SAPIEN on a GPU server is a documented scenario, and there are two ways to do it with different system dependencies. The first is offscreen rendering on a server with no display at all. The README says the only system dependencies required are libegl1 and libxext6. If you are using an NVIDIA Docker environment, you enable graphics, utility and compute by setting an environment variable in the Dockerfile: ```dockerfile ENV NVIDIA_DRIVER_CAPABILITIES graphics,utility,compute ``` The second is a virtual desktop on the same server, which is what you want when a human needs to watch the simulation or use the viewer. That additionally needs xvfb, x11vnc and any window manager such as fluxbox or xfce, and the NVIDIA capabilities have to include display as well: ```dockerfile ENV NVIDIA_DRIVER_CAPABILITIES graphics,utility,compute,display ``` With a window manager in place you start a VNC server, and the README gives the exact invocation, with the environment variables that point it at the right display and geometry: ```shell x11vnc -create -env FD_PROG=/usr/bin/fluxbox -env X11VNC_FINDDISPLAY_ALWAYS_FAILS=1 -env X11VNC_CREATE_GEOM=${99:-1920x1080x16} -gone 'pkill Xvfb' -nopw ``` The command ends with -nopw, which disables the password, and the README immediately follows it with the warning that you should use a strong password and or only allow local access. That is the correct warning to repeat, because a VNC server on port 5900 with no password on a shared GPU box is an open door, and the README is explicit that you connect to the server at that port. The two paths are worth distinguishing on cost. Offscreen rendering is two system libraries and one environment variable, and it is enough for training runs and headless evaluation. The virtual desktop exists for the cases where the GUI has to be seen, and it costs a window manager, a VNC server and a port. ## Version 3 replaced actors with entities The 3.0 changelog entry is the most consequential thing in the repository, and it is titled as a major API and infrastructure overhaul. The core of the change is a refactor of SAPIEN to use an entity-component system. An actor from SAPIEN 2 becomes an entity in SAPIEN 3, and the functionalities that used to be an actor now become components attached to an entity. The documentation was updated for the new usage, and the README notes that the actor builder and articulation builder APIs are mostly unchanged, with one exception: the color and material parameters for visual shapes are unified into a single material parameter. The second half of 3.0 is physics. SAPIEN 3 supports PhysX 5 GPU simulation, and the changelog points at ManiSkill as the place to see how that is used to build robot learning environments, which is a useful pointer if you are building on top of the simulator rather than using it directly. The migration story for existing code is that builder APIs mostly survive, so a scene that assembles actors and articulation can be ported by renaming the type and reassigning behaviours to components, but anything that reached into an actor's internals will need rework. The README points at the updated documentation for usage. The release history in the repository shows how this is being shipped: a nightly release on 2026-10-09, version 3.0.3 on 2026-03-10, and the 3.0.0 beta from 2024-05-22. The last push matches the nightly release date, so the main branch is where the work happens. ## The renderer history explains today's API names Reading the changelog from the bottom up explains the API surface better than any single version entry can. Version 1.0 replaced the old Vulkan based renderer completely, with the new one documented in the renderer core module, and it reimplemented the Vulkan viewer in Python rather than wrapping an existing one. It also exposed the GUI functionality and the PhysX shape wrapper to Python, so collision shapes could be retrieved from an actor, collision groups could be set on a shape, and shapes became available in contacts. The 1.0 entry also lists the API changes that still bite, such as render material creation moving to a factory method on the renderer and the actor builder's shape functions being renamed to collision functions. Version 1.1 introduced the entity as the base class of actors, added light classes that inherit from entity, and supported nonconvex static and kinematic collision shapes, with a warning for small mass and inertia values. The 2.1 entry refactored the camera system so that cameras no longer require mounts, and then the 2.2 entry renamed the renderer to SapienRenderer while keeping the old VulkanRenderer name as an alias, added ray tracing support, deprecated the KuafuRenderer in favour of the ray tracing shader, and added a GPU-accelerated stereo depth sensor simulation and a render server. The Python version floor moved with them: 3.10 in 2.1, 3.11 in 2.2. Two patterns are worth noticing. The camera refactor and the render shape refactor both changed where you set scale and pose, and both shipped as breaking changes, which tells you how much the render layer is still moving. And the deprecation pattern, keeping an alias for the old renderer name, is the polite version of the actor to entity rename. ## The Python package is built from C++ sources, and the flags say so The setup.py at the root of the repository is the clearest statement of what a pip install actually does, and it is worth reading before you try to build it yourself. It imports setuptools and defines a custom Extension, plus custom build, build_ext, bdist_wheel and bdist commands, so the wheel is assembled by the project rather than by setuptools defaults. The argument parser is where the interesting options live. There is a debug flag and a profile flag, and two that determine how much gets built: a sapien-only flag that builds the library without the Python binding, and a pybind-only flag that builds the binding assuming the library is already built. That pair is the answer to the most common build question. Building the simulator and the binding separately is supported, and it is how you would work when iterating on the C++ side without paying for a full rebuild of the binding every time. There is a build directory option, defaulting to sapien_build, and a get-version flag that prints the version string and exits, which is the convention for a build system that needs to stamp a version into the artefacts. The version itself comes from git. The file contains a function that shells out to git rev-parse to get the full commit hash, the short hash and the branch name, and it handles the case where those calls fail by falling back to an empty string, so a source tree without git history still builds. Around it sits the rest of the project: a CMakeLists.txt and a cmake/ directory for the native build, docker/ for container recipes, docs/, scripts/, toolchains/, a serialization.md document, and separate test/ and unittest/ directories. The release practice is the last thing to note. Alongside the versioned releases there is a nightly release, and the repository listing shows the nightly tag dated the same day as the last push, so the main branch is continuously built and published. For a research dependency, that is a reasonable source of the newest fixes, and a reason to pin when you need reproducibility. ## Conclusion Adopt SAPIEN if your research needs articulated objects, part-level geometry and a renderer you can drive on a GPU server, since that combination is what the project is built around and what the dataset lineage supports. Do not adopt it expecting a stable API, because version 3 was a major overhaul of the core and the project publishes nightly builds alongside 3.0.3, so pin a version and read the changelog for your generation. Before you deploy on a server, install libegl1 and libxext6 for offscreen rendering or the virtual desktop stack for a display, and set the NVIDIA driver capabilities in your container, since without them the renderer has no graphics path at all. ## FAQ ### What is SAPIEN used for? It is a realistic and physics-rich simulated environment hosting a large-scale set of articulated objects, for robotic vision and interaction tasks that require detailed part-level understanding. The dataset is a continuation of ShapeNet and PartNet, and the project is a collaborative effort between researchers at UCSD, Stanford and SFU. ### How do I install SAPIEN and check that it works? It is distributed via PyPI and the README says the installation is just pip install sapien. It requires Linux with an NVIDIA, AMD or Intel GPU to run, and the verification step given in the README is running the hello world module with python. ### Can SAPIEN render on a GPU server without a display? Yes, and the README says the only system dependencies required are libegl1 and libxext6. In an NVIDIA Docker environment you also set NVIDIA_DRIVER_CAPABILITIES to graphics,utility,compute. For a virtual desktop instead, add xvfb, x11vnc and a window manager, and include display in those capabilities. ### What changed in SAPIEN 3.0? It was a major API and infrastructure overhaul. The core was refactored to an entity-component system, so an actor from SAPIEN 2 becomes an entity and its functionalities become components attached to it, while the actor and articulation builder APIs are mostly unchanged apart from unifying color and material into a single parameter. Version 3 also adds support for PhysX 5 GPU simulation. ### How is SAPIEN released? The repository lists a nightly release dated 2026-10-09, version 3.0.3 from 2026-03-10 and the 3.0.0 beta from 2024-05-22, and the last push matches the nightly tag. The Python package is built from the C++ sources with setup.py, which has flags to build only the library or only the Python binding. ## Sources - [haosulab/SAPIEN on GitHub](https://github.com/haosulab/SAPIEN) - [Issues](https://github.com/haosulab/SAPIEN/issues) - [Project website](https://sapien.ucsd.edu/) - [README](https://github.com/haosulab/SAPIEN/blob/master/README.md) - [Releases](https://github.com/haosulab/SAPIEN/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/haosulab-sapien