Hydra: Building 3D Scene Graphs Incrementally, in Real Time
A system for building 3D Scene Graphs from sensor data in real-time
At a glance
- What is it?
- Hydra is research code from MIT-SPARK that constructs 3D scene graphs from sensor data while the robot is moving, on ROS 2 Jazzy and Ubuntu 24.04. It comes from two papers, one on real-time spatial perception and one on hierarchical representations, and it has Python bindings, an evaluation harness and a set of documented caveats about support.
- Who is it for?
- Hydra fits a robotics research group working on spatial perception or scene graph construction, especially one already running ROS 2 Jazzy on Ubuntu 24.04 and comfortable with the Hydra, Kimera and GTSAM stack, and it does not fit a team that needs a supported product, since the README describes it as research code maintained by busy graduate students with no bandwidth to adapt it to new applications, and it does not fit a non-Linux setup at all.
- Can I use it commercially?
- Yes. BSD-2-Clause is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly C++, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on October 23, 2026, and from our analysis. They are not legal advice.
Editorial analysis
Scene Graphs That Grow While the Robot Moves
Hydra contains code to incrementally build 3D scene graphs in real time, and that phrase carries the whole design.
A scene graph here is a hierarchical representation of a space, with nodes for objects and rooms and edges for spatial relationships. Building one has usually meant running a mapping and labelling pipeline over recorded data. Incremental means the graph is updated as the robot moves, so the structure exists while the robot is still exploring, and the graph is both the output and the working memory of the perception system.
The two papers the repository is based on describe the approach from different angles. Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization, by Nathan Hughes, Yun Chang and Luca Carlone, appeared at Robotics: Science and Systems. Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systems, with Siyi Hu, Rajat Talak, Rumaisa Abdulhai and Jared Strader added to the author list, appeared in The International Journal of Robotics Research.
The repository asks that you cite one or both if the code is relevant to your work, and provides the bibtex entries for them, which is the normal courtesy for a research release and also a hint that the intended use is academic work built on the system.
The ROS 1 Version Is Archived and ROS 2 Is the Default
The most consequential line in the README is the update from 07/09/25, which says the ROS 1 version of Hydra has been archived and the project has switched to the ROS 2 version by default. The pinned ROS 1 code is still available on a branch named archive/ros_noetic, for anyone who needs it, and the project is unlikely to support issues that come up with the archived ROS 1 version.
That is a blunt statement, and it is the kind of announcement that saves a week of debugging when your build breaks after a pull. A project that is going to change its ROS generation in place needs the old code preserved somewhere findable, and the branch naming makes that findable.
The current target platform is stated plainly: Hydra has been tested on Ubuntu 24.04 and ROS 2 Jazzy. The installation instructions are not in this repository, they are in the Hydra-ROS repository, and the README links to the getting started guide there. The ROS-related code was itself moved out into that separate repository in an earlier release, with the note that the code and the installation process still rely on the ROS ecosystem.
So the shape of the project is a core library in this repository, with the ROS integration, the installation and the launch files living next door.
Semantic Labels, Colors and the Projects That Build on It
The January 2025 release changed something that sounds small and is not. Hydra no longer assigns colors to nodes when building a scene graph, and that is done by the visualizer instead, which the release notes describe as a clear separation between colors and semantic labels.
The reasoning is about what a scene graph node means. A semantic label is a fact about the world, and a colour is a rendering decision. When the library also picks the colour, every consumer that wants a different visualisation has to fight it, and every semantic change comes with a colour change. Moving the colour out means the downstream projects can render the same graph differently.
The release also added open-set semantic capabilities, described as used by downstream projects, and the two named are Khronos and Clio, both from the same MIT-SPARK group. That is the clearest evidence about the level the project operates at, since the downstream users are research systems building on top of the graph.
The same release added instructions for using a real semantic segmentation model via the semantic_inference repository, which closes a gap that matters in practice. A scene graph with no semantics is a geometric skeleton, and the segmentation model is what puts objects into it.
Python Bindings, the Habitat Interface and the Evaluation Harness
The June 2024 release added Python bindings around Hydra and an interface for running with the habitat simulator, and both are still in the tree.
The bindings are a separate package in the same repository, hydra_python, built with scikit-build-core and pybind11, so the C++ library is exposed through a native extension rather than through a reimplementation. The build is configured with cmake.source-dir pointing at the python directory, and the console entry point is a hydra command. The package version is 0.0.1, requires Python 3.8 or newer, and is classified as Development Status 4, Beta.
A version number of 0.0.1 on the bindings is worth noting. It tells you the Python surface is young, so code written against it should expect churn, and that the authors themselves are labelling it beta rather than treating it as a stable interface.
The Python side also carries the evaluation harness, with the eval/ directory and its own README, and the habitat interface means you can run the system against simulated environments rather than only against recorded sensor data. For a research project, that is what makes the evaluation reproducible.
The dependencies for the bindings are the usual research stack: numpy, scipy, networkx for the graph work, pyqtgraph for visualisation, imageio for output, click for the command line, pyyaml for configuration, distinctipy for colour assignment, and the dataset interfaces package from the same group.
The Dependency Simplification of June 2024
That release also changed the dependency set, and the release notes call it out as a note rather than a feature, which tells you how much friction it caused.
The changes were a move towards the ROS packaged version of GTSAM, dropping Kimera-Semantics, and replacing voxblox with Spatial-Hash. The README asks readers to double-check the rosinstall files to make sure they have all the new dependencies, and to feel free to remove unused old dependencies.
Each of those is a change of a load bearing piece. GTSAM is a dependency of the spatial pipeline, so moving to a ROS packaged version changes where it comes from and how it is versioned. Kimera-Semantics was dropped from the dependency set, which removes one more component from the build. Spatial-Hash replaced voxblox as the spatial data structure, and a hash based structure is a different tradeoff in memory and lookup behaviour from a voxel grid.
The same January 2025 release also updated the rosinstall file to point at the main branch of Kimera-PGMO, with a note to make sure the right branch is checked out, which is the same kind of dependency churn handled through a branch pointer rather than a version.
Research Code With Stated Caveats
The README is unusually direct about the support model, and you should read it before planning around the project.
It asks you to understand that this is research code maintained by busy graduate students, which comes with some caveats. There are three of them. The project does its best to maintain and keep the code up to date, but things may break or change occasionally. The project does not have bandwidth to help adapt the code to new applications. And the documentation, the codebase and the installation instructions are geared towards practitioners familiar with ROS and 3D scene graph research.
The third one is the real constraint. The documentation assumes you already know the field, so there is no tutorial for the scene graph concepts, and the debugging page in the doc directory exists because the failure modes are the kind that need domain knowledge to interpret.
Platform support is even narrower. There is a warning that other platforms are not supported, and that issues requesting support on other platforms, Ubuntu 18.04 and Windows named as examples, will be summarily closed.
The funding and the disclaimer are also in the README, which is normal for a sponsored academic release. The work was partially funded by the AIA CRA, the ARL DCIST CRA and the ONR RAIDER programme, and the disclaimer reproduces the standard United States Air Force Research Laboratory text about the views being the authors' own.
Editorial conclusion
Hydra fits a robotics research group working on spatial perception or scene graph construction, especially one already running ROS 2 Jazzy on Ubuntu 24.04 and comfortable with the Hydra, Kimera and GTSAM stack, and it does not fit a team that needs a supported product, since the README describes it as research code maintained by busy graduate students with no bandwidth to adapt it to new applications, and it does not fit a non-Linux setup at all. Before you plan around it, note that the ROS 1 version is archived on its own branch and the ROS 2 version is the default, that the Python bindings carry a version of their own at 0.0.1 and are marked Beta, and that the installation instructions live in the separate Hydra-ROS repository rather than in this one. The project is BSD licensed, the last push was on 2026-10-14, and the newest tagged release is v2.0.0a1 from 2023-06-26.
Frequently asked questions
What is Hydra and what does it build?
It is MIT-SPARK research code that incrementally builds 3D scene graphs in real time from sensor data, based on the papers on real-time spatial perception for 3D scene graph construction and on hierarchical representations for robotics.
Which ROS version does Hydra support?
ROS 2 is the default. The ROS 1 version was archived on 07/09/25, with the pinned code kept on the archive/ros_noetic branch, and the project is unlikely to support issues with the archived version. Hydra has been tested on Ubuntu 24.04 and ROS 2 Jazzy.
Where do I find the Hydra installation instructions?
In the separate Hydra-ROS repository, whose getting started guide the README links to. The ROS related code was moved into that repository in an earlier release, and the installation process still relies on the ROS ecosystem.
Does Hydra still assign colors to scene graph nodes?
No. Since the January 2025 release there is a clear separation between colors and semantic labels, and Hydra no longer assigns colors to nodes when building a scene graph, which is done by the visualizer instead. That release also added open-set semantic capabilities used by downstream projects.
What are the limitations of using Hydra?
The README states it is research code maintained by busy graduate students: things may break or change, the project has no bandwidth to adapt the code to new applications, the documentation assumes familiarity with ROS and 3D scene graph research, and other platforms are not supported, with issues requesting them closed.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/mit-spark-hydra)Was this useful?