Khronos tracks what changed in a scene, not just where things are

A SLAM system that treats the scene as a function of time

Khronos is described as a unified approach that can reason about short-term dynamics and long-term changes when performing online metric-semantic simultaneous mapping and localization in dynamic environments. Most SLAM systems assume a largely static world and treat moving objects as noise to be filtered out, which is exactly the assumption that breaks in a warehouse, a lobby or a construction floor where furniture moves. Khronos changes the question. Instead of producing a single map, it maintains a spatio-temporal map whose instances represent the scene state at all times, and it reasons about two different kinds of change. The first is short-term dynamics, where humans and inanimate objects such as a cart are detected and compared against observed actions over the corresponding time interval. The second is long-term change, where the scene is re-examined when the robot revisits a place and closes a loop. The project comes from MIT-SPARK, the group behind SPARK, and the citation is a Robotics: Science and Systems paper from July 2024 in Delft, which won the Outstanding Systems Paper Award at RSS.

Short-term motion and long-term change are separate problems

The README's description of the demo is the clearest account of what the system actually does, so it is worth following. In the short-term case, the spatio-temporal map shows dynamics in magenta and compares them against observed human actions over the matching time interval, and both people and objects like a cart are detected. The long-term case is a different scene shown at three time instances. At time 0:20 the robot passes a table with a chair beside it. As the robot moves through the hallways, a chair is removed and a red cooler is placed on the table, and that pair of changes is detected when the robot revisits the area and closes the loop at time 1:52. Later the cooler is removed again, detected at time 3:35. The design consequence is significant. A conventional front end would have to discard the moved chair and the cooler as outliers or fold them into a static map, and it could not answer what changed or when. Khronos keeps a background mesh, generates object nodes for detected objects, and represents a scene graph, so a change becomes a first-class event with a timestamp rather than an error in the estimate.

Building the ROS 2 workspace from the install files

The setup path is fully scripted, which is a relief for a research codebase. Khronos has been tested on Ubuntu 24.04 with ROS 2 Jazzy, it should build on newer distributions, and it will not build on ROS 2 distributions older than Iron. First you create a workspace and install the tooling the import step needs:

sudo apt install python3-vcstool python3-tk
mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src

The system dependencies include the factor graph library and two common C++ packages:

sudo apt install ros-$ROS_DISTRO-gtsam libgoogle-glog-dev nlohmann-json3-dev

The source itself is fetched with vcs, using the rosinstall file in the repository:

git clone [email protected]:MIT-SPARK/Khronos.git khronos
vcs import . < khronos/install/ssh.rosinstall

That import uses the ssh.rosinstall file, so it needs GitHub ssh keys configured. The README notes that an https.rosinstall variant is available for people without ssh set up. The build then follows the standard colcon pattern:

cd ~/ros2_ws
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release

The repository layout follows the ROS 2 workspace convention, with the algorithm in khronos/, the ROS wrapper in khronos_ros/, message definitions in khronos_msgs/, evaluation tooling in khronos_eval/ and the install files in install/. A .pre-commit-config.yaml and a .clang-format file sit at the root, so formatting is enforced by tooling rather than by review comments.

Running from a launch file and saving the reconstruction

Execution is driven by a launch file that you edit to point at the right rosbag directory, followed by a standard launch command:

source ~/ros2_ws/install/setup.bash
ros2 launch khronos_ros uhumans2_khronos.launch.yaml

RViz launches automatically and shows the method building an incremental background mesh, detecting and segmenting static and dynamic objects, and generating object nodes. The right side panel of RViz toggles the visualisation options, including inputs, object and motion detection, the factor graph, the scene graph and backend visuals, and most of those are disabled by default, so a first run looks emptier than the paper figures unless you switch them on. At the end of rosbag playback the reconstruction is complete, and the README is explicit that you must then call a service to guarantee the output is saved before terminating the process:

ros2 service call /khronos_node/experiment/finish_mapping_and_save std_srvs/srv/Empty

Skip that call and you can lose the run, which is the single most likely way to waste an afternoon with this repository. One detail catches people out with the data itself. ROS 2 bags are directories, not single files, so the downloaded datasets have to be extracted into their respective directories rather than loaded as one archive.

Semantic inference is optional and partly gated by TensorRT

By default Khronos uses ground-truth semantic labels for simulated datasets and pre-recorded segmentation topics for real datasets. That default is the fastest path to running the system on the paper datasets, and it also means the semantic side of the pipeline is not being tested on your own data unless you change it. To run with semantic_inference for closed-set or open-set online segmentation, the README points to the separate semantic_inference package, which is included in the rosinstall files and built according to its own instructions. There is a hardware-related constraint here. Without TensorRT you can only use open-set segmentation. The README also says to source the virtual environment for semantic_inference before running Khronos with open-set segmentation, and that the first run downloads the pre-trained model weights, which may take a while. Between the optional dependency, the virtual environment step and the weight download, the optional path is where a first-time build most often stalls, and it is worth budgeting time for it before you commit to an evaluation schedule.

The limitation: an unstable ROS 2 port, heavy dependencies and a need for data

The README carries an explicit warning that the ROS 2 version of Khronos is in active development, is unstable and may not be fully feature complete. That is the first fact to weigh, and it is stated by the authors rather than inferred. The second is the dependency weight. GTSAM, glog and nlohmann-json3 are system packages you must install, the factor graph library in particular being the heaviest piece of the stack, and the build assumes a source build with colcon rather than a binary distribution you can pull. The third is data. The simulated datasets used in the paper are hosted on Google Drive under the name tasse_cd, with the longer of the two, tasse_cd_office, containing the most changes and dynamic objects, and the real dataset is a mezzanine rosbag with pre-recorded semantic labels under khronos_real. Evaluation therefore needs the datasets before any comparison is possible, and a review of the result needs the evaluation tooling in khronos_eval/. A system this young, this dependency-heavy and this dependent on specific data files is a research instrument rather than a deployable component, and treating it as a drop-in navigation front end is the wrong expectation.

Time-aware mapping versus static-scene SLAM

The realistic alternative is a conventional metric or semantic SLAM front end that assumes the environment does not change, optionally with dynamic object filtering that discards anything detected as moving. The difference in approach is that the conventional pipeline produces one map for the whole run, and a moved chair is either an error to be rejected or a distortion to be smoothed away. Khronos produces a sequence of scene states, keeps detected objects as nodes, and treats a change as an event that the system reasons about, which is what makes the loop-closure detections in the demo possible at all. The cost of that difference is visible in the code and the dependencies. You need a factor graph that can hold temporal structure, a scene graph representation, and a change detection stage on top of a normal SLAM front end, plus an inference stack if you want semantics from raw sensor data rather than from ground truth labels. If your question is where the robot is and what the static environment looks like, a static-assumption pipeline is simpler and sufficient. If your question includes what moved while the robot was not looking, the static pipeline has no answer to give you, and that is the gap Khronos fills.

Editorial conclusion

Khronos is a good fit for research in dynamic-scene mapping where the question is what changed over time, not only where the robot is, and you are willing to build from source and supply your own datasets. It is a poor fit for production robotics today, because the README warns the ROS 2 version is unstable and may not be feature-complete. Before starting, confirm your distribution is Jazzy on Ubuntu 24.04 or newer, since the project will not build on ROS 2 distributions older than Iron, and decide early whether the ground-truth segmentation defaults are acceptable for your evaluation.

Frequently asked questions

Official sources

1. Issues
2. License: BSD-3-Clause
3. MIT-SPARK/Khronos on GitHub
4. README