fuse: a ROS framework for sensor fusion as a graph of constraints

Everything is a plugin around one optimiser

fuse is a ROS framework for performing sensor fusion using nonlinear least squares optimization techniques, and the way it is put together is more interesting than the list of features. It provides a plugin-based system for modeling sensor measurements, a similar plugin-based system for motion models, a plugin-based system for publishing optimized state values, an extensible state variable definition, a contract on how an optimizer will interact with those components, and some common implementations to get everyone started. The applications named are state estimation, localization, mapping and calibration. The design decision that everything else follows from is that the optimiser does not know what a wheel encoder is. It knows about constraints, variables and a contract, and the sensor models, motion models and publishers are plugins loaded into it. The primary language is C++, the default branch is devel, and the last push was on 2026-08-19.

The data flow, and why it is asynchronous in practice

The flow through the system is described as a loop, and each step names the component responsible. A sensor model receives raw sensor data, generates a constraint and sends it to the optimiser. The optimiser receives the new sensor constraint, and a request goes to each configured motion model to generate a constraint between the previous state and the new state involved in the sensor constraint. The motion model generates the required constraints connecting the new state to the previously generated motion model chain and sends them to the optimiser. The optimiser then adds the new sensor model and motion model constraints and variables to the graph and computes the optimal values for each state variable. The optimal state values are sent to each configured publisher, and to the sensor models and motion models as well, and the publishers publish any derived quantities on ROS topics. Then it repeats. The documentation is explicit that much of this flow happens asynchronously in practice.

Batching is why the optimiser can keep up

The note about asynchrony is followed by the reason it works. Sensors are expected to operate independently from each other, so each sensor sends constraints to the optimiser at its own frequency, and the optimiser caches the constraints and processes them in small batches on some schedule. That is the difference between a filter that runs as fast as its slowest input and one that keeps up with a laser at 10 Hz and encoders at 100 Hz. The second consequence is latency. The publishers may require considerable processing time, which introduces a delay between the completion of the optimization cycle and the publishing of data to the ROS topic. So a system built on fuse has an inherent lag between when a measurement arrived and when the optimised state is visible to the rest of the stack, and if your controller needs the freshest pose rather than the most consistent one, that trade-off is the thing to think about.

State variables are nodes, measurements are edges

The worked example is a typical indoor differential-drive robot with wheel encoders and a horizontal laser, and the first step is defining the state variables. At a minimum you want the robot pose at each timestamp, modelled as a 2D position and an orientation, and each 2D pose at a specific time gets a unique variable name. The example calls them X1, X2 and X3 for ease of notation, though in reality each variable gets a UUID, which are much harder to write down. Each 2D pose is instantiated as an example_robot::Pose2D derived from the fuse_core::Variable base class, and the framework ships with several basic variables already, such as 2D and 3D versions of position, orientation and velocity variables, but you can derive your own as needed. The graph pictures that follow make the mental model concrete: the large circles are state variables at a given time and the small squares are measurements, and the connectivity shows which variables are involved in which measurements.

The wheel encoder constraint, written out

The example then shows how a sensor becomes a constraint, which is the part worth reading in detail. The wheel encoders provide an incremental pose measurement, and given a starting pose X1 and a wheel encoder delta z, the predicted pose X2' comes from some measurement function f, written as X2' = f(X1, z). The error term for the constraint is the difference between the predicted pose and the actual pose, written as error = X2'^-1 * X2, where the exponent is the inverse of the pose X2'. A fuse_core::Constraint is derived that implements that error function, and the same is done for the laser data with a scan-to-scan matching producing an incremental pose constraint between consecutive scans. When the sensors are synchronised, in the simplest example, that is enough to construct the first fuse system, with the two sensor models configured as plugins to an optimiser implementation. The two are never exactly satisfiable together, so instead the error of all the constraints is minimised using nonlinear least squares optimization.

The first adaptation: unsynchronised sensors disconnect the graph

This is the problem you will hit first in practice, and the documentation treats it as a first-class case rather than an edge case. Typically the laser measurements and the wheel encoder measurements are not synchronised: the encoder measurements are sampled faster than the laser, and are sampled at different times using a different clock. If nothing different is done in this situation, the constraint graph becomes disconnected, and the diagrams in the documentation show exactly what that looks like. The fix is the motion model, which differs from a sensor model in that constraints can be generated between any two requested timestamps, and whose constraints are generated upon request rather than due to their own internal clock. The motion model connects the states introduced by the other sensor measurements, and a class is derived from the fuse_core::MotionModel base class implementing a differential drive kinematic constraint for the robot. The motion models are also configured as plugins, and the optimiser requests their constraints whenever new states are created by the sensor models.

Publishers are plugins too, and there can be more than one

The publisher side is where the optimised state becomes visible to the rest of the stack, and the first version of it is deliberately minimal. A fuse_core::Publisher class is derived and added to the optimiser, and derived publishers have access to the optimised values of all state variables. The specific publisher implementation determines what type of messages are published and at what frequency. In the example the goal is to visualise the current pose of the robot in RViz, so a publisher is created that finds the most recent pose, converts it into a geometry_msgs::PoseStamped message and publishes it to a topic. The second adaptation section then asks the obvious follow-up question: nothing about the framework limits you to a single publisher, and if you want to visualise the entire robot trajectory rather than just the most recent pose, you create another derived publisher class that publishes all of the robot path. The module layout in the repository reflects the plugin boundaries, with separate directories for fuse_core, fuse_constraints, fuse_variables, fuse_models, fuse_optimizers, fuse_publishers, fuse_msgs, fuse_graphs, fuse_loss, fuse_viz, fuse_doc and fuse_tutorials.

Editorial conclusion

fuse suits a robotics team that wants to compose a state estimator out of separate sensor and motion model plugins rather than write one monolithic filter, and that needs the constraint graph visible so unsynchronised sensors can be handled explicitly. It does not suit a project outside ROS, and it does not suit someone who wants a working filter on the first day, because the example builds one from scratch. Before adopting it, read the two adaptation sections in the documentation, because the disconnected graph problem appears as soon as your sensors run on different clocks, and check that the licence in the repository file is one your organisation accepts, since the licence field on the repository is a custom one that GitHub cannot classify.

Frequently asked questions

Official sources

1. Issues
2. locusrobotics/fuse on GitHub
3. README