isaac_ros_nvblox and turning depth into a Nav2 costmap on the GPU

The node turns a depth image into something Nav2 can plan against

Isaac ROS Nvblox contains ROS 2 packages for 3D reconstruction and cost maps for navigation. The `isaac_ros_nvblox` node processes depth and pose to reconstruct a 3D scene in real-time and outputs a 2D costmap for Nav2, where the costmap is used in planning during navigation as a vision based solution to avoid obstacles.

That framing is worth pausing on, because it is narrower than a mapping system. The output is a local costmap, which is the obstacle representation a local planner consults, not a global map of the whole building. The distinction matters if you are coming from a SLAM stack that produces a static map you then localise against. Here the map is derived continuously from live sensor data, so it is always current, and it is also local, so it covers what the sensors can see rather than what you have visited.

The inputs are a depth image, a color image and a pose. The color image is optional for the reconstruction itself and required for the people segmentation path. The pose comes from outside the package: in the reference graph the pose is computed using `visual_slam`, or some other pose estimation node. That is an important architectural fact. The package is not doing visual SLAM, it is consuming an already solved pose and fusing depth against it, which means the accuracy of the reconstruction is bounded by the pose estimate that another component supplies.

The package is designed to work with depth cameras and or 3D LiDAR, and it uses GPU acceleration to compute the reconstruction and the costmaps using nvblox, the underlying framework independent C++ library.

Three mapping modes, and the choice is about what moves in the scene

The package offers several modes of operation, and the choice between them is the main configuration decision in this repository.

In the default mode the environment is assumed to be static. That is the cheapest path, since nothing has to be separated out of the reconstruction, and it suits a warehouse with fixed racking, a corridor, or any space where the only moving things are the robot itself.

Two additional modes exist for scenes containing dynamic elements. People reconstruction is for mapping scenes containing people, and dynamic reconstruction is for scenes containing more general dynamic objects.

The people path is the more interesting one mechanically, and the readme walks through it. The color image corresponding to the depth image is processed with `unet`, using the PeopleSemSegNet DNN model to estimate a segmentation mask for persons in the color image. Nvblox then uses that mask to separate the reconstructed persons into a separate people-only part of the reconstruction. In other words, the segmentation is not a post-process on the output, it is an input to the reconstruction, and the reconstruction is split into a static part and a dynamic part because the two need different treatment.

The distinction between the two dynamic modes is that people are handled with a trained segmentation model and a specific mask, while general dynamic objects are handled by the reconstruction machinery itself. The performance table makes a further point about the second mode, since dynamics are marked as not supported on the Jetson Nano.

Technical details on all three types of mapping are linked from the readme to a page on the Isaac ROS documentation site.

The measured numbers show where the cost actually is

The readme carries a timing table for functions of nvblox core across platforms, and it is more informative than a single headline figure because the components cost different amounts.

The rows are TSDF integration, color integration, meshing, ESDF and dynamics, and the columns are a 5090 desktop, a 3090 desktop, an A3000 laptop, AGX Thor, AGX Orin and Orin Nano, on the Replica and Redwood datasets at a voxel size of 0.05.

On the Replica dataset, the TSDF integration takes 0.1 ms on the 5090 and 2.1 ms on the Orin Nano. Color integration is 0.3 ms and 3.6 ms respectively. Meshing is 0.3 ms on the 5090 and 13 ms on the Orin Nano, which makes meshing by far the most expensive row. ESDF, the distance field that a costmap is derived from, is 0.3 ms on the 5090 and 6.2 ms on the Orin Nano.

Two things follow. First, the cost of the reconstruction scales steeply as you move down the hardware list, and the spread is largest for the rows that do the most work, so the meshing step is what will hurt on a small Jetson. Second, on the Redwood dataset every figure is lower, roughly half the TSDF and color cost of Replica, and the dynamics row is marked N/A on the Jetson Nano with a footnote saying dynamics not supported on Jetson Nano.

None of this is a benchmark of the ROS 2 node itself. The table is explicitly for nvblox core functions, and the ROS wrapper adds message conversion and image transport on top.

The repository is a family of packages, not a single node

The top level directory listing shows that this repository is a set of ROS 2 packages that together implement the integration, and reading the names tells you what a user actually has to work with.

`isaac_ros_nvblox/` is the main node package. Around it sit `nvblox_msgs/` for the message definitions, `nvblox_ros/` and `nvblox_ros_common/` for the ROS specific pieces of the nvblox binding, `nvblox_message_adapters/` for converting between message types, and `nvblox_ros_python_utils/` for the Python side.

Then there are the integration packages, which are where the practical value sits. `nvblox_nav2/` is the cost map plugin that feeds the reconstruction into Nav2, and the readme describes the reference graph as providing the sliced cost map through a cost map plugin into Nav2. `nvblox_rviz_plugin/` is the visualisation side, and the readme explains that an optional colorized 3D reconstruction is delivered into `rviz` using the mesh visualization plugin, with nvblox streaming mesh updates to RViz so the reconstruction updates in real-time as it is built. `nvblox_foxglove/` is a Foxglove integration, and `nvblox_examples/` holds example graphs.

The test directories are named as well, `nvblox_test/` and `nvblox_test_data/`, with a `resources/` directory at the root. There is also a `SECURITY.md` and a `CONTRIBUTING.md`.

The language is C++, and the repository carries a `.clang-format` file and a `.style.yapf` file, which tells you the Python in the tree is formatted with yapf rather than black.

No build commands in the readme, because the documentation lives elsewhere

This repository does not come with a self contained quick start, and that is a deliberate structural choice rather than an omission.

The readme's Documentation section says plainly: please visit the Isaac ROS Documentation to learn how to use this repository, and it links to the page for this specific package. The homepage recorded for the project is the Isaac ROS Gems site, and the topics listed for the repository include ros2-humble, which is a strong hint about the ROS 2 distribution the package targets.

What the repository does give you is the conceptual layer. The readme explains the node graph, the three mapping modes, the inputs, the outputs, and the measured performance of the underlying library, and it points at the technical details page for the mapping modes. The build, the sensor configuration, the Nav2 integration and the launch files are the documentation site's job.

For an evaluator this means the assessment has to be split in two. The questions the repository answers well are whether the architecture is right for your problem, what the inputs and outputs are, which hardware it was measured on, and which operating modes exist. The questions it does not answer are how to build it, which launch file to use, how to configure your depth camera, and what the ROS 2 distribution requirements are beyond the humble topic tag.

If your team evaluates packages by reading them, this one will look thin on the quick start and rich on the design. If your team evaluates them by running them, you will spend your time in the Isaac ROS documentation, and the repository is the reference you consult once the graph is running and you want to know what the modes do.

Where the GPU dependency makes this the wrong tool

There are three situations where this is the wrong choice, and all three come back to the same fact.

The first is hardware. The package uses GPU acceleration to compute the reconstruction and the costmaps, and the performance table is a table of GPUs. A robot without a supported NVIDIA GPU, or with a Jetson model where the measured cost is high, is a robot where this approach may not fit. The meshing row is the sharpest illustration, since it is the most expensive step and the one with the widest spread across the hardware list, and meshing exists to feed the RViz visualisation, which means the expensive part is optional for navigation.

The second is a static environment. If nothing in the scene moves, the default mode already assumes that, and the question becomes whether you need a GPU reconstruction at all. A depth camera or LiDAR feeding a conventional occupancy grid would give a costmap without a GPU, without nvblox and without the pose dependency. The vision based approach earns its cost when the scene is dynamic or when the depth sensor is the only sensor you have.

The third is the pose dependency. The reconstruction is built against a pose that another node computes, and the reference graph uses `visual_slam` for that. If your team has no visual SLAM stack, the accuracy of the whole thing is bounded by whatever you plug in, and the package does not include that solution. In a system where the robot localises by other means, the reconstruction has to be slaved to that estimate instead, which is a different integration problem.

And the scope limit is worth restating because it is easy to misread from the name. This is a local costmap provider, not a mapping system. A robot that needs a global map for navigation over a large building wants something else.

Against the standalone nvblox library, the difference is the ROS wiring

The alternative is to use nvblox directly, or to build your own costmap provider on top of it, and the difference in approach is entirely in the integration layer.

nvblox itself is described as a framework independent C++ library. That phrase is doing the work: it does not know about ROS topics, and it does not know what Nav2 expects. What Isaac ROS Nvblox adds is the part that makes the library usable inside a ROS 2 system.

Concretely, that is the message definitions in `nvblox_msgs/`, the ROS specific binding pieces in `nvblox_ros/` and `nvblox_ros_common/`, the message conversion layer in `nvblox_message_adapters/`, the Nav2 cost map plugin in `nvblox_nav2/`, and the RViz mesh visualisation plugin in `nvblox_rviz_plugin/`. None of that is in the framework independent library, and all of it is the reason a robotics team would use this repository rather than the library.

The second alternative is the classic one: build a costmap from the depth or LiDAR data with the standard navigation stack and the depthimage to laser scan approach, and skip reconstruction entirely. The difference is what the map can represent. A projected costmap is a flat obstacle view derived from the current sensor frame. A 3D reconstruction sliced into a costmap is a view of a scene that has been accumulated and, in the people mode, separated into static and dynamic content. The second is more work on the GPU and more information in the result.

So the decision is between doing the integration yourself with a library that knows nothing about ROS, and accepting the dependency on a specific set of packages built for a specific ROS 2 distribution and a specific set of NVIDIA hardware. For a team already inside Isaac ROS, the first option is a lot of work for a result the second already produces.

Editorial conclusion

isaac_ros_nvblox fits a robotics team that already runs on NVIDIA hardware, already has depth or 3D LiDAR on the robot, and needs a live obstacle map rather than a periodically refreshed one. It is the wrong choice for a CPU only robot, for a static environment where a plain occupancy grid from the LiDAR would do, and for a team that needs a full map rather than a local costmap. Before you plan around it, verify three things: that the platform you have is in the measured set, because the readme's own timing table covers specific desktop GPUs and Jetson boards and marks dynamics as unsupported on the Nano, that your sensors produce the depth and pose inputs the node expects, since pose estimation sits outside this package, and that the ROS 2 distribution you target matches the one the package is built for. The code is Apache-2.0, the current release is v5.0-0 from 2026-09-22, and the last push was on the same day.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. NVIDIA-ISAAC-ROS/isaac_ros_nvblox on GitHub
3. Project website
4. README
5. Releases