Nuitrack turns a depth sensor into a tracked skeleton on any platform

One tracking library across a long list of depth sensors

Nuitrack is described as the 3D body and skeletal tracking library for 3D and depth sensors, developed by 3DiVi Inc. The point of the library is that the sensor is decoupled from the tracking, and the documentation is explicit about the breadth of the hardware list. Supported depth sensors include Orbbec Astra2, Persee2, Femto Bolt, Femto Mega, Kinect v1 and v2, Kinect Azure, Intel RealSense, Asus Xtion, LIPS and Structure Sensor.

The supported hardware platforms are x64, x86, ARMv7 and ARMv8, and the supported operating systems are Windows, Linux, Android and iOS. That combination is the reason the tool is positioned the way it is. The project describes its mission as providing a strong skeletal tracking baseline for the next generation of immersive and analytical applications, inspired by the first Microsoft Kinect, and summarises it as a Kinect for everything.

The application range follows from the hardware range. The documentation names real-time gesture recognition on a Raspberry Pi 4 at one end and large-scale multisensor analytical systems at the other, with a beta programme for multi-sensor fusion sitting in between.

For a developer, the practical question is whether the sensor you own is on that list. A tracking library is only as useful as the device coverage it has, and this one covers the depth cameras that the original Kinect ecosystem made common, plus the newer RealSense and Orbbec lines.

Two tracking engines, and the trade-off between them

Nuitrack has two skeletal tracking engines, and the documentation contrasts them directly. The classical engine is described as fast, stable and lightweight, and as highly optimized for embedded hardware and limited CPU usage. The AI engine is a deep-learning based engine that provides greater coverage for complex poses.

That is a genuine engineering trade-off rather than a marketing claim about one being newer. The classical engine is the one you would choose for a device with a small CPU budget, a kiosk running continuously, or a system where predictable latency matters more than the ability to handle an unusual pose. The AI engine is the one you would choose when users will be in awkward positions and coverage failures show up as a tracked person disappearing.

The output of that tracking work is broader than a skeleton. The feature table lists a depth map and 3D point cloud, scene analysis with key elements such as the floor plane, persons tracking with pixel-perfect segmentation for each of them, and a hand tracker with gesture recognition presented as high-level APIs for UI control. Basic facial analysis such as age and emotion detection is also listed, and it is described as basic rather than as a face recognition stack.

The framing the documentation uses is a human-centric spatial understanding tool for applications that engage with a user in a natural and intelligent way, which is a fair description of a library that returns both a skeleton and the scene around it.

Trying it out is a runtime download, not a library install

The evaluation path is deliberately short and it is worth being precise about, because it is not the normal software library experience. The documentation gives three steps. Download the Nuitrack Runtime package for your platform of choice from the GitHub releases. Install it, following the installation instructions in the doc directory if there are issues. Then plug in the sensor and launch the Nuitrack executable from the start menu.

So the entry point is a runtime application, not a header file. That runtime is what talks to the sensor and what your application then talks to. The repository also ships Dockerfiles and a Docker image, and the development bindings table lists C++, C#, Python, Unity, Unreal Engine 5 Blueprints, TouchDesigner and Docker as the supported integration paths, with the Python binding marked as a beta under a separate directory in the repository.

The repository layout backs this up. There are platform directories for Platforms and iOS-beta, an UnrealEngine directory, a Unity3D directory, a TouchDesigner directory, a Dockerfiles directory, an Examples directory and a doc directory.

The example list is a good map of what the SDK can do, since it names one sample per capability: a console sample, a device API sample, a C# device API sample, a C# sample, a GL sample, an NI GL sample, a multisensor sample, an Android Studio GL sample, and a cmake directory for building them.

Embedding the runtime instead of installing it

There is a second deployment model, and it is the one that matters for a shipped product. The integration section states that Nuitrack provides the ability to embed it into your application, which means it does not need to be separately installed on the PC of your customers. A separate document covers this in doc/Nuitrack_Integration.md.

This distinction is the difference between a demo and a product. Asking end users to install a runtime package before your application works is a support burden and a conversion leak, and the embedded model removes it. The cost is that your application carries the tracking stack, which affects the size of your installer and the platforms you can ship on.

The documentation also points at a Licensing Dashboard for licenses and subscriptions management, which tells you the commercial model: this is not a free SDK with a community edition, it is a product with per-subscription licensing managed through a dashboard. Nuitrack.com is listed for general information and license purchasing.

The FAQ section of the documentation also points to a troubleshooting page, and the repository carries a failure cases recorder documented separately, which suggests the maintainers treat field failure as a first-class problem rather than an edge case.

Versioning moved to a year-based scheme in 2026

The release history shows a clear change in how the project numbers its versions. The newest tag is v2026.1, released on 2026-08-09, and before that come v0.38.5 from 2025-07-13 and v0.38.4 from 2025-06-08. So a long 0.3x sequence has been replaced by a calendar-based scheme, which is a signal that the project stopped treating the 0.x line as unstable and started versioning like a shipping product.

The last push was on 2026-08-09, the same day as the v2026.1 tag, so the branch and the release are aligned rather than drifting apart.

The what's new list explains what the 2025 work actually was, and it is a mix of engine and tooling changes. Version 0.38.2 added Unreal Engine 5 MetaHuman native support and came with a redesigned Nuitrack App. Version 0.37.24 made Unreal Engine 5 available directly from the official Fab marketplace, and also added the ability to write CSV files using the Nuitrack App with no knowledge of C++ required. The TouchDesigner release became official, and multisensor tracking is documented in a separate page, with a beta programme for the multi-sensor fusion work.

Coming soon is listed too: UE Niagara particle system support, a body measurements API, holistic skeletal tracking with multiple sensors, and a major update on tracking accuracy. That last one is the item to watch if accuracy is the reason you would consider this library at all.

What the licence situation means for your project

The repository is published under a licence the platform does not classify, and the documentation points you at a licensing dashboard and a purchase page rather than describing an open source licence. So the reasonable reading is that this is a commercial product with a public SDK repository, where the code is visible and the terms of use are set by a commercial agreement.

That is not a criticism, it is a different kind of project, and the trade is straightforward. You get a solved problem that is genuinely hard, one that works across a wide range of depth sensors, CPU architectures and operating systems, with a mature set of bindings including Unity and Unreal, and with two tracking engines to choose between. The price is a licence cost, a runtime component, and the fact that the roadmap is set by a vendor rather than by contributors.

Two practical checks follow. First, the sensor coverage, because a tracking library is only useful if it supports the device you have, and second, the deployment model, because the difference between a runtime your users install and a runtime you embed decides how you package your product. Both are answerable from the documentation before you write any code.

Editorial conclusion

Adopt Nuitrack if you are building a motion-controlled application on a depth sensor and want a solved skeleton, hand tracking and segmentation rather than a vision stack of your own, and if you can accept a commercial licence and a runtime that must be present on the machine. Do not adopt it if you need a free, self-contained library for a webcam only, since the value here is depth sensor support rather than a general pose estimator. Before you start, check three things: which sensor you own, since the supported list names Orbbec Astra2, Persee2, Femto Bolt and Mega, Kinect v1, v2 and Azure, Intel RealSense, Asus Xtion, LIPS and Structure Sensor, whether your target hardware is covered at the architecture level, since the list is x64, x86, ARMv7 and ARMv8, and whether embedding the runtime into your application, described as the integration path, is acceptable to your distribution model.

Frequently asked questions

Official sources

1. 3DiVi/nuitrack-sdk on GitHub
2. Issues
3. Project website
4. README
5. Releases