NVTX: annotating your code so a profiler can show what it meant

What NVTX is and why the calls do nothing

NVTX is a cross-platform API for annotating source code to provide contextual information to developer tools. The official description calls it a C-based Application Programming Interface for annotating events, code ranges and resources in your applications, and the API is written in C with wrappers provided for C++ and Python.

The design decision that defines the library is this: by default, NVTX API calls do nothing. When you launch a program from a developer tool, the NVTX calls in that program are redirected to functions in the tool, and developer tools are free to implement NVTX API calls however they wish.

That is the whole reason the library can live in production code. An annotation that costs nothing when nobody is looking is a different thing from an annotation that opens a connection, spawns a thread or writes to a file. Here, the code path is a no-op, and the moment a profiler attaches, the same call sites start doing something.

The tool side is where the value is, and the documentation gives examples of what a tool might do with the calls. It might print a message to the console, record a trace of when the calls occur and display them on a timeline, build a statistical profile of the calls or of the time spent in ranges between them, enable or disable tool features in ranges bounded by calls matching some criteria, or forward the data to other logging APIs and event systems.

The audience is performance engineers and the developers who work with them. A team that wants its Nsight Systems timeline to show what a kernel launch was rather than showing a stream of unnamed events is the group this library was written for.

The loop iteration example, and what a range buys you

The documentation's example is small enough to read in full and it shows the two things you need to understand: how to mark a scope, and how nested scopes look on a timeline.

#include <nvtx3/nvtx3.hpp>

void some_function()
{
    NVTX3_FUNC_RANGE();  // Range around the whole function

    for (int i = 0; i < 6; ++i) {
        nvtx3::scoped_range loop{"loop range"};  // Range for iteration

        // Make each iteration last for one second
        std::this_thread::sleep_for(std::chrono::seconds{1});
    }
}

NVTX3_FUNC_RANGE() marks the whole function body and ends automatically when the function returns, so there is no matching end call to forget. The scoped_range object marks the loop body, and its lifetime is the block it is declared in, which means the iteration range is closed when the loop body ends.

The nesting is the point. The function range contains the loop ranges, and the documentation notes that the loop iteration ranges are nested within the function range. When the same program is launched from NVIDIA Nsight Systems, the NVTX row shows the function's name in the top level range and the loop range message in the nested ranges, with the loop iterations each lasting for the expected one second.

Launched normally, the documentation says the program waits for 6 seconds and does nothing else. Same binary, two behaviours, depending on whether a tool is attached.

Markers, ranges and the two ways to scope a range

The annotation kinds are three, and the distinction between the two range types is worth being precise about.

Markers annotate a specific point in a program's execution with a message, and optional extra fields may be provided, a category, a color and a payload value.

Ranges annotate a range between two points in a program's execution, like a related pair of markers, and there are two types. Push and pop ranges can be nested to form a stack, and the pop call is automatically associated with a prior push call on the same thread. Start and end ranges may overlap with other ranges arbitrarily, the start call returns a handle which must be passed to the end call, and these ranges can start and end on different threads.

So the choice is between a stack discipline that is convenient and thread bound, and a handle based discipline that is explicit and can cross threads. A profiler that renders a flame graph of a parallel application needs the second kind, because the interesting structure in a parallel program is not a stack.

The C++ and Python interfaces provide objects and decorators for automatically managing the lifetimes of ranges, so in Python the equivalent of the scoped_range above is a decorator, and the lifetime handling is the same idea expressed for that language.

Naming and tracking resources, including mutexes

The third annotation kind is resource naming and tracking, and it is the one that turns a numeric id into something a human can read on a timeline.

Resource naming associates a displayable name string with an object. The example given is naming CPU threads, which lets a tool that displays thread activity on a timeline have more meaningful labels for its rows than a numeric thread ID.

Resource tracking extends the idea of naming to include object lifetime tracking, as well as important usage of the object. The worked example is a mutex provided by a platform API, such as a pthread_mutex or a CriticalSection, which can be tracked by a tool that intercepts the platform lock and release calls, so naming those mutex objects with NVTX would be sufficient to see the names of mutexes being taken and released on a timeline.

There is an honest limitation in that same paragraph, and it is the kind that decides whether this feature is usable in your codebase. Manually implemented spin locks may not have an interceptable API, so tools cannot automatically detect when they are used. The guidance is to use NVTX to annotate these types of mutex where they are taken and released, so that tools can track them like the standard platform API mutexes. In other words, for spin locks the annotation becomes mandatory manual work rather than something the tool discovers.

That is a real cost to plan for. A codebase with a lot of hand written spin locks, or a lock hierarchy you want to see on a timeline, needs someone to go through and annotate each acquisition site, and an annotation that is missing is a gap you will not notice in the profile.

Using NVTX from C and C++ without linking anything

For C and C++, NVTX is a header-only library with no dependencies. You include the headers you want to use and call the NVTX functions, and NVTX initializes automatically during the first call to any NVTX function.

It is not necessary to link against a binary library or add any link-time parameters. There is one platform caveat: on older POSIX platforms with glibc versions prior to 2.34, adding the -ldl option to the linker command is required.

The two include styles differ between the languages:

#include <nvtx3/nvToolsExt.h>

void example()
{
    nvtxMark("Hello world!");
}

#include <nvtx3/nvtx3.hpp>

void example()
{
    nvtx3::mark("Hello world!");
}

The include path uses the nvtx3 directory as a prefix, and the documentation is explicit that this is how you ensure you are including NVTX v3 rather than an older version, since older versions of NVTX did require linking against a dynamic library. NVTX v3 provides the same API and removes the need to link with any library.

Two more practical points. On Windows, be aware that NVTX includes windows.h, and without intervention this will define tokens in the global namespace such as min, max and small, so take care to define any desired macros such as WIN32_LEAN_AND_MEAN and NOMINMAX before including any NVTX headers. And since the C and C++ APIs are header-only, dependency-free and do not require explicit initialization, they are suitable for annotating other header-only libraries, which is the case where the zero cost of an unused annotation matters most.

The CMake targets and how to wire them into a build

For projects that use CMake, the CMake scripts included with NVTX provide targets nvtx3-c and nvtx3-cpp, and you use target_link_libraries to make any CMake target use nvtx3-c for the C API only or nvtx3-cpp for both the C and C++ APIs.

The example in the documentation shows the whole pattern:

# Example C program
add_executable(some_c_program main.c)
target_link_libraries(some_c_program PRIVATE nvtx3-c)
# main.c can now do #include <nvtx3/nvToolsExt.h>

# Example C++ program
add_executable(some_cpp_program main.cpp)
target_link_libraries(some_cpp_program PRIVATE nvtx3-cpp)
# main.cpp can now do #include <nvtx3/nvtx3.hpp>

Since NVTX is a header-only library, these targets simply add the include search path for the NVTX headers, and they add the -ldl linker option if applicable. That is the whole mechanism, and it explains why there is nothing to install and nothing to ship.

The documentation then notes that NVTX provides two different ways to define the CMake targets, one of which is normal non-IMPORTED targets, which are global to the entire build. In a typical CMake codebase, add_subdirectory is used to include every directory in a source tree, and the text explains what that means for target visibility, so if your build pulls in every directory, the scope of an NVTX target is wider than the directory that needed it.

The repository layout is consistent with the three language story. There is a c/ directory holding the C API, a python/ directory, a rust/ directory, a tests/ directory, a tools/ directory, docs/ for the documentation, and a top-level CMakeLists.txt. The C API reference, the C++ API reference and the Python documentation are published under the NVTX GitHub Pages site, with a doxygen build for the C and C++ guides.

Where NVTX is the wrong choice

The first limit is that NVTX does nothing without a tool that understands it. The annotations are invisible to a plain profiler, to a printf style log, and to anything else that has not opted in. If your team wants the information to appear in a standard log, or in a tool you already use that has no NVTX support, this library will not get you there.

The second is the Windows headers problem. Because NVTX includes windows.h on Windows, it defines tokens in the global namespace such as min, max and small, and you have to define macros such as WIN32_LEAN_AND_MEAN and NOMINMAX before including the NVTX headers. For a project with a strict include hygiene, that is a requirement to document and enforce, and it is the kind of thing that gets missed in a port to a new platform.

The third is the manual work behind resource tracking. Spin locks have no interceptible API, so the tool cannot discover them, and every acquisition site needs an annotation by hand. The same applies to any resource whose lifetime you want on the timeline but whose usage the tool cannot intercept.

The fourth is that this is a diagnostic aid rather than a measurement. NVTX tells a tool when and where something happened, and a tool that knows the timings can compute the intervals. It does not tell you that a lock is contended, and it does not replace a lock analysis tool.

Compare that with logging. The difference in approach is that a logging framework records what the program says happened, always, and in a format you control, while NVTX records what a developer tool observes, only when attached, and in a format the tool decides. If you need the events in your own telemetry, a logging library is the better tool. If you need them on a profiler timeline next to the kernel activity, NVTX is the only sensible choice.

Licence, releases and what to check before you instrument a codebase

The licence is one of the custom ones that GitHub cannot categorise, with the terms in the LICENSE.txt file at the repository root. Because this is a library you link into your own application, the licence position decides whether you can ship a commercial product with it, and you should read that file yourself and take your own advice on it rather than relying on a summary.

The release history shows a steady 3.x line. v3.3.0 came on 2025-08-13, v3.4.0 on 2025-12-01 and v3.5.0 on 2026-03-18. The default branch is dev, and the repository is not archived with the last push on 2026-10-16, so the tree is about a week old relative to today.

What to check before you instrument a codebase is a short, practical list. Start with one scoped range in one hot function and look at it under Nsight Systems, because the entire value proposition is the timeline and a marker you never see on one is a marker you will not debug with later. Then check the Windows include situation, since windows.h arriving in your translation unit is a change you want to make deliberately.

Then check the -ldl requirement against the oldest glibc your build supports, since the documentation calls it out specifically for glibc versions prior to 2.34. And decide where the annotations live in your code, because NVTX3_FUNC_RANGE() gives you a function scope for free while the scoped range objects need a place to be declared, and in a codebase with a house style for RAII that is a design conversation rather than a mechanical change.

Editorial conclusion

NVTX fits a team that already profiles with Nsight Systems or another tool that understands the API, and that needs its timelines to carry meaning rather than numeric thread IDs. It does not fit a project that expects a profiler to work without changing the code, since the calls do nothing until a tool intercepts them, and it does not fit a codebase that cannot tolerate the Windows headers NVTX pulls in. Before you adopt it, add one scoped range to a hot function and look at it under a profiler, read the Windows note about min and max in the global namespace, and check which of your platforms are affected by the -ldl requirement on glibc older than 2.34.

Frequently asked questions

Official sources

1. Issues
2. NVIDIA/NVTX on GitHub
3. README
4. Releases