ViSP: the Inria visual servoing library for closed-loop robot control

What ViSP solves, and who it is actually for

Most computer vision libraries stop at perception. They hand you a pose, a bounding box or a set of matched points, and the control problem starts after that. ViSP is built around the opposite assumption: the camera is a sensor inside a control loop, and the library's job is to compute the control law. The README describes it as a cross-platform library that allows "prototyping and developing applications using visual tracking and visual servoing technics", able to "compute control laws that can be applied to robotic systems".

The intended user is therefore not the average app developer adding a barcode scanner. It is a robotics researcher or engineer who has a manipulator, a mobile base or a pan-tilt head, a calibrated camera, and a task expressed in image or pose space. The README lists robotics, computer vision, augmented reality and computer animation as the areas where the library is useful, and the codebase carries the marks of that audience: a demo directory, an example directory, a tutorial directory, and a separate development directory for work in progress.

The lineage matters for expectations. ViSP comes from the Inria Rainbow team, and before 2018 from the Lagadic team, and the README asks you to cite a 2005 IEEE Robotics and Automation Magazine paper, a 2018 IROS paper on a model-based tracker, and a 2016 TVCG survey on pose estimation for augmented reality. This is a research platform that has been maintained long enough to accumulate depth, not a product with a marketing site. The homepage is visp.inria.fr, and the Q&A channel is GitHub Discussions rather than a support contract.

How the tracking and servoing pipeline is put together

The architecture visible from the repository is a C++ library plus CMake build, organised into modules. The README states that ViSP "provides a set of visual features that can be tracked using real time image processing or computer vision algorithms" and that it "provides also simulation capabilities". That sentence is the core data flow: an image source feeds a tracker, the tracker produces a visual feature, and a control law consumes that feature to produce a velocity command for the robot.

The feature is the pivot of the whole design. In classical visual servoing you choose what the error is computed on, and the library's value is that it ships a range of those choices rather than one. The related searches people run include "Visp pbvs", which points at position-based visual servoing, the variant where the error is expressed in 3D pose rather than directly in image coordinates. The README's citation list reinforces this: the 2016 survey is specifically about pose estimation for augmented reality, and the 2018 paper describes a modular framework for model-based visual tracking using edge, texture and depth features. A model-based tracker of that kind needs a CAD model of the object and a renderer to compare against, which is why the simulation capability is not a side feature but part of the tracking story.

The repository layout supports the reading that ViSP is meant to be embedded rather than run as a standalone application. There is no server binary or service described in the README. There is a 3rdparty directory, a cmake directory, a macros directory, a platforms directory, and separate apps, samples, demo, example and tutorial trees. The practical consequence for an adopter is that integration means linking a C++ library into your own node or program, and that the examples are the primary documentation of the API surface.

Installing ViSP and running a first example

The README does not contain install instructions. It points to INSTALL.txt at the repository root and to the wiki and doxygen documentation, so treat those as the authoritative sources rather than any snippet elsewhere. What the README does establish is the build system and the supported targets: CMake, with CI covering Ubuntu 20.04 and 22.04, macOS 13 and 14, iOS on macOS 11.0, Windows 10, aarch64 and s390x on Ubuntu 22.04, and ROS2 Humble, Jazzy, Kilted and Rolling. The repository also carries flake.nix, flake.lock, default.nix, pixi.toml and pixi.lock, so Nix and Pixi environments are part of the project's own tooling.

The conventional path is a CMake configure and build from a clone of the master branch. The repository root contains CMakeLists.txt, which is what you point CMake at:

git clone https://github.com/lagadic/visp.git
cd visp
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

After the build finishes, the executables land under the build tree. The project ships a demo directory containing wireframe-simulator alongside its CMakeLists.txt, and separate tutorial and example directories. Check what your configuration actually produced before assuming a target exists, because optional dependencies gate large parts of the library:

ls demo/ example/ tutorial/

The README's resource list gives the doxygen pages at visp-doc.inria.fr as the code documentation, and that is where each tutorial binary is described. If you are working inside ROS2, the README shows CI jobs for Humble, Jazzy, Kilted and Rolling on Ubuntu, which is the signal that a ROS2 integration path exists; the README itself does not document the node names or topics, so read the ROS2 build job and the wiki before wiring anything into a robot.

Where ViSP is the wrong tool

The licence is the first hard boundary. ViSP is GPL-2.0. If you are shipping a closed-source product that links the library, the GPL-2.0 obligations apply to the combined work, and that is a decision for your legal counsel rather than something this article can settle. For a proprietary product, the realistic options are to keep ViSP on the research side of the wall and reimplement, or to accept the licence. There is no permissive dual-licence path described in the README.

The second boundary is scope. ViSP is not a general-purpose vision toolkit and it is not a deep learning framework. The README describes visual features tracked with "real time image processing or computer vision algorithms", which in practice means classical tracking: edges, texture, depth, model-based rendering. If your problem is detecting arbitrary objects in cluttered scenes with a trained network, ViSP does not solve it, and you would be adding a heavy C++ dependency for no benefit.

The third boundary is the documentation shape. The README is a landing page, not a manual. It gives no install steps, no API walkthrough, and no statement about rollback or version pinning beyond the release tags. Everything operational lives in INSTALL.txt, the wiki, and the doxygen pages. That is normal for a research library, but it means the cost of adoption is front-loaded into reading source and examples. A team that expects a quickstart page with copy-paste code for every feature will be frustrated.

Finally, the release cadence is uneven. v3.5.0 shipped on 2022-02-15, v3.6.0 on 2023-09-22, and v3.7.0 on 2025-12-19. That is roughly one tagged release every one to two years, with the last push to master on 2026-09-15. Development is continuous, but if your process requires frequent stable tags, plan around that rhythm.

How ViSP differs from OpenCV and from ROS2 vision packages

The obvious comparison is OpenCV, and the difference is not quality but centre of gravity. OpenCV is a perception library: it gives you image processing, feature detection, calibration, and a very large catalogue of algorithms that end at a result. ViSP starts where that result becomes an input to control. The README's own framing, "compute control laws that can be applied to robotic systems", is a capability OpenCV does not claim. If you need a descriptor matcher, use OpenCV. If you need the error signal and the control law for a camera-in-the-loop task, that is ViSP's territory, and the two libraries are routinely used together rather than as substitutes.

The second comparison is with ROS2 vision packages. Those tend to be nodes: they subscribe to an image topic, publish a detection or a pose, and leave the control to whatever consumes the topic. ViSP is a library you link into a node, and the README's CI matrix shows builds against Humble, Jazzy, Kilted and Rolling, so the intended pattern is ViSP inside a ROS2 process rather than ViSP as a node. That gives you direct access to the control law in the same address space as your robot interface, at the cost of writing more glue yourself.

A third reference point is the model-based tracking literature itself. The README cites the 2018 IROS paper on a modular framework using edge, texture and depth features, which is a description of an approach rather than a competing product: multiple feature types combined in one tracker, with a CAD model rendered and compared against the image. If your task is tracking a known object with a known geometry, that design is directly relevant; if your objects are unknown, it is not.

Maintenance, releases and what the GPL-2.0 means in practice

The repository is not archived, and the last push to master was on 2026-09-15, so the project is under current development. That said, the tag history is the number to plan against: v3.5.0 on 2022-02-15, v3.6.0 on 2023-09-22, v3.7.0 on 2025-12-19. An upgrade from one release to the next can therefore span a year or more of accumulated changes, and the ChangeLog.txt at the repository root is where those changes are recorded. Read it before moving a pinned version, because there is no documented long-term-support branch in the README.

The CI matrix is the best available statement of what the maintainers actually test: Ubuntu 20.04 and 22.04, macOS 13 and 14, iOS on macOS 11.0, Windows 10, aarch64 and s390x, plus the four ROS2 distributions, Valgrind, sanitizers and code coverage. A platform outside that list is not necessarily broken, but it is untested by the project, which shifts the burden onto you.

On licensing: the repository carries LICENSE.txt and the README badges the project as GPL-2.0. The GPL-2.0 is a copyleft licence, so distributing a binary that links ViSP triggers source-availability obligations for the combined work. This article is not legal advice; the point for an engineering decision is simply that the licence, not the API, may be the blocking constraint for a commercial deployment. If you are integrating into a ROS2 stack, note that the ROS2 build jobs in the README are a separate concern from the licence of your own packages, and the two need to be checked independently.

Editorial conclusion

Adopt ViSP if your problem is a control loop closed around camera measurements and you are comfortable in C++ with CMake; the library's own framing is prototyping and developing visual tracking and visual servoing applications, and its Inria research lineage shows in the breadth of servoing and model-based tracking code. Do not adopt it if you need a permissive licence for a closed product, or if you only want a trained detector or a descriptor matcher, which is OpenCV's ground rather than ViSP's. Before committing, verify three things on your own machine: that a from-source or package build succeeds with the optional dependencies you need, that one of the demo or tutorial programs runs against your camera, and that the specific visual feature or tracker you intend to use is documented in the doxygen pages for the release you are pinning.

Frequently asked questions

Official sources

1. lagadic/visp on GitHub
2. License: GPL-2.0
3. Project website
4. README
5. Releases