reacTIVision: fiducial marker and multi-touch tracking over TUIO

What reacTIVision solves, and who ends up using it

Building a tangible table from scratch means writing a vision pipeline before you write any application logic: threshold the image, find connected regions, classify each region as a marker or a finger, compute position and rotation, assign stable session IDs across frames, and ship all of that to your application over some protocol you also have to design. reacTIVision exists to remove that work. It is a standalone application, not a library, and it emits Open Sound Control messages over a UDP network socket using the TUIO protocol, which the README describes as having been designed specifically for transmitting the state of tangible objects and multi-touch events from a tabletop surface.

The audience is narrow and specific. The project grew out of the Reactable, a tangible modular synthesizer, and the README frames it as a toolkit for rapid development of table-based tangible user interfaces and multi-touch interactive surfaces. If your work is a museum installation, a music interface, a research prototype or a teaching demo where physical objects on a surface should drive software, this is the intended use. If your work is batch image analysis or a production vision service, the architecture will fight you, because the whole design assumes a live camera and a listening client.

The pipeline: camera frame in, TUIO state out

The mechanism is a single-window application that consumes a live camera feed and performs image analysis on it. The README states that the application shows a single window with the B&W camera video in realtime. Analysis runs against a binary thresholded image, which you can inspect by pressing `T`; pressing `S` returns to the original camera image. The thresholder has a gradient gate adjustable with the `G` key, and the README notes that lowering the value can improve thresholder performance in low light conditions with insufficient finger contrast, with the caveat that you should lower it gradually before noise appears. The tile size of the thresholder is also adjustable.

From that binary image the application derives three classes of object. Fiducial markers are printed symbols from the amoeba set, 216 of them in `./symbols/default.pdf`, and the application detects their ID, position and rotation angle. The new yamaarashi fiducials, described in the README as currently in beta, are said to allow detection of more than one million different symbols within a smaller and more uniform footprint. Fingers are treated as round white regions of a given size, with an average finger size in pixels and a sensitivity percentage where 75 is less sensitive and 125 more sensitive. Blobs are untagged objects, added in version 1.6, which send overall footprint size and orientation through the TUIO blob profile; the maximum blob size is configured in `./reacTIVision.xml`, and blob messages for fingers and fiducials can be enabled so those objects report their size too. Blobs share the session ID of the finger or fiducial they correspond to.

The output side is deliberately narrow. TUIO assigns a session ID to each object in the scene and transmits it alongside the fiducial ID, which is what allows several objects carrying the same marker ID to be tracked separately. That detail matters more than it looks: without session IDs, two copies of marker 12 on the same table would be indistinguishable. Transport is configurable, and up to 32 simultaneous TUIO transports are allowed.

Installing reacTIVision and getting a first marker into your client

The repository does not ship a package. The top-level entries are source directories (`common/`, `ext/`, `linux/`, `macosx/`, `windows/`, `calibration/`, `symbols/`) plus `README.md`, `CHANGELOG.txt`, `GPL.txt` and `LICENSE.txt`, so the practical route is to build from source for your platform. The README has a Compilation section, but the extracted text stops before its contents, so the exact build commands are not reproduced here; read that section in `README.md` before starting.

What the README does give is the runtime workflow. Before starting the application, connect a supported camera, because the README is blunt that it will not work at all without one. Print `./symbols/default.pdf` and attach a label to an object. Then start the application and confirm the camera is being read.

# after building, run the application from the repository root
./reacTIVision

The window should show the black and white camera image in realtime. Press `K` to browse and select available cameras and image formats if the default is not the camera you want. Press `T` to check the thresholded image and `G` to adjust the gradient gate if the surface is dark or the contrast is poor.

The default output is TUIO over UDP to port 3333 on localhost. The README gives this XML tag as the way to change it, and notes that the file is `./reacTIVision.xml`:

<tuio type="udp" host="127.0.0.1" port="3333">

The type options listed in the README are udp, tcp, web and flc, corresponding to TUIO/UDP, TUIO/TCP, TUIO/WEB (websocket) and TUIO/FLC (Flash Local Connection). Add or edit the tag, then run a TUIO client from the example projects for C/C++, C#, Java, Processing, Pure Data, Max/MSP or Flash. Press `V` in the application to print symbol and finger data to the console, which is the fastest way to confirm that detection is happening before you debug the client. Changes made in the configuration are stored automatically when the application closes.

Where reacTIVision stops being the right tool

The most consequential limitation is architectural. reacTIVision is a standalone application, so you cannot link it into your own program and call its tracker. If your application needs to embed marker detection inside a larger process, or to run headless on a server without a display, the design is against you. The README describes a single window with realtime video, and the keyboard shortcuts (`S`, `T`, `N`, `G`, `O`, `K`, `V`, `H`, `F1`, `P`, `ESC`) all assume a person at the machine. Pressing `N` turns the display off to reduce CPU usage, which is the closest thing to a headless mode the documentation describes.

Camera support is another boundary. Under Windows the README specifies any camera with a proper WDM driver, such as most USB, FireWire and DV cameras. Under Mac OS X it says most UVC compliant USB as well as FireWire cameras should work. Under Linux it names FireWire cameras and many Video4Linux2 compliant USB cameras. A camera that does not fall into those classes is not covered by the documentation, and there is no stated fallback.

The third limitation is the release history. The most recent release listed is 1.5.1 from 2016-05-18, while the README is titled reacTIVision 1.6 and describes 1.6 features such as blob tracking and the beta yamaarashi fiducials. So the 1.6 work exists in the repository and in the README, but the release list does not include a 1.6 release. If your project requires a tagged, versioned artifact, that gap matters. The last push to the repository was on 2026-06-28, so the code is not abandoned, but the release channel and the README are out of step, and the beta label on yamaarashi is the project's own wording.

Finally, the README does not document rollback, migration between fiducial sets, or what happens to a client when you switch from amoeba to yamaarashi. Changing the marker set changes what your client sees, and the documentation is silent on the transition.

The alternative: writing your own detector against OpenCV

The obvious alternative is to build the pipeline yourself on top of a general computer vision library such as OpenCV. The difference is not quality, it is where the abstraction sits. OpenCV gives you primitives: thresholding, contour finding, camera capture, and in its contrib modules fiducial detection. You get to decide the data model, the threading, the transport and the failure handling. reacTIVision gives you the opposite: a fixed data model (TUIO objects with session IDs, positions, angles and sizes), a fixed transport (OSC over UDP, TCP, websocket or Flash Local Connection, up to 32 simultaneous transports) and a fixed interaction model (a window and a keyboard).

That trade is worth naming precisely. With OpenCV you will spend time reimplementing session ID assignment across frames, which is the part that makes multi-touch and multi-object tracking usable, and you will own every bug in it. With reacTIVision you get that for free but you inherit its camera assumptions, its binary thresholding approach and its lack of an embeddable API. A team that already has a vision engineer and a specific detection requirement should probably write the detector. A team that wants markers on a table working this week should not.

Licence and maintenance cost

The repository contains both `GPL.txt` and `LICENSE.txt`, and the licence is reported as NOASSERTION, meaning the repository metadata does not map cleanly onto a recognised SPDX identifier. The README carries a copyright line for Martin Kaltenbrunner covering 2005 to 2026. The presence of `GPL.txt` at the top level is the strongest signal available here, but the two licence files together mean you should read them yourself rather than assume. If you plan to distribute reacTIVision bundled with a commercial product, or to modify it, the terms of those files govern what you can do. This is not legal advice; it is a pointer to the two files you need to open.

Upgrade cost is shaped by the release situation. Because the newest listed release is 1.5.1 from 2016 while the README documents 1.6, anyone wanting 1.6 features such as blob tracking is effectively tracking the source tree rather than a release. That means reading `CHANGELOG.txt` and the commit history rather than watching a release feed. The configuration file is a second upgrade surface: `./reacTIVision.xml` stores settings automatically on close, so a hand-edited file can be rewritten by the application, and on Mac OS X the file lives inside the application bundle's Resources folder, reached through Show Package Contents. Rebuilding the bundle can therefore displace configuration you edited there. Budget for keeping a copy of your `reacTIVision.xml` outside the bundle.

Editorial conclusion

Adopt reacTIVision if you are building a tabletop surface and want marker IDs, positions and angles arriving as TUIO without writing vision code. Do not adopt it if you need a library you can link into your own application, if you require a documented release cadence, or if your camera is not a WDM, UVC, FireWire or Video4Linux2 device. Before committing, verify three things on your own hardware: that the camera appears in the K key camera browser, that the printed amoeba sheet from symbols/default.pdf detects reliably under your lighting, and that your client receives packets on the transport you configured in reacTIVision.xml.

Frequently asked questions

Official sources

1. Issues
2. mkalten/reacTIVision on GitHub
3. README
4. Releases