# MediaPipe TouchDesigner: GPU Pose, Hand and Face Tracking Without a Python Install > The plugin wraps Google's MediaPipe vision tasks in a Chromium browser launched from a .tox, exposing each model as DAT and TOP outputs. It is a fast route to tracking in TouchDesigner, with a 720p input ceiling and model exclusions worth knowing before you commit. **torinmb/mediapipe-touchdesigner** — GPU Accelerated MediaPipe Plugin for TouchDesigner - Repository: https://github.com/torinmb/mediapipe-touchdesigner - Stars: 2,768 · Forks: 134 - Language: JavaScript - License: MIT - Published: 2026-09-28 · Updated: 2026-09-28 · Language: en - Canonical page: https://hysenlabs.com/projects/torinmb-mediapipe-touchdesigner ## The gap this fills in a TouchDesigner patch TouchDesigner has no built-in pose or hand landmark detection. The usual workaround is a Python environment with the MediaPipe pip package, a script CHOP or DAT bridge, and a separate process to keep alive while your patch runs on stage. Each of those is a place where a show can fail. This plugin removes the Python layer entirely. It bundles a Chromium browser and the MediaPipe vision tasks into a single .tox, so the models run in the browser process and TouchDesigner reads the results back through native operators. The README describes it as "GPU Accelerated, self-contained" and says it runs on Mac and PC with no installation. Who it is for: installation artists, VJ tooling, interactive kiosks, and anyone prototyping gesture or face-driven control inside TouchDesigner who does not want to maintain a parallel Python stack. The repository topics place it in the td-olib-project ecosystem, which is where TouchDesigner users look for reusable components. ## How the browser, the models and the DAT outputs connect The MediaPipe.tox component launches a Chromium web browser. That browser hosts and runs every MediaPipe vision task, and the component exposes one DAT output per task plus a TOP output carrying the video feed and any overlays. So the data flow is: webcam or Spout/Syphon frame into the browser, model inference in the browser, structured results back out to TouchDesigner as DAT rows and a rendered TOP. The supporting .tox files are consumers, not engines. Face detector, face tracking, hand tracking, object tracking, pose tracking, image segmentation and image classification each process the results of the corresponding task. The README links each one to its MediaPipe task guide, which is a useful signal that the plugin is a thin transport layer over Google's API rather than a reimplementation. On the build side, package.json pins @mediapipe/tasks-vision at 0.10.18 and uses Vite 6 with vite-plugin-static-copy. The repository ships index.html, src/, public/ and a yarn.lock, with yarn 4.6.0 as the declared package manager. That means the browser bundle is a real front-end project you can rebuild, not an opaque binary. If you only ever use the release.zip, you never touch any of it. ## Installing the plugin and getting one model running There is no package manager step for end users. The README points at the release section of the repository, and the download badge targets release.zip from the latest release (v0.5.3, published 2026-08-10). Unzip it and open the included MediaPipe TouchDesigner.toe file. Inside that project, the components live in the /toxes folder. MediaPipe.tox is the main component; the rest are examples showing how to load and display each model's data. The README carries one warning in bold terms: when dragging the MediaPipe component into a new project, select "Enable External .tox", otherwise your .toe file size will be massive. That setting is the difference between a project that references the component and one that swallows it whole. Once the component is loaded, the workflow is a drop-down and a set of toggles. Select your webcam from the drop-down on the MediaPipe component, turn on the MediaPipe model you want, enable the preview overlay to confirm tracking, then read the matching DAT output for that task. The README also notes sub-menus on each model for further customisation. What you should see after enabling the overlay is the video feed with landmarks or boxes drawn over it in the TOP output, and rows appearing in the task's DAT. If the webcam drop-down is empty, the browser has not been granted device access, and that is a browser-level problem rather than a TouchDesigner one. If you want to rebuild the browser side yourself, the repository is a Vite project with dev, build and preview scripts declared: ```bash yarn install yarn dev ``` Nothing in the README suggests you need this path to use the plugin. ## Feeding TouchDesigner TOPs back into the models on Windows and Mac Webcam input is the default, but the more interesting case is sending a TOP from your own patch into MediaPipe. On Windows the README describes a Spout route: put a Syphon Spout Out TOP on any TOP, install SpoutCam, and register a virtual webcam. The setup steps are specific. Run SpoutCam Settings.exe (there is no installer; it runs from wherever you unzip it), enter the frame rate and resolution to match your source, enter the default TouchDesigner Spout output name TDSyphonSpoutOut into the Starting Sender box, then click register. In MediaPipe, select SpoutCam as the webcam source. The README claims the media appears with no frames of delay. The troubleshooting note is the most valuable part of that section and deserves attention before you buy into the approach. When SpoutCam shows only noise, the maintainer's own diagnosis was a laptop with both integrated graphics and a discrete GPU: texture sharing was failing because the sender and receiver were on different graphics pipelines. The fix was to force every Spout process onto the same pipeline via Windows .exe graphics settings, documented in SpoutSettings rather than SpoutCamSettings. On Mac there is no SpoutCam equivalent. The README's suggested path is Syphon into OBS, then OBS Virtual Webcam into the MediaPipe task. The README itself calls this "not the most elegant solution" and the text is truncated at that point, so the full caveats are not documented in the source available here. Treat the Mac TOP-input route as workable but second-class compared with Windows. ## The 720p ceiling and the two models you cannot use The README states plainly that the model is currently limited to 720p input resolution, and that as long as your webcam supports that resolution you are fine. This is a hard constraint, not a tuning suggestion. If your source is a 4K camera, a high-resolution render, or a Spout feed at 1080p, you are either downscaling before the plugin sees it or working outside the documented envelope. For face and hand landmarks on a single performer, 720p is usually plenty. For tracking small objects across a wide stage, it is a real limit on how much detail the model receives. The second constraint is coverage. The README says the project currently supports all MediaPipe vision models except Interactive Segmentation and Image Embedding. Those two are simply absent, and no workaround is documented. If your concept depends on generating embeddings for similarity search, or on interactive segmentation with user-guided points, this plugin is the wrong starting point. A third, softer limitation sits in the segmentation section. The README advises using the MultiCam model in MediaPipe.tox for the most accurate webcam cutout, and notes there is a toggle to display only the background cutout while on the multiclass model. That is guidance, not a guarantee, and cutout quality against a busy background is the kind of thing you have to judge on your own footage rather than from a README. ## What you give up compared with a Python MediaPipe pipeline The obvious alternative is running MediaPipe directly in Python and bridging results into TouchDesigner yourself, typically through a script CHOP or a shared memory or OSC channel. The difference in approach is where the model lives. In the Python route, inference runs in a process you control, you can call any MediaPipe API including the ones this plugin omits, you can feed frames at whatever resolution you like, and you can post-process results in the same language as the model call. The plugin trades that control for integration. You get DAT outputs already shaped for TouchDesigner, a TOP with overlays drawn, and no environment to rebuild on a new machine. You lose API surface, the resolution ceiling applies, and you inherit a Chromium process as part of your patch. For a touring show where the machine gets reimaged, the plugin's self-contained nature is worth more than the flexibility. For research or anything needing embeddings, it is not close. A second alternative is doing the tracking in a separate application entirely and sending only the coordinates into TouchDesigner over OSC or a similar channel. That keeps TouchDesigner light but adds a second machine or process to synchronise, which is the exact problem the plugin was built to avoid. ## Maintenance, licence and what a future upgrade costs you The repository is not archived, and the last push was on 2026-08-19, roughly six weeks before this writing. Releases are irregular rather than frequent: v0.5.1 on 2025-06-15, v0.5.2 on 2025-11-06, and v0.5.3 on 2026-08-10. That cadence suggests a project that ships when there is something to ship rather than on a schedule. Plan upgrades around releases, not around a support contract. The licence is MIT, which is permissive and places few obligations on how you redistribute or embed the plugin. Note that the plugin wraps Google's MediaPipe and ships @mediapipe/tasks-vision as a dependency; that dependency carries its own terms, and the MIT licence on this repository does not speak for it. That is a fact about the dependency graph, not legal advice, and if you are shipping a commercial product you should read the MediaPipe terms yourself. Upgrade cost is low in one direction and non-zero in another. Because the component is external, replacing MediaPipe.tox is mostly a file swap. But the DAT output schema is what your patch is wired to, and the README does not document a versioned output contract or a migration path between releases. Before upgrading a working installation, open the new release's example .toe files alongside your own patch and compare the DAT columns your expressions reference. ## Conclusion Adopt it if you are building TouchDesigner work that needs face, hand, pose, object or segmentation tracking and you want it running in an afternoon rather than a week. Skip it if you need Interactive Segmentation or Image Embedding, if your source is above 720p, or if you are on a dual-GPU laptop where Spout texture sharing is already misbehaving. Before you build anything, drag MediaPipe.tox into a scratch project with Enable External .tox checked, confirm your webcam appears in the drop-down, and toggle one model on to watch the DAT output fill. ## FAQ ### How do I install the MediaPipe TouchDesigner plugin? Download release.zip from the repository's release section, unzip it, and open the included MediaPipe TouchDesigner.toe file. The components sit in the /toxes folder, with MediaPipe.tox as the main component. ### What is MediaPipe TouchDesigner? It is a GPU accelerated, self-contained MediaPipe plugin for TouchDesigner that runs on Mac and PC with no installation. It launches a Chromium browser to host the MediaPipe vision tasks and exposes each task as a DAT output plus a TOP output for the video feed and overlays. ### Does the plugin run on Mac? Yes, the README says it runs on Mac and PC with no installation. Sending TouchDesigner TOPs into the models is the weaker path on Mac: there is no SpoutCam equivalent, so the README suggests Syphon into OBS and then the OBS Virtual Webcam output into the MediaPipe task. ### Which MediaPipe models does the plugin not support? The README states the project currently supports all MediaPipe vision models except Interactive Segmentation and Image Embedding. No workaround for those two is documented. ### Why is my SpoutCam feed showing only noise in MediaPipe TouchDesigner? The README attributes this to texture sharing failing, often on a laptop with both integrated graphics and a discrete GPU, where the sender and receiver sit on different graphics pipelines. The documented fix is to put every Spout process on the same pipeline using the Windows .exe graphics settings described in SpoutSettings. ## Sources - [Issues](https://github.com/torinmb/mediapipe-touchdesigner/issues) - [License: MIT](https://github.com/torinmb/mediapipe-touchdesigner/blob/main/LICENSE) - [README](https://github.com/torinmb/mediapipe-touchdesigner/blob/main/README.md) - [Releases](https://github.com/torinmb/mediapipe-touchdesigner/releases) - [torinmb/mediapipe-touchdesigner on GitHub](https://github.com/torinmb/mediapipe-touchdesigner) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/torinmb-mediapipe-touchdesigner