Nextcloud Recognize: on-premises media tagging with TensorFlow.js in Node
👁 👂 Smart media tagging for Nextcloud: recognizes faces, objects, landscapes, music genres
At a glance
- What is it?
- Recognize is a Nextcloud app that adds Collaborative Tags to photos, videos and audio using pre-trained models that run on your own server. It is a good fit for self-hosters who want searchable media without sending files to a cloud API, and a poor fit for Alpine-based containers expecting native inference speed.
- Who is it for?
- Adopt Recognize if you run Nextcloud on glibc Linux with roughly 4GB of free RAM and x86-64 with AVX, and you want face, object, landmark, video action and music genre tags written into Collaborative Tags without any file leaving the server. Do not adopt it on end-to-end encrypted libraries, on Alpine or Nextcloud AIO if you expect native inference speed, or anywhere the app's stated limited-effort maintenance is a problem.
- Can I use it commercially?
- Yes, with strict conditions. AGPL-3.0 is a network copyleft licence: if people use a modified version over a network, for example as a hosted service, you must offer them its source code under the same licence.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly PHP, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What Recognize adds to a Nextcloud media library
A stock Nextcloud photo library is searchable by filename, date and whatever tags a human typed in. Recognize fills that gap automatically. According to the README, the app walks the media collection and adds fitting tags, covering faces and grouping photos by the faces that appear in them, animals, landscapes, food, vehicles, buildings and other objects, landmarks and monuments, music genres, and human actions in video. The tagging surface is Nextcloud's Collaborative Tags, which means downstream apps can consume the results: the README points to the audioplayer app for tagged music and the memories app for tagged photos and videos, and notes that the face grouping GUI lives in memories rather than in Recognize itself. The target user is a self-hoster who already runs Nextcloud and wants a searchable library without shipping personal photos to a third-party inference API. The privacy section states plainly that no sensitive data goes to cloud providers, because all image processing happens on the Nextcloud machine.
The inference stack: TensorFlow.js in Node, five pre-trained models
Recognize does not train anything. It bundles TensorFlow.js running in Node.js, and the README lists the models it loads: an EfficientNet v2 pre-trained model for ImageNet object detection, a model trained on the Landmarks v1 dataset, face-api.js for extracting and comparing face features, a Musicnn architecture for classifying audio into genres, and a pre-trained MoViNet model for video classification. The data flow is one-directional: files are read from the Nextcloud storage layer, temporarily staged in /tmp for recognition, passed through the relevant model, and the output is written back as Collaborative Tags. Model sizes are given in the README and matter for planning: 1GB for object recognition, 300MB for landmarks, 50MB for video action recognition and 50MB for music genre recognition. Face recognition is handled by feature extraction and comparison rather than a named model download, so the README does not attach a size to it. Tag names are not hardcoded in PHP. The README links a src/rules.yml file as the list of recognized things and their current category mappings, and invites pull requests against that file to fine-tune predictions, which is the practical place to look when a label you expect is missing or lands in an odd category.
Requirements, install paths and the /tmp staging area
The README gives two install routes. The one-click route is Apps, search for recognize, click install, with a wiki page linked for when that fails. The manual route is a git clone into /path/to/nextcloud/apps/ followed by make, and it lists make, git, Node.js v16.x and npm, php 8.0 or later, and composer as dependencies. Runtime requirements are php 8.0 and above, the Collaborative Tags app enabled, and roughly 4GB of free RAM, with a note to have swap available if you are close to that limit. The speed split is the part most people skim past. Native speed needs an x86 64-bit processor with AVX support and a glibc system, and the README explicitly excludes FreeBSD, Alpine Linux and therefore Nextcloud AIO from that group. Those systems fall back to a WASM mode that also runs on arm64 and armv7l and works with glibc or musl, but the README labels it sub-native speed. Configuration lives in Settings/Recognize. Directory exclusions are filesystem markers rather than config keys: drop .noimage, .nomusic, .novideo or .nomedia into a folder to skip image, music, video or all recognition for that subtree. For Docker, the README suggests mounting a tmpfs at /tmp with the exec flag (--mount type=tmpfs,destination=/tmp:exec on the command line, or a volumes entry with type: tmpfs and target: /tmp:exec in docker-compose.yml) to speed up processing and reduce disk load, with a warning that RAM must be large enough to hold big files or public uploads will fail.
Where Recognize stops: encryption, Alpine and shared disks
The clearest hard boundary is end-to-end encryption. The README states that end-to-end encrypted files cannot be processed by recognize, because the server by design cannot read them. That is not a bug to be fixed; it is the encryption model working as intended, and it means a library protected that way will simply stay untagged. The second boundary is the platform split described above. Running on Alpine or Nextcloud AIO does not block the app, but it moves you to WASM mode, which the README describes as sub-native speed. On a large photo library that difference compounds across every file. The /tmp staging design is the third pressure point. Recognition writes temporary files there, and the README's own warning is that insufficient RAM for large files causes public uploads to fail, which is a failure mode that surfaces in an unrelated feature and can be confusing to diagnose. Resource pressure is the fourth: 4GB of free RAM is a floor, and the object model alone is 1GB, before landmarks, video and audio models are loaded. Finally, the README's maintenance section states the app is maintained with limited effort, that the main functionality works for the majority of use cases, and that the maintainers will ensure it continues to work for future releases. That is an honest disclosure, and it should be read as a commitment to keep the app working rather than a promise of new capability.
How it compares with a standalone ML service
The obvious alternative for a self-hoster is to skip in-app inference and run a separate machine learning service, for example a container that exposes an HTTP endpoint for image classification or face embedding, then write a small Nextcloud integration that posts new files to it and applies tags. The difference is architectural rather than cosmetic. Recognize keeps the models inside the Nextcloud app and calls them from PHP through a bundled Node.js and TensorFlow.js runtime, so there is no second service to deploy, no network hop, and no extra API surface to secure. The cost is that inference competes with Nextcloud for the same CPU and RAM, and that the app is bound to the Node and model versions it ships. A standalone service inverts that: you get independent scaling, a choice of model runtime, and the ability to upgrade the model without touching Nextcloud, at the price of running and monitoring another component and building the glue that maps predictions onto Collaborative Tags. Recognize also has a narrower scope than a general ML service. It targets a fixed set of outputs (faces, ImageNet objects, landmarks, video actions, music genres) and routes them through src/rules.yml. If your requirement is a custom classifier over your own labels, the app's model list is the constraint, not a starting point.
Licence and the ethical AI rating
Recognize is licensed AGPL-3.0, which is the same licence family as Nextcloud itself and is the standard for apps distributed through the Nextcloud app store. The practical consequence for most self-hosters is nil. For anyone embedding Recognize in a modified hosted service, AGPL-3.0 carries network-use obligations that differ from permissive licences, and that is a question for your own counsel rather than something to settle from a README. The README also carries Nextcloud's Ethical AI Rating per model. Photo object detection, photo face recognition and video action recognition are rated green, with the stated positives being open source training and inference software, a freely available trained model that can run on-premises, and freely available training data that allows external checking for bias or optimization of performance and CO2 usage. Music genre recognition is rated yellow: the software and the trained model are open source and can run on-premises, but the training data is not freely available, which the README says limits external parties from checking and correcting for bias or optimizing the model. If auditability of the training data matters in your context, that single yellow rating is the item to weigh, and it is the one model in the stack where you cannot independently inspect what the classifier learned from.
Maintenance, upgrades and what to verify before adopting
Upgrade cost is mostly model downloads and reprocessing. The README lists model sizes of 1GB, 300MB, 50MB and 50MB, so a fresh install or a model change pulls roughly that much before any tagging happens, and the app then has to work through the existing library. Because tagging output is written into Collaborative Tags, reprocessing is not free of side effects: tags already applied to files are part of your Nextcloud data, and a change in src/rules.yml or in a model can shift which tags appear. The README treats rules.yml as a living mapping file open to pull requests, which is a signal that category names can change between versions. On the maintenance side, the README states limited effort and commits to keeping the app working for future releases. That is a reasonable posture for an app whose models and dependencies are pinned, but it means you should not expect the recognized category list to expand quickly. Before adopting, verify four things: that Collaborative Tags is enabled, since the app depends on it; that your platform is glibc x86-64 with AVX if you want native speed, or accept WASM mode; that /tmp has adequate space and RAM for your largest files, given the public upload failure mode; and that your libraries are not end-to-end encrypted, because those files will never be processed. If face grouping is the feature you actually want, check that the memories app is installed, since the README places that GUI there and not in Recognize.
Editorial conclusion
Adopt Recognize if you run Nextcloud on glibc Linux with roughly 4GB of free RAM and x86-64 with AVX, and you want face, object, landmark, video action and music genre tags written into Collaborative Tags without any file leaving the server. Do not adopt it on end-to-end encrypted libraries, on Alpine or Nextcloud AIO if you expect native inference speed, or anywhere the app's stated limited-effort maintenance is a problem. Before installing, confirm that the Collaborative Tags app is enabled, check that /tmp has room for the largest file you will process, and read src/rules.yml to see which ImageNet categories map to which tag names.
Community notes