tensorflow/tfjs-models: pretrained models that run in the browser tab

What tfjs-models actually is, and who it is for

This repository is a collection of pretrained models that have been ported to TensorFlow.js. It is not a framework and not a training toolkit. Each model lives in its own top-level directory with its own README, its own demo and, for most entries, its own npm package under the @tensorflow-models scope. The README states the intent plainly: the models are hosted on NPM and unpkg "so they can be used in any project out of the box", and they can be used directly or in a transfer learning setting.

The audience is narrower than "JavaScript developers". It is people who need image classification, object detection, pose estimation, face landmarks, speech command classification or text toxicity scoring inside a page, and who do not want to stand up a Python inference service. The README adds a second audience signal: the APIs generally try to hide tensors so they can be used by non-machine learning experts. That design choice is the whole pitch. You get a promise-returning function, not a graph.

If you are training models or need custom architectures, this repository is the wrong layer. It ships weights and inference wrappers. The training story, if you want one, is transfer learning on top of what is already here.

The directory-to-package mapping is the real interface

The root README is a table, and the table is the contract. Each row pairs a model name, a live demo link, a one-line description and an install command. MobileNet classifies images using ImageNet labels. COCO-SSD localizes and identifies multiple objects in a single image and is described as based on the TensorFlow object detection API. DeepLab v3 does semantic segmentation. Pose and Hand pose detection give real-time detection in the browser. Face Landmark Detection infers approximate surface geometry of a face. Speech Commands classifies one-second audio snippets. Universal Sentence Encoder turns text into a 512-dimensional embedding. Text Toxicity scores a comment from "Very toxic" to "Very healthy". Portrait Depth sits under Depth Estimation.

The mapping is not one-to-one with the directory list. The repository root also contains body-pix, body-segmentation, face-detection, handpose, posenet, blazeface, gpt2, qna, knn-classifier, model-playground, shared, tasks and tools. Several of those are older generations of models that the table has since replaced with the newer pose-detection and hand-pose-detection packages. The README does not explain the relationship between posenet and pose-detection, or between handpose and hand-pose-detection. A reader who finds the older directory first will not learn from the root README that a newer package exists.

That gap is worth naming. The table is the supported surface; the directory listing is history plus infrastructure. Treat them as different things.

Installing one model and running a first inference

Installation is per model, not per repository. There is no single @tensorflow-models package that pulls everything in. The README gives the exact npm command for each model in the table, so the safest path is to copy the command for the row you want. For image classification that row is MobileNet:

npm i @tensorflow-models/mobilenet

After the install, the package is imported and loaded by name. The README does not print a full code sample in the table, but it does state that the API hides tensors from the caller, so the shape of a first use is a load call followed by a classify call. The model weights are fetched at load time from the hosting the project uses, which means the first call is a network operation, not a local file read. The README points to a live demo and a source demo under ./mobilenet for the working example, and says to look at the README in each model's directory for its API.

For object detection the command changes and so does the package:

npm i @tensorflow-models/coco-ssd

COCO-SSD is described as localizing and identifying multiple objects in a single image, so its output is a list of detections with boxes rather than a single label. The same pattern holds across the table: pose detection installs as @tensorflow-models/pose-detection, face landmarks as @tensorflow-models/face-landmarks-detection, speech commands as @tensorflow-models/speech-commands, toxicity as @tensorflow-models/toxicity, and the sentence encoder as @tensorflow-models/universal-sentence-encoder. Each of those has a live demo linked from the table, which is the fastest way to confirm the model behaves as you expect before writing any code.

One caution about the repository itself. Its package.json is named tensorflow-models at version 0.0.1 and contains only devDependencies: rollup, typescript 4.9.5, tslint, jasmine, ts-node, yalc and similar build tooling. Cloning the repository and running npm install gives you the monorepo build environment, not the models. The models come from the individual published packages.

Where the browser-first design costs you

The first limitation is payload. Every one of these models downloads weights into the client. The README does not publish size figures for any model, so you cannot budget from the table alone; you have to load the model and measure. On a mobile connection this is the difference between a usable page and an abandoned one, and the repository offers no documented caching or quantization guidance in the root README.

The second is that the root README is a directory of directories. It says to look at the README in each model's directory for APIs, and it does not document versioning policy, deprecation timelines or a migration path between the old and new pose packages. If you need to know whether posenet is still receiving fixes, the root README will not tell you. You have to read the individual directory.

The third is scope. This is a fixed catalogue. There is a contribution path, but it is gated: the README asks anyone wanting to contribute a model to first file a GitHub issue on tfjs to gauge interest, and says the project is trying to add models that complement the existing set and can be used as building blocks. That is a deliberate filter, and it means you cannot assume a model you need will ever land here.

Finally, the release history is thin. The two releases listed are tasks-v0.0.1-alpha.6 from 2021-05-13 and pose-detection-v0.0.1-rc.0 from 2021-04-08. The tasks package is still labelled alpha and the pose detection package was at release candidate at that point. Neither label suggests a frozen, long-term-stable API, and the root README does not state a stability guarantee for any package.

The alternative: run the model server-side instead

The obvious alternative is not another JavaScript library. It is the same class of model running in a Python service, using the TensorFlow or PyTorch ecosystem the weights originally came from. TensorFlow's object detection API is named in the README as the basis for COCO-SSD, which tells you the upstream lineage directly.

The difference in approach is where the compute and the data live. With tfjs-models, the image or audio never leaves the device, latency is bounded by the client's hardware, and you pay nothing per inference. With a server-side model, you get access to the full model zoo and to GPU instances you control, you can swap weights without shipping a new frontend build, and you can log inputs for evaluation. You also take on the cost of uploading user media, running inference infrastructure, and handling the privacy questions that come with it.

There is a middle position worth knowing about, because the repository itself contains it: the tasks directory and the shared directory exist alongside the model packages, which indicates the project has been moving toward a unified task API layered over the individual models. The release list shows tasks-v0.0.1-alpha.6, so that layer is still alpha. If you are choosing between per-model packages and a unified API, the per-model packages are the ones with the documented install commands in the README.

Licence and the cost of keeping up

The repository is Apache-2.0, and the root package.json declares "license": "Apache-2.0" as well. Apache-2.0 is a permissive licence that includes an express patent grant, which matters for a model catalogue because model weights and the code that loads them can carry different obligations upstream. The repository does not spell out the licence of each individual model's weights or its training data. ImageNet is named as the label source for MobileNet and the speech commands dataset is named for the audio model, but the root README does not state the licensing terms attached to those datasets. If you are shipping a commercial product, that is the item to check in each model's own directory before you rely on it.

Upgrade cost is low in the ordinary case and unclear in the awkward one. Because each model is a separate npm package, upgrading MobileNet does not touch your pose detection code. That isolation is the main practical benefit of the layout. The awkward case is the old-to-new transition: posenet and handpose sit next to pose-detection and hand-pose-detection in the same repository, and the README does not describe how to move from one to the other. Budget time for reading both directories if you are starting from an older tutorial. The repository also carries a presubmit script and a run_python_tests.sh at the root, and the package.json exposes presubmit via ts-node, which tells you contributions are expected to pass a check before merge. That is a cost if you plan to fork and patch a model rather than consume it.

Editorial conclusion

Adopt tensorflow/tfjs-models when the model you need is in the table and inference has to happen in the browser or in Node without a Python service: MobileNet and COCO-SSD are the safest bets because their packages are published and their demos are linked. Do not adopt it for a task the table does not list, and do not assume a directory under the repository root is installable, since gpt2, qna and knn-classifier have no npm command in the README. Before committing, install the exact package name from the table, confirm the model weights actually download in your target browser, and read that model's own directory README, because the root README deliberately defers API details to it.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. tensorflow/tfjs-models on GitHub