Sapiens2 and the cost of a 1K human vision backbone

One billion human images, six checkpoint sizes

Sapiens2 is a family of high resolution vision transformers, and the README describes it as pretrained on 1 billion human images. The model card lists six configurations, from Sapiens2-0.1B at 0.114 B parameters up to Sapiens2-5B at 5.071 B, with embedding widths from 768 to 2432 and depth from 12 to 56 layers. The headline number for most readers is the 1B entry: 1.462 B parameters, 4.715 T FLOPs, embed dim 1536, 40 layers, 24 heads.

The scope is human centric rather than general purpose. The task table in the README covers pose estimation over 308 whole-body keypoints, body-part segmentation into 29 parts, per-pixel surface normal estimation, per-pixel 3D pointmap estimation, and human matting that returns an alpha matte together with a foreground. Each task has an inference page and a separate training page under docs/.

Resolution is the part that separates this from a typical vision backbone. Every model uses patch size 16 and is trained at 1024x768, except Sapiens2-1B (4K), which is trained at 4096x3072 with use_tokenizer=True. A 1K native input is a different memory and latency profile from a 224 or 336 pipeline, and that cost lands on your inference budget before it lands on any accuracy chart.

The standalone backbone file that needs only torch

The most interesting engineering decision in the project is how little it asks of you to run a forward pass. The Quick Start snippet imports Sapiens2 from a single self-contained module, sapiens/backbones/standalone/sapiens2.py, and the project states that torch plus safetensors for checkpoint loading is all that is required. The README calls this Zero-Dependency Usage and the point is worth taking seriously: no model registry, no config system, no inference framework.

That file can be fetched on its own with a single curl, which means you can vendor it into an existing project without taking the whole repository as a dependency:

curl -O https://raw.githubusercontent.com/facebookresearch/sapiens2/main/sapiens/backbones/standalone/sapiens2.py

For the previous generation, the README points to sapiens.py in the same directory instead. Anyone migrating from Sapiens v1 therefore has to change the module path.

The mechanism underneath is a plain PyTorch module that returns a feature tensor, and the README documents it as a dense backbone output. The tradeoff is that the standalone path gives you features and little else: task heads, the optional detector, and the training recipes live in the full package, so a project that only needs embeddings can stay tiny while a project that needs pose or matting has to install the repository.

A checkpoint tree that is really a task contract

Checkpoint handling is where the repository layout tells you the most, and the directory tree in the Getting Started section is effectively the contract between the pretraining stage and the downstream tasks. Files are grouped by task under a root directory that defaults to ~/sapiens2_host and can be pointed elsewhere with SAPIENS_CHECKPOINT_ROOT. The pretrain folder holds one file per backbone size, and a separate 4K file. Each task folder then holds task-specific weights for a subset of sizes.

The first half of the tree:

sapiens2_host/
├── pretrain/
│   ├── sapiens2_{0.1b,0.4b,0.8b,1b,5b}_pretrain.safetensors
│   └── sapiens2_1b_4k_pretrain.safetensors
├── pose/
│   └── sapiens2_{0.4b,0.8b,1b,5b}_pose.safetensors
├── seg/
│   └── sapiens2_{0.4b,0.8b,1b,5b}_seg.safetensors

The second half:

├── normal/
│   └── sapiens2_{0.4b,0.8b,1b,5b}_normal.safetensors
├── pointmap/
│   └── sapiens2_{0.4b,0.8b,1b,5b}_pointmap.safetensors
├── matting/
│   └── sapiens2_1b_matting.safetensors
└── detector/                  # [optional] only needed for pose inference
    └── detr-resnet-101-dc5/

Read that carefully before you plan. The 0.1B backbone ships a pretrain checkpoint but no task checkpoint for any of the five tasks. Matting exists only at 1B, which matches the May 15, 2026 news entry announcing the Sapiens2-1B human matting model. Pose inference has an extra dependency the other tasks do not: a DETR ResNet-101 dc5 detector directory marked optional in the tree but required when you run pose inference.

Cloning, installing and running the first forward pass

The installation path is a standard editable install into a Python 3.12 or newer environment. The repository is cloned, SAPIENS_ROOT is exported so that scripts can find assets, and the package installs itself from pyproject.toml, where the project name is sapiens and the version is 2.0.0:

git clone https://github.com/facebookresearch/sapiens2.git
cd sapiens2
export SAPIENS_ROOT=$(pwd)

The install itself is one command:

pip install -e .

The pyproject.toml file requires Python 3.12 or above and PyTorch 2.7 or above, and it pulls in a fairly ordinary list of scientific Python dependencies including torchvision, numpy, opencv-python, pillow, safetensors, scipy, matplotlib, iopath, accelerate, rich, transformers and timm. Pointmap work is the one optional extra, declared as an optional dependency named pointmap that installs open3d.

Once a checkpoint sits in the tree, the first real use is a forward pass. The snippet below builds the 1B model at the native training resolution, loads the safetensors checkpoint, and runs a single random image through it:

import os
import torch
from safetensors.torch import load_file
from sapiens.backbones.standalone.sapiens2 import Sapiens2

# Build the model and load a pretrained checkpoint
model = Sapiens2(arch="sapiens2_1b", img_size=(1024, 768), patch_size=16).eval().cuda()  # img_size is (H, W)
ckpt = os.path.expanduser("~/sapiens2_host/pretrain/sapiens2_1b_pretrain.safetensors")
model.load_state_dict(load_file(ckpt))

# Forward pass on a single image (RGB; ImageNet normalization recommended)
x = torch.randn(1, 3, 1024, 768).cuda()
with torch.no_grad():
    features = model(x)[0]  # dense backbone features

Three details in that snippet are load bearing. The img_size argument is height by width, so 1024 by 768 is taller than wide. The call is placed on a CUDA device, so the code assumes a GPU. And the tensor is random with a comment recommending ImageNet normalization, which is a reminder that no preprocessing helper is bundled here.

What the documentation leaves unsaid

Documentation coverage is uneven, and the gaps matter for anyone planning an adoption. The README links to a per-task inference document and a per-task training document for all five tasks, plus MODEL_ZOO.md as the place to download checkpoints, and it points at a project page, a paper PDF and a HuggingFace Hub collection. Demos exist for pose, segmentation, normals, pointmap and matting as HuggingFace Spaces.

What the README does not document is the operational side. There is no rollback procedure for a checkpoint download that turns out to be the wrong model, no statement of which input resolutions the 4K tokenizer variant accepts at inference, and no note about how a task head is selected when you load a pose checkpoint against a seg backbone. The stand-alone file is described as self-contained for the backbone, so the task heads have to come from somewhere else, and the answer is the docs/ tree rather than a short inline example.

The second gap is the release history. The project has no GitHub releases at all, and the news section in the README is the only version record, with two dated entries: an initial Sapiens2 release on April 24, 2026 covering pose, body-part segmentation, surface normals and pointmaps, and the matting model on May 15, 2026. There is no changelog, no migration guide from Sapiens v1, and no published checkpoint format version. For a research group that pins dependencies, the absence of tagged releases means you have to track the main branch and the MODEL_ZOO.md list yourself.

Licence and maintenance, taken one at a time

The licence is the first thing to resolve, and it is not a standard permissive one. The repository is published under a Sapiens2 License in LICENSE.md, and the packaging metadata lists it under the Other and Proprietary category rather than MIT, Apache-2.0 or the BSD family. The README calls out contributions from DINOv3, OpenMMLab and Accelerate, which signals that the code borrows from projects with their own terms. This is not legal advice, and nothing here should be treated as a clearance to ship, but the practical point is narrow: the licence text has to be read before any commercial or hosted use, because a custom research licence may carry conditions that a permissive licence would not.

On maintenance, the repository is not archived, and the last push was on 2026-05-24. The project is therefore not frozen, but the window between that push and the two dated news entries is short and the total history is young, so treating this as a stable platform with predictable release cadence would be reading more into the record than it holds. A single contributor cannot assume that API details in sapiens/backbones/standalone will stay fixed across a branch update, which is an argument for copying the module into your own tree and pinning it, rather than tracking main.

Weight sizes are the other upgrade cost. The 1B backbone is 1.462 B parameters and the 5B variant is 5.071 B, so downloading a set of task checkpoints for several sizes is a bandwidth and disk decision before it is an engineering one. The 4K variant at 1.607 B parameters adds the cost of 4096x3072 inputs, and the 5B model is listed at 15.722 T FLOPs per forward pass, which is a very different service bill from the 0.342 T of the 0.1B entry.

Where a different approach would suit you better

The real alternative here is not another human-centric transformer, it is a different class of solution: a general purpose backbone that you fine-tune for your specific body task, or a purpose built perception stack tuned for one downstream job. Sapiens2 optimises for breadth across five human tasks from one pretraining corpus, which is why the same feature tensor can feed pose, segmentation, normals, pointmaps and matting. A specialist system gives you fewer moving parts and usually a smaller deployment target.

The second alternative is a smaller footprint. The model card spans more than an order of magnitude in cost, from 0.114 B parameters and 0.342 T FLOPs to 5.071 B and 15.722 T, and the documentation for a task sits in a per-task document, so a team that needs one output on one device can pick the smallest size that carries that task checkpoint. The catch documented in the tree is that the smallest size has no task weights at all, so the floor in practice is 0.4B.

The third alternative is a stable, versioned artefact. Projects that publish tagged releases and a changelog let a team pin a known good checkpoint and upgrade on a schedule. Sapiens2 has no GitHub releases, so the equivalent of a version pin is a commit and a local copy of the checkpoint files, and someone in your team has to own that.

Editorial conclusion

Sapiens2 fits research teams that need per-pixel human understanding and can afford 1024x768 inference and a large checkpoint download, and it is a poor match for teams needing a permissively licensed model, a small footprint, or a stable versioned release stream. Before committing, read LICENSE.md end to end, confirm the 4.715 T FLOP cost of the 1B backbone fits your budget, and check that docs/MODEL_ZOO.md has the task checkpoint you actually need.

Frequently asked questions

Official sources

1. facebookresearch/sapiens2 on GitHub
2. Issues
3. README