EUPE distils one vision encoder to replace several specialists

One encoder for classification, segmentation and vision-language work

The problem EUPE addresses is that a vision backbone is usually specialised. A classification encoder, a segmentation encoder and a vision-language encoder are three sets of weights, three sets of fine-tuning work and three sets of deployment costs, even when they share an architecture. The project description calls EUPE a single on-device vision encoder with versatile representations that match or exceed specialised experts across multiple task domains, and the README expands the family as versatile efficient vision encoders producing high-quality features and achieving outstanding performance on various vision tasks including image understanding, dense prediction and vision-language modeling. The mechanism behind it is a distillation recipe, and the README's teaser caption says that applying that recipe to ViT-B gives a well-balanced universal encoder that does better than both ViT-B domain experts and existing agglomerative ViT-Bs. The intended user is a research group, not a product team, and the licence reflects that.

Six checkpoints across two architectures, all pretrained on web images

The weights come in two architecture families and three sizes each, all pretrained on the LVD-1689M web dataset. The ViT line is ViT-T/16 with 6M parameters, ViT-S/16 with 21M and ViT-B/16 with 86M. The ConvNeXt line is ConvNeXt Tiny at 29M, Small at 50M and Base at 89M. Every one is hosted on Hugging Face under the facebook organisation, and the README instructs you to follow the link to access the weights, then point torch.hub.load() at the local files through the weights parameter. That two-step arrangement, download the checkpoint yourself and then pass its path, is how the README sets things up.

Loading a backbone through torch.hub with a local checkpoint

The loading API is PyTorch Hub rather than a bespoke entry point, and the README's snippet is the whole contract. You set a local directory, then call torch.hub.load with the entry point name, source='local', and a weights argument that takes a checkpoint, a URL or a path:

REPO_DIR = <PATH/TO/A/LOCAL/DIRECTORY/WHERE/THE/EUPE/REPO/WAS/CLONED>

# EUPE ViT models pretrained on web images
eupe_vitt16 = torch.hub.load(REPO_DIR, 'eupe_vitt16', source='local', weights=<CHECKPOINT/URL/OR/PATH>)
eupe_vits16 = torch.hub.load(REPO_DIR, 'eupe_vits16', source='local', weights=<CHECKPOINT/URL/OR/PATH>)
eupe_vitb16 = torch.hub.load(REPO_DIR, 'eupe_vitb16', source='local', weights=<CHECKPOINT/URL/OR/PATH>)

PyTorch itself is the only required dependency for loading the model, and the README strongly recommends installing it with CUDA support. The entry point names follow a pattern, eupe_vitt16, eupe_vits16, eupe_vitb16 and the three convnext variants, and the repository contains a hubconf.py at the root, which is where those entry points are registered. The setup.py declares the package name as eupe with a Python floor of 3.11, and the version is read out of the package's __init__.py rather than hard-coded.

The standard ImageNet transform, written out in full

Inputs go through the standard ImageNet evaluation transform, and the README spells it out rather than pointing at a helper, because the preprocessing has to match what the weights were trained with. The transform resizes to a square with antialiasing, converts to float with scaling, and normalises with the usual ImageNet channel means and standard deviations:

def make_transform(resize_size: int = 256):
    to_tensor = v2.ToImage()
    resize = v2.Resize((resize_size, resize_size), antialias=True)
    to_float = v2.ToDtype(torch.float32, scale=True)
    normalize = v2.Normalize(
        mean=(0.485, 0.456, 0.406),
        std=(0.229, 0.224, 0.225),
    )
    return v2.Compose([to_tensor, resize, to_float, normalize])

The resize size defaults to 256 in the function signature, and the composable pieces come from the v2 transform API in torchvision, with v2.ToImage, v2.Resize, v2.ToDtype and v2.Normalize. The README notes that the training and evaluation code requires PyTorch 2.7.1 or newer plus a few other third party packages, and that the code has only been tested with the specified versions and also expects a Linux environment.

Training and evaluation run from a conda environment definition

There is no pip install path in the README. The recommended route is micromamba, where you create and activate an environment named eupe from the provided environment definition, and the repository ships a conda.yaml for exactly that:

micromamba env create -f conda.yaml
micromamba activate eupe

The training requirements themselves are in requirements.txt, and they are short: ftfy, omegaconf, regex, scikit-learn, submitit, termcolor, torch, torchmetrics and torchvision. The setup.py parses that file at build time, and it has a little logic to pull out any --extra-index-url lines separately, which suggests the dependency set points at more than one index.

Data preparation is manual for ADE20K and NYUv2

The datasets are not downloaded for you. For ADE20K the README tells you to set a dataset root, create a directory for it, and fetch the ADEChallengeData2016 archive with wget before unzipping it, and then it gives the exact directory layout the loader expects, with training images and their annotation masks in parallel images and annotations subtrees, and the same for validation. Two details matter for evaluation. The annotation masks contain labels from 0 to 150, where 0 refers to other objects, and those pixels are not considered in the evaluation. And a file called objectInfo150.txt holds the information about the 150 semantic categories, including indices, pixel ratios and names. For NYUv2 depth the same pattern applies, with a dataset root and a directory for the NYU data, and the README describes using the NYU subset.

A noncommercial licence is the real boundary of this project

This is the fact that decides whether the project is usable for you, and it is in the source rather than in a badge. The setup.py header says the software may be used and distributed in accordance with the terms of the FAIR Noncommercial Research License, and the licence field in the repository metadata is left as a custom licence that GitHub cannot classify, with a LICENSE.md file at the root. That is not an open source licence, and the name says what it means: noncommercial research use. The code itself is a reference implementation with a hubconf.py, a eupe/ package, a setup.py, a pyproject.toml and a conda.yaml, and the tooling configuration is modern, with mypy and ruff configured for Python 3.11 and pylint enabled only for miscellaneous and similarities checks. A team that intends to ship a product using these weights has a different question to answer from the one this repository is set up to answer.

Editorial conclusion

EUPE suits a research team that needs one set of pretrained weights to cover classification, dense prediction and vision-language tasks, and that wants to compare its own distillation recipes against a published one. It does not suit a product team, because the weights and the code are covered by the FAIR Noncommercial Research License, which rules out commercial use regardless of how the code is integrated. Before adopting, confirm your use is noncommercial, pick the backbone size that matches the device, and note that the last push to the repository was on 2026-04-14 with no GitHub releases published.

Frequently asked questions

Official sources

1. facebookresearch/EUPE on GitHub
2. Issues
3. README