MobileCLIP trades architecture for speed, and MobileOne reparameterization comes with it

Two papers, one repository, two generations of models

This is the official code for MobileCLIP (CVPR 2024) and MobileCLIP2 (TMLR, August 2025, listed as Featured). Both cover the same ground: CLIP style image-text models trained with what the authors call multi-modal reinforced training, on datasets called DataCompDR for the first generation and DFNDR for the second.

The repository layout follows that split. The mobileclip/ directory holds the first generation package, mobileclip2/ the second, training/ contains the training code, eval/ the evaluation harness, results/ stored outputs, and ios_app/ a demonstration app for real-time zero-shot classification on device. A separate repository, ml-mobileclip-dr, holds the code for generating the reinforced datasets at scale, so this repository is the model side only.

What the models are for is stated concretely in the README's highlight list. MobileCLIP-S0 is reported to reach similar zero-shot performance to OpenAI's ViT-B/16 while being 4.8x faster and 2.8x smaller. MobileCLIP-S2 is claimed better than SigLIP's ViT-B/16 on average zero-shot performance while being 2.3x faster and 2.1x smaller and trained on 3x fewer seen samples. MobileCLIP-B (LT) is reported at 77.2% zero-shot ImageNet. MobileCLIP2-S4 is said to match SigLIP-SO400M/14 accuracy with half the parameters and to surpass DFN ViT-L/14 at 2.5x lower latency on iPhone 12 Pro Max. Those are the authors' own numbers, reproduced here as claims rather than as measurements anyone has independently confirmed.

Setting up a Python 3.10 environment and installing in place

Setup is an editable install inside a conda environment, not a published package:

conda create -n clipenv python=3.10
conda activate clipenv
pip install -e .

The dependencies come from requirements.txt and are current: torch 2.8.0 or newer, torchvision 0.23.0 or newer, open-clip-torch 2.20.0 or newer, timm 0.9.5 or newer, datasets 2.21.0 or newer and clip-benchmark 1.4.0 or newer. Note that setup.py is stale by comparison: it declares version 0.1.0 and Python 3.7 through 3.10 classifiers with a Development Status :: 3 - Alpha marker, while the README's environment pins 3.10 and the requirement file demands a far newer torch. Treat setup.py as packaging scaffolding, not as the source of truth for versions.

Checkpoints are fetched from Hugging Face rather than from the repository. get_pretrained_models.sh exists for the first generation and downloads into a checkpoints directory:

source get_pretrained_models.sh   # Files will be downloaded to `checkpoints` directory.

The second generation is distributed through hf download against the apple organisation on the Hub, one repository per size: MobileCLIP2-S0, S2, B, S3, L-14 and S4, with the first generation spanning S0, S1, S2, B, B-LT, S3, L-14 and S4.

The OpenCLIP route, which the README now leads with

The README's primary path runs the models through OpenCLIP rather than importing mobileclip directly. That requires cloning OpenCLIP and applying a patch that lives in this repository:

git clone https://github.com/mlfoundations/open_clip.git
pushd open_clip
git apply ../mobileclip2/open_clip_inference_only.patch
cp -r ../mobileclip2/* ./src/open_clip/
pip install -e .
popd

pip install git+https://github.com/huggingface/pytorch-image-models

The patch is named inference_only, which is the first thing to notice: it is written for loading, not for training through OpenCLIP. Separate scripts for training with OpenCLIP live in training/, added in June 2024, so the two use cases are deliberately kept apart.

Nine model/weight combinations are documented as directly available by passing a pretrained argument and omitting the image mean and standard deviation values, including MobileCLIP-B with pretrained='datacompdr' and pretrained='datacompdr_lt', and the MobileCLIP2 family with pretrained='dfndr2b'. For the others, the inference snippet passes explicit image_mean and image_std of (0, 0, 0) and (1, 1, 1) as a model kwarg, so the preprocessing is not uniform across the checkpoint family and a wrong choice here silently changes accuracy.

Two steps in the inference snippet that people skip

The example encodes an image and three text prompts, normalises both feature tensors, and takes a softmax of the scaled dot product:

import torch
import open_clip
from PIL import Image
from mobileclip.modules.common.mobileone import reparameterize_model

model_name = "MobileCLIP2-S0"
model_path = "/path/to/mobileclip2_s0.pt"
model_kwargs = {}
if not (model_name == "MobileCLIP2-S3" or model_name == "MobileCLIP2-S4" or model_name.endswith("L-14")):
    model_kwargs = {"image_mean": (0, 0, 0), "image_std": (1, 1, 1)}

model, _, preprocess = open_clip.create_model_and_transforms(model_name, pretrained=model_path, **model_kwargs)
tokenizer = open_clip.get_tokenizer(model_name)

The first trap is eval mode. The comment in the snippet attributes it to batchnorm layers unlike ViTs, which is the architectural fingerprint of the MobileOne blocks these models inherit. Calling encode_image without model.eval() leaves those layers in training behaviour, and the result will not match the published numbers.

The second is reparameterize_model, imported from mobileclip.modules.common.mobileone. The README states that for inference and model exporting purposes you should reparameterize first. This is a structural transformation that folds the multi branch training time architecture into a single inference time one. The order matters: reparameterize after eval, and before any export to Core ML or ONNX, or you ship a model with training time branches still in it. It also means the Python package under mobileclip/ is required at inference time even when you load weights through OpenCLIP.

Wrong tool for a general purpose CLIP deployment

There are three boundaries worth stating plainly. First, this is research code, and setup.py says so with an alpha status classifier; there is no versioned release, the repository has no GitHub releases, and the CHANGELOG equivalent is absent. Pin your torch version and expect to read source when something fails.

The entire argument for MobileCLIP is efficiency at the small end, and the phone measured in the README is an iPhone 12 Pro Max.

Third, if you want to reproduce the training data generation itself, this is the wrong repository. The README redirects that to ml-mobileclip-dr. What lives here is training on data others generated.

The alternative approach is OpenCLIP unmodified.

Three licence files, and a last push in September 2026

The repository ships LICENSE, LICENSE_DATA and LICENSE_MODELS as separate files, and the project is marked as a custom licence rather than a standard one. A team intending to ship a model or to retrain on the accompanying datasets has to read all three; none of them is a stock OSI identifier you can assume.

Maintenance is modest rather than absent. The last push was on 2026-09-11, and the README's update log records the MobileCLIP2 model release and the second generation training code on 2025-08-28, the TMLR acceptance on 2025-07-22, the iOS demo app on 2024-11-22, and the OpenCLIP training scripts plus Hugging Face availability of models and the DataCompDR datasets on 2024-06-13. Recent activity is therefore small commits rather than frequent releases, and the absence of tagged versions means upgrading means tracking the main branch.

Editorial conclusion

MobileCLIP fits teams that need zero-shot image-text retrieval under a phone latency budget and are willing to follow the OpenCLIP patch route, and it fits research work on multi-modal reinforced training through training/ and the sibling ml-mobileclip-dr repository. It does not fit a pip install and go: setup.py still declares version 0.1.0 with a Development Status :: 3 - Alpha classifier, and the LICENSE, LICENSE_DATA and LICENSE_MODELS are three separate files you must read before shipping anything. Verify first that reparameterize_model runs before export on your target device, since the README makes that a prerequisite rather than an optimization.

Frequently asked questions

Official sources

1. apple-aiml-research/ml-mobileclip on GitHub
2. Issues
3. README