PicQuery: natural language search over local Android photos with MobileCLIP2

What PicQuery actually solves for Android users

Android's built-in gallery search depends on metadata, filenames and whatever labels the system has already attached. It does not answer a query like a dog on a beach if nothing in the file says so. PicQuery builds its own index of the photos you select and matches a text description, in English or Chinese, against image embeddings produced on the device. You can also use a photo as the query, which makes it a similarity search rather than a keyword search. The intended user is someone with a large local gallery who wants retrieval without sending images to a server. The README states the app is free with no in-app purchases, and the repository is MIT licensed. It is not a cloud service and it does not claim to be one.

How the MobileCLIP2 index and two runtimes fit together

The branch described in the README integrates Apple's MobileCLIP2-S0 / dfndr2b through two Android product flavors, ONNX Runtime and TFLite / LiteRT. They can be installed side by side because they use different application IDs, and each keeps a separate index, so the same album can be indexed twice for comparison. Both flavors use FP32 image inference and dynamic INT8 text weights with normalized FP32 embeddings. The `onnx` flavor packages `mobileclip2_s0_image.onnx` and `mobileclip2_s0_text_int8.onnx`; the `tflite` flavor packages `image_model.tflite` and `text_model_dynamic_wi8.tflite`. Model binaries are Git-ignored, so a clone does not contain them. Threading differs: ONNX uses 4 image and 4 text CPU threads, while TFLite uses native XNNPACK with 4 image and 2 text threads. English queries pass through translation, and candidates that normalize to the same CLIP BPE text are encoded once. That last detail matters on a phone, where tokenization and encoding cost is real.

Building and installing a PicQuery debug APK

The README lists JDK 17, Android SDK platform 37 (`platforms;android-37.0`), command-line tools 22+ or current Android Studio, NDK 29.0.14206865, CMake 3.22.1, and a device on Android 10 / API 29 or newer. Set the SDK location in Android Studio or `local.properties` and use the included Gradle wrapper. Both flavors require NDK and CMake because of the native delegate bridge. Model binaries must be downloaded separately from the Google Drive folder linked in the README, verified against SHA-256 following the steps in `script/model-MobileCLIP2/downloads/README.md`, and placed in `app/src/main/assets/`. The README also notes that the shared `bpe_vocab_gz` and bundled `mlkit/` assets must be retained. Build both flavors with the wrapper:

./gradlew :app:assembleOnnxDebug :app:assembleTfliteDebug

On Windows, replace `./gradlew` with `.\gradlew.bat`. Then install each APK to a specific device, replacing `DEVICE_SERIAL` with the serial shown by `adb devices -l`:

adb -s DEVICE_SERIAL install -r app/build/outputs/apk/onnx/debug/app-onnx-debug.apk
adb -s DEVICE_SERIAL install -r app/build/outputs/apk/tflite/debug/app-tflite-debug.apk

Default APKs include ARM64 and ARMv7. For an x86_64 emulator, the README says to append `-Pmobileclip2Abis=x86_64` to the build command. You should end up with two launcher entries, PicQuery MC2 ONNX and PicQuery MC2 TFLite.

A first real search: index one album, then query it

The README's own walkthrough starts small. Create an album such as `Pictures/PicQuery-Demo` before opening the app, grant photo access, then use Index, Add album, and select that album. Check the photo count, tap Index, wait for completion, and tap Finish. Only selected albums are encoded; startup enumerates accessible media metadata without automatically indexing every album. If a newly added album is missing from the list, restart the app. After indexing, search for `dog` or `astronaut`, or supply an image as the query. To narrow an existing index, open Range, turn off All albums, pick one album and tap Finish. Note that the range resets when the app process restarts, and that indexes are managed or deleted in Settings, Album Index Manager. To compare backends, index the same album in the other flavor, since each flavor keeps its own index. The repository also documents how to run the unit and lint checks:

./gradlew :app:testOnnxDebugUnitTest :app:testTfliteDebugUnitTest \
  :app:ktlintCheck :app:lintOnnxDebug :app:lintTfliteDebug

What the measured results do and do not cover

The repository separates an image-quantization experiment from the flavor comparison, and the README is explicit that it does not compare the ONNX and TFLite flavors. The experiment compares FP32 image ORT against mixed INT8 image ORT, both with the same dynamic INT8 text model, on a Pixel 8a / Tensor G3 / Android 17 with ONNX Runtime 1.29.0 at 4 CPU threads. CIFAR-100 Top1 was 74.60 percent for FP32 image and 73.90 percent for mixed INT8; Imagenette Top1 was 98.04 percent and 97.96 percent. Image encoding P50 was 67.58 ms and 53.46 ms respectively, and the image ORT file shrank from 43.54 MiB to 13.52 MiB. Those are class-label benchmarks with fixed English prompts, not free-form gallery search. The README states the timing protocol excludes photo decoding, resizing, tokenization and database search, so the numbers are not end-to-end search latency. It also states that the current flavors retain FP32 image inference, so adopting a different image encoder would require rebuilding the photo index.

Where PicQuery stops being the right tool

The limitations section is unusually candid, and it should shape adoption. Recorded device validation used Debug APKs on a Pixel 8a with 4 KB memory pages. Release and R8 builds, GPU and NPU paths, other phones, and whole-APK 16 KB page compatibility remain unverified, and a compatibility notice appeared on the tested device. The README also states that class-label benchmarks and a small demo album do not establish quality for personal galleries, free-form descriptions, Chinese translation or OCR. So if your gallery is mostly screenshots, receipts or documents, this is not the tool for you. Host and phone results use different CPU kernels, and historical runs used different runtime versions. Anyone who needs a signed, maintained release rather than a source build should weigh that the newest published release is v1.1.4 from 2025-02-04, while the README says published releases may use earlier models than this branch.

PicQuery against the alternatives you already have

The obvious comparison is Android's own gallery search. That works on metadata and whatever labels the system exposes, requires no model download and no index build, and covers every photo automatically. PicQuery trades that convenience for semantic matching: you choose the albums, you wait for encoding, and you get text-to-image and image-to-image retrieval. The second alternative is a desktop or server-side CLIP pipeline, where you copy photos off the phone and index them on a machine, then search from a browser. That approach can use larger models and more compute, and it does not depend on the phone's runtime kernels, but it moves the images off the device and adds a sync step whenever the gallery changes. PicQuery's whole argument is that the index and the query stay on the phone. The cost of that argument is the build chain: NDK, CMake, model assets placed by hand, and two separate indexes if you want to compare runtimes.

Maintenance, licensing and upgrade cost

The repository is not archived, and the last push was on 2026-09-13. The published releases are older than the branch: v1.1.4 on 2025-02-04, v1.1.3 on 2024-11-30, and v1.1.2 on 2023-10-20. Upgrading is not only an APK swap. Model binaries are Git-ignored and must be re-downloaded and SHA-256 verified, and the README states that adopting a different image encoder requires rebuilding the photo index. Switching between the ONNX and TFLite flavors means indexing again in the other app, since the two keep separate indexes. On the licence side, the repository is MIT, which is permissive, but the model assets come from elsewhere and their own terms are not stated in the README; verify those separately before redistributing anything. The ktlint baseline at `app/config/ktlint/baseline.xml` records existing findings, and the README asks that it not be regenerated automatically to hide failures, which is a reasonable signal about how the maintainers want contributions handled.

Editorial conclusion

PicQuery is worth adopting if you want to search a local Android gallery by description or by example image without uploading anything, and you are comfortable building from source with JDK 17, NDK 29.0.14206865 and CMake 3.22.1. It is the wrong tool if you need a maintained release channel: the newest published release is v1.1.4 from 2025-02-04, and the README states that published releases may use earlier models than this branch. Before committing, verify that your phone is running Android 10 / API 29 or newer, that you can place the model files in app/src/main/assets/ after the SHA-256 check, and that the album you care about is indexed correctly on your own gallery, because the project's own class-label benchmarks and small demo album do not establish quality for personal photos or free-form descriptions.

Frequently asked questions

Official sources

1. greyovo/PicQuery on GitHub
2. License: MIT
3. Project website
4. README
5. Releases