clip-retrieval's client defaults to filtering unsafe images, and its wheel ships the web UI
Easily compute clip embeddings and build a clip retrieval system with them
At a glance
- What is it?
- A seven-part toolkit for CLIP embeddings whose Python install carries 28 dependencies, four of them unbounded, whose packaging walks a compiled front-end directory into the wheel, and whose newest release predates its last commit by seven months.
- Who is it for?
- clip-retrieval fits a team that already has images with captions and wants semantic search over them without writing the index layer, because the pieces from download through inference, index, filter, backend and UI are already wired together and end2end runs the lot in two commands. Three things to check before you build on it.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Activity is slowing. The repository last received commits 7 months ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- rom1504/clip-retrieval2,801 ★ · +3
| Repository | 2026-09-11 | 2026-11-15 |
|---|---|---|
| rom1504/clip-retrieval | 2.8k | 2.8k |
3 Stars in the last 65 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
The client filters unsafe images unless you switch that off
The remote client takes nine parameters at construction, and four of them decide what comes back from a query. use_safety_model and use_violence_detector both default to true, so unsafe images and images containing violence are removed from results by default rather than flagged afterwards. deduplicate defaults to true as well, collapsing results that share an image embedding. num_images defaults to 40.
Three more shape ranking. aesthetic_score defaults to 9, the score from LAION's aesthetic predictor, and aesthetic_weight defaults to 0.5, so half the ranking weight comes from a quality model rather than from CLIP similarity. modality defaults to Multimodal.IMAGE, with Multimodal.TEXT as the other option, and use_mclip defaults to False, which is what keeps a multilingual CLIP model out of the query path.
Two parameters are required and neither is named index_name. The client takes indice_name, with the spelling the library uses throughout, so that string appears in your code as written rather than as corrected.
The Python wheel ships the compiled front end
One install gives you both halves of the system. The console entry point is a single script named clip-retrieval, mapped to clip_retrieval.cli:main, and the packaging step walks a directory called front/build, collects every file inside it, and attaches the result as package data with include_package_data enabled.
So the compiled web UI travels inside the Python distribution rather than being fetched at runtime. A separate data_files entry puts README.md at the root of the install as well. The two badge links at the top of the page reflect the same split: one points at the PyPI project for the Python package, the other at an npm badge for a front package.
That arrangement has a practical consequence for deployment. The service side is Flask with flask_restful and flask_cors in the requirements, and the UI is static files baked into the wheel, so there is no separate node build step in the documented install, and no way to update the UI without reinstalling the package.
Twenty-eight dependencies, four without an upper bound
requirements.txt is where this project's real dependency surface shows. It lists 28 entries, and the weight is in the machine learning stack: torch, torchvision, clip-anytorch, open-clip-torch, all_clip, multilingual-clip, sentence-transformers, transformers, img2dataset, autofaiss, faiss-cpu, webdataset, h5py, pyarrow and pandas.
Most entries carry a ceiling, written as a range such as >=1.46.0,<2 for img2dataset or >=2.17.0,<3 for autofaiss. Four do not. transformers, fsspec, scipy at >=1.11.0 and urllib3 at >=1.26.0 are listed with no upper limit, while every neighbouring line is bounded, which means those four can move underneath a working install.
Two other choices are worth naming. faiss-cpu is a hard requirement rather than an extra, so the index runs on CPU by default, and the classifier records Development Status :: 4 - Beta. The keyword list inherited from the sibling project includes download, which describes img2dataset rather than this package.
The deployment tarball pins what the library leaves open
The Makefile has a build-pex target that produces a single-file executable, and it pins two packages by exact version that requirements.txt leaves open. The command builds a virtualenv with python3.12, installs pex, and passes numpy==1.26.4 and opencv-python-headless==4.11.0.86 on the command line while the rest of the tree comes from setup.py.
It also splits the output in two. One tarball, clip_retrieval_torch.tgz, contains the torch dependencies from inside the pex layout, and a second, clip_retrieval.tgz, contains everything else with the torch dependencies excluded. That is a workaround for the size of a torch install, and it means the shipped artifact is two files to move rather than one.
The lint target gives the same shape a third view. make lint runs mypy over clip_retrieval, then pylint over the same package, then black in check mode at a line length of 120, with mypy.ini, .pylintrc and pytest.ini sitting at the root beside the Makefile.
end2end runs the chain and opens localhost:1234
The end-to-end command is two lines. Download a parquet file of image URLs and captions, then hand it to the CLI with an output directory:
wget https://github.com/rom1504/img2dataset/raw/main/tests/test_files/test_1000.parquet
clip-retrieval end2end test_1000.parquet /tmp/my_outputThat single command runs img2dataset, inference, index, back and front in sequence, and the result is a search interface on http://localhost:1234. Two flags adjust it. --run_back False skips the backend when you only want the index, and exporting CUDA_VISIBLE_DEVICES with nothing after it keeps the run off the GPU when the card does not have enough VRAM.
The inference step has its own recipe for gathering inputs, which downloads images with img2dataset using a URL list, a thread count of 64 and an image size of 256, and then notes that you can put text files with the same names as the images in that folder to get text embeddings for the same set.
The throughput numbers are tied to one 3080
Two figures appear in the README and both name the same piece of consumer hardware. The headline is that 100M text and image embeddings can be processed in 20 hours using a 3080, and the inference component is described as computing image and text embeddings at 1500 samples per second on a 3080. No batch size, no precision setting and no method accompany either number.
The adoption list gives a sense of scale without benchmarks. cah-prepro preprocesses the 400M image and text crawling at home dataset and uses the library to compute 400M embeddings and their indices. Criteo's autofaiss uses clip-retrieval for a multimodal notebook example, an OpenAI demo searches the 1M examples released for the DALL-E demonstration, and antarctic-captions uses autofaiss together with clip inference to generate anchors for an image to text task.
Related projects fill in the parts this one does not cover: all_clip to load any CLIP model, img2dataset to download images from URLs, open_clip to train CLIP models and CLIP_benchmark to evaluate them.
The newest release is from 2025, the last commit from March
The dates in this repository need reading before the features. The last push to main is dated 2026-03-28, and the newest release is 2.45.0 published on 2025-08-15, which is the version string hard-coded in setup.py. So the packaged release and the source tree are about seven months apart, and anyone installing from a package index gets the older of the two.
Before that, the gap was longer. Releases 2.44.0 and 2.43.0 were both published on 2024-01-13, forty-eight minutes apart, and then nothing until 2.45.0 nineteen months later. HISTORY.md at the root carries the change log for all of it.
None of this says the project is finished. It says the cadence is irregular, which for a research library is a normal shape and for a dependency you intend to pin is a reason to choose a version on purpose rather than take the newest one.
Editorial conclusion
clip-retrieval fits a team that already has images with captions and wants semantic search over them without writing the index layer, because the pieces from download through inference, index, filter, backend and UI are already wired together and end2end runs the lot in two commands. Three things to check before you build on it. Check the dependency floor, since torch, faiss-cpu, autofaiss, sentence-transformers and open-clip all come along and four requirements carry no upper bound. Check the parameter names, because the required index argument is indice_name and your code will carry that spelling. And check the pin you install against, because the newest release is 2.45.0 from 2025-08-15 while the branch itself took its last commit on 2026-03-28, so the packaged release and the source tree are different things.
Frequently asked questions
What does image retrieval mean for clip-retrieval?
The repository implements it rather than defining it. clip-retrieval computes CLIP embeddings for images and text, builds indices out of them with autofaiss and faiss-cpu, and serves queries through a Flask backend with a small web front end, all from one pip install.
What does clip() do in clip-retrieval?
No function by that name appears in the documented API. The entry point is the ClipClient class, imported from clip_retrieval.clip_client along with Modality, and queried with client.query(text=...), client.query(image=...) or client.query(embedding_input=...).
What is embedding-based retrieval in this project?
It is the pipeline the seven components implement: img2dataset downloads images from URLs, clip inference computes image and text embeddings, clip index builds the indices, clip filter filters data through an index, clip back hosts the indices behind a Flask service and clip front is the UI that queries it.
What is a CLIP model and how does clip-retrieval use one?
The project treats the model as a given and depends on clip-anytorch, open-clip-torch, all_clip and multilingual-clip rather than implementing one. all_clip is the related project for loading any CLIP model, and multilingual-clip is what backs the use_mclip parameter, which defaults to False.
How do I install clip-retrieval?
With pip install clip-retrieval, which pulls the 28 entries in requirements.txt, including torch, torchvision, faiss-cpu, autofaiss, flask and sentence-transformers. The console command installed is clip-retrieval, mapped to clip_retrieval.cli:main in setup.py, and a separate doc covers running the laion5B index.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/rom1504-clip-retrieval)Star history chart for your README
[](https://hysenlabs.com/projects/rom1504-clip-retrieval)Was this useful?