Model or dataset
tonywu71/colpali-cookbooks avatar
tonywu71/colpali-cookbooks

colpali-cookbooks: a wheel that ships a notebooks folder, and a tag from last year

Recipes for learning, fine-tuning, and adapting ColPali to your multimodal RAG use cases. 👨🏻‍🍳

357 stars29 forksUnknownMIT

At a glance

What is it?
colpali-cookbooks is six Jupyter notebooks for the ColVision family, covering transformers-native inference, similarity maps and LoRA fine-tuning, and its packaging quirks say more about how it is meant to be used than its documentation does.
Who is it for?
colpali-cookbooks fits someone who wants to watch ColPali or ColQwen2 actually run rather than install a library, and it fits badly as a dependency to build on: the newest tag is v0.5.0 from 2 June 2025 while the branch was pushed on 3 September 2026, the version number comes from the VCS rather than from the file, and the engine dependency is capped below 0.4.0. Check three things first.
Can I use it commercially?
Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 73 days ago.
What is it written in?
GitHub does not report a main language for this repository.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • tonywu71/colpali-cookbooks357 ★ · +0
Star history
Repository2026-09-112026-11-15
tonywu71/colpali-cookbooks357357

0 Stars in the last 65 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

The wheel ships the examples directory rather than a library

The build configuration is the first surprise. The project is called colpali_cookbooks, its version is declared dynamic and read from version control through hatch-vcs, and the wheel target is configured with a single include entry: `examples`. So the artifact that would be published contains the notebook folder and nothing resembling a library surface. That is a reasonable choice for a repository whose entire content is teaching material, and it does mean `pip install` gives you files to read and open rather than a module to import. The interpreter floor is set at Python 3.9 or newer in the project metadata, while the repository also carries a `.python-version` file at the root, which is how the exact interpreter for development is pinned separately from what the package claims to support.

There is no install command anywhere, only a Colab button

The instructions section is two paragraphs and neither contains a command. The stated easiest path is to open a notebook from the `examples` directory and click the Colab button underneath it, which opens the notebook in Google Colab where the code can be run and the models experimented with. The alternative is to clone the repository and open the notebooks in Jupyter Notebook or in an IDE. That is the whole setup story, which makes sense for a repository whose value is in reading and editing cells, and it also means the pyproject file is doing quiet work: it exists so the dependency set can be resolved somewhere, not because the documented flow uses it. Nothing in the visible documentation shows a pip or uv install line, so anyone who wants a local environment has to derive it from the metadata rather than copy it.

The engine dependency is capped inside one minor version

Four runtime dependencies carry the load, and the important one is `colpali-engine` with the interpretability and train extras, requested as 0.3.2 or newer and below 0.4.0. Two consequences follow. The extras are not optional in practice: an inference-only user still pulls the training and interpretability paths, which is the price of a single declared extra list rather than per-notebook requirements. And the ceiling means an upstream 0.4 release cannot arrive without editing the pin. Alongside it sit datasets held between 3.0.1 and 4.0.0, huggingface_hub, and ipykernel, the last one there so the notebooks have a kernel to run in. Tooling is configured rather than improvised: ruff runs with a line length of 120 and the E, F, W, I and N rule sets, with unused imports ignored only in `__init__.py`.

A skypilot directory and an extra the README never mentions

Two things exist in the packaging that the notebooks section does not account for. The first is a `skypilot` optional dependency pinned tightly, as `skypilot==0.6.1,<1.0.0`, together with a `skypilot/` directory at the repository root. The second is `nvitop`, a GPU monitoring tool, sitting in the development extra alongside pytest and ruff. Nothing visible in the documentation describes what the SkyPilot directory contains or when a reader would need it, and the configuration group named after it exists purely as an install switch. The pattern repeats elsewhere too: the top level also holds an `assets/` directory and a `CHANGELOG.md`, while the examples directory carries a `data/` folder that no notebook in the table of contents references by name.

The newest notebooks are ColQwen2, and the ColPali ones stay beside them

Six notebooks are listed from most recent to oldest, and the split between the two models in the ColVision family is visible in the ordering. The two newest are inference and interpretability notebooks for the transformers-native implementations, one for ColQwen2 and one for ColPali, each covering inference, scoring and interpretability through the plain transformers stack rather than a project-specific wrapper. Two more generate similarity maps, again one per model, so a prediction can be inspected rather than merely scored. The oldest is the fine-tuning notebook, which uses LoRA with optional 4bit or 8bit quantization, which is the only notebook in the set that trains rather than infers. The remaining notebook is the end-to-end RAG example described separately. Both model families are therefore kept in parallel, so a reader working on ColPali is not pushed onto ColQwen2.

One notebook swaps adapters so a free T4 is enough

The RAG notebook is the one with a stated budget constraint, and its description is blunt about why: by using a single vision language model for the entire pipeline through adapter hot-swapping, VRAM is saved, and it works even on Colab's free T4 GPU. That is a different exercise from the inference notebooks. Instead of loading one model and scoring documents, the pipeline needs a model for indexing and a model for answering, and the technique described is to keep one set of weights resident and swap adapters between stages rather than holding two models in memory. The entry in the table is labelled RAG rather than inference or interpretability, which matches that: the deliverable is a retrieval pipeline that fits a free tier, not a benchmark.

The newest tag predates the last push by more than a year

Three releases are visible and the newest is old. v0.3.1 was tagged on 17 November 2024, v0.4.0, described as the transformers-native ColPali cookbook, on 17 December 2024, and v0.5.0, the transformers-native ColQwen2 cookbook, on 2 June 2025. The branch itself was pushed on 3 September 2026, so there is well over a year of commits after the most recent tag, and the two transformers-native notebooks that head the table of contents have no release of their own attached to them in this list. Version numbers are also not the authority here: the project metadata declares its version dynamic and sourced from version control, so what a given install reports comes from the commit it was built from rather than from a number anyone edited.

Late interaction over page images is the idea the notebooks teach

The notebooks only make sense against the model they demonstrate. ColPali is a retrieval model that works on visual features instead of extracted text: each page is treated as an image, and a Paligemma-3B backbone captures not just the words but the layout, tables and charts, producing multi-vector embeddings per page. Retrieval then scores documents by computing pairwise late interaction similarity rather than collapsing a page into one vector, which is why layout can matter to a match. The stated advantage over a pipeline built on text extraction and OCR is that nothing has to be parsed before the model sees it. The repository positions itself as recipes for the ColVision family rather than one model, pointing at the ColPali engine, the ViDoRe benchmark, the Hugging Face vidore organisation and the arXiv paper, and it credits the paper's authors for the citation block at the end.

Editorial conclusion

colpali-cookbooks fits someone who wants to watch ColPali or ColQwen2 actually run rather than install a library, and it fits badly as a dependency to build on: the newest tag is v0.5.0 from 2 June 2025 while the branch was pushed on 3 September 2026, the version number comes from the VCS rather than from the file, and the engine dependency is capped below 0.4.0. Check three things first. The wheel payload is the examples directory, so installing the distribution gives you a notebook collection rather than an importable API. The engine dependency requests the interpretability and train extras, which brings training and interpretability code into an environment that only wanted inference. And a skypilot directory sits in the tree with a matching optional extra that the visible documentation never explains, so if you were expecting the provisioning configs to be described, they are not.

Frequently asked questions

What is ColPali?

ColPali is a document retrieval model that reads pages as images instead of extracting text from them. It is built on Paligemma-3B and produces multi-vector embeddings that capture text, layout, tables and charts, scoring documents with pairwise late interaction similarity so that no OCR step sits in front of the model.

What does colpali-cookbooks actually contain?

Six Jupyter notebooks under examples/: transformers-native inference and scoring for ColPali and ColQwen2, similarity map generation for both models, an end-to-end RAG notebook that hot-swaps adapters, and a ColPali fine-tuning notebook using LoRA with optional 4bit and 8bit quantization. The examples directory also holds a data folder.

How do I run the colpali-cookbooks notebooks?

The documented route is to open a notebook from the examples directory with its Colab button, which opens it in Google Colab. Locally, clone the repository and open the notebooks in Jupyter Notebook or in an IDE. No pip install line appears in the documentation, so a local environment has to be derived from the project metadata.

What does the colpali-cookbooks package depend on?

colpali-engine with the interpretability and train extras, requested as 0.3.2 or newer and below 0.4.0, plus datasets, huggingface_hub and ipykernel. Python 3.9 or newer is required, and the optional extras add development tooling including nvitop, and a separate pinned skypilot dependency.

Official sources

  1. License: MIT
  2. Project website
  3. README
  4. Releases
  5. tonywu71/colpali-cookbooks on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/tonywu71-colpali-cookbooks.svg)](https://hysenlabs.com/projects/tonywu71-colpali-cookbooks)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/tonywu71-colpali-cookbooks.svg)](https://hysenlabs.com/projects/tonywu71-colpali-cookbooks)

Was this useful?