# Q-Align, teaching a multimodal model to score images and video

> Q-Align is the Q-Future research code for a visual scoring model that turns a large multimodal model into a quality and aesthetics scorer by asking it in text-defined discrete levels. The published weights are OneAlign on Hugging Face, and the newer generation is Q-ReAlign, which the authors say reaches the same performance with half the parameters.

**Q-Future/Q-Align** — ③[ICML2024] [IQA, IAA, VQA] All-in-one Foundation Model for visual scoring. Can efficiently fine-tune to downstream datasets.

- Repository: https://github.com/Q-Future/Q-Align
- Website: https://q-align.github.io
- Stars: 629 · Forks: 35
- Language: Python
- License: NOASSERTION
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/q-future-q-align

## Scoring by asking a model in words

Q-Align is the research code behind a paper presented at ICML 2024 in Vienna, titled Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels. The project description calls it an all-in-one foundation model for visual scoring across image quality assessment, image aesthetics assessment and video quality assessment, with the ability to fine-tune efficiently to downstream datasets.

The mechanism is in the title. The model is not given a regression head and asked to output a number. It is taught to score through discrete text-defined levels, so a quality score is produced by asking the model a question whose answer space is a set of words rather than a continuous scale.

That design has a practical consequence for the user. The task is expressed as a `task_` argument with the values `quality` and `aesthetics`, and the input type is an `input_` argument with the values `image` and `video`. Those two arguments are the entire user-facing contract in the quick start, and they are what makes one model serve four jobs.

The project is from Q-Future, with authors at Nanyang Technological University, Shanghai Jiao Tong University and Sensetime Research, and the weights are published as OneAlign on Hugging Face under the q-future organisation. There is also a OneScorer space on Hugging Face linked from the top of the documentation.

Two things are worth knowing before you clone anything. First, the model is built on mPLUG-Owl2, a large multimodal model, so the checkpoint is large and the inference is a full forward pass of that model. Second, the repository has been superseded for new work: the documentation points to Q-ReAlign, a new version trained on modern vision-language models, in a separate repository, and the authors state that it gains the same performance with 50% fewer parameters.

There is a second line of work as well, Q-SiT and Q-SiT-mini, described as the first models that can deal with both image quality scoring and interpreting tasks, in its own repository.

## The pinned dependency set is the real constraint

The packaging metadata is where the practical difficulty of this project lives, and it is worth reading before anything else.

The project builds with setuptools and requires Python 3.8 or newer. The dependency list is pinned hard: torch at 2.0.1, torchvision at 0.15.2, transformers at 4.36.1, tokenizers at 0.15.0, sentencepiece at 0.1.99, accelerate at 0.21.0, peft at 0.4.0, bitsandbytes at 0.41.0, scikit-learn at 1.2.2, gradio at 3.35.2, einops at 0.6.1, timm at 0.6.13, httpx at 0.24.0, pydantic below 2 and at least 1. The optional training extra adds deepspeed at 0.9.5, ninja, wandb and flash_attn.

Exact pins on a stack this deep mean a conflict with almost any other project in the same environment. The project itself admits the problem in the documentation, with a section titled Important Note about v1.1 and transformers 4.36.1 or later. It explains that this LLaVA-style repository was built on transformers 4.31.0, which is incompatible with many new models available on Hugging Face, and that this requires building a separate environment for the MLLM repository, which it calls somewhat troublesome, specifically because the Q-Align and OneAlign model is expected to boost other disciplines such as image and video generation.

The fix they shipped was to modify the mPLUG-Owl2 code to adapt to transformers 4.36.1, so that you do not need to create a separate outdated environment while using it alongside other projects. The cost is stated in the same paragraph: the updated code is no longer compatible with the old Q-Align releases, v1.0.1 and v1.0.0 and before.

So there are two incompatible lines, and the upgrade is not optional.

```shell
git pull
pip install -e .
```

That pair of commands is the documented migration, and the version conflict is a real operational tax for anyone who wants this model in a shared environment.

## Scoring an image without installing anything

The fastest path to a score does not involve the repository at all, and the documentation calls it Installation Free.

The idea is to load the published checkpoint through the transformers AutoModel class, with trust_remote_code enabled so the custom scoring methods are available on the model object.

```python
import requests
import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("q-future/one-align", trust_remote_code=True, 
                                             torch_dtype=torch.float16, device_map="auto")

from PIL import Image
model.score([Image.open(requests.get("https://raw.githubusercontent.com/Q-Future/Q-Align/main/fig/singapore_flyer.jpg",
                                     stream=True).raw)], task_="quality", input_="image") # task_ : quality | aesthetics; # input_: image | video
```

The `score` method takes a list, so it handles a batch, and the two keyword arguments select the task and the media type. The float16 dtype with device_map set to auto is what makes it usable on a single consumer GPU, since the underlying model is a full multimodal model rather than a small scoring head.

There is a second integration path through IQA-PyTorch, the metrics library from one of the authors, which is described as supporting instantiation of this model via the latest version. The library exposes a `create_metric` call with a metric name, and you pass the task through the same `task_` argument.

```python
import pyiqa
import torch

qalign = pyiqa.create_metric('qalign').cuda()
quality_score = qalign(input, task_='quality')
aesthetic_score = qalign(input, task_='aesthetic')
```

For an engineer who already has an image quality pipeline built on IQA-PyTorch, this is the more interesting of the two routes, because it drops the model into an existing evaluation harness rather than asking you to write your own loop.

## The local install splits inference from training

When you do want the repository, the installation is split by what you are doing, and the split is explicit in the documentation.

If you only need to infer or evaluate, three commands.

```shell
git clone https://github.com/Q-Future/Q-Align.git
cd Q-Align
pip install -e .
```

For training, you add the extra and the attention kernel.

```shell
pip install -e ".[train]"
pip install flash_attn --no-build-isolation
```

The `--no-build-isolation` flag on flash_attn is there for a reason: it compiles against the local torch installation rather than pulling its own copy, which is the usual way to avoid a second torch in the environment.

The package name on PyPI is q_align, the version in the packaging metadata is 1.2.0, and the console entry point is not something you would call directly. The package discovery excludes assets, benchmark, docs, dist, playground, scripts and tests from the wheel.

The top-level tree shows the shape of the project. There is `q_align/` for the package itself, `model_zoo/` for the checkpoints, `lora_finetune/` for the fine-tuning path, `playground/` for the Gradio interface, `scripts/`, `fig/` for the images used in the documentation, and a `preprocessor_config.json` at the root, which is the image preprocessing specification that has to match the model's expectations.

The `model_zoo/` directory is the piece to look at first for an evaluation, because it holds the per-task checkpoints and the fine-tuned variants, and the Q-Align and OneAlign relationship means the base model and the task-specific fine-tunes are not the same artifact.

One documentation detail: the repository records that the multi-GPU inference problem has been fixed, which is worth knowing if you plan to run this on more than one device, since the repository ships a dedicated `preprocessor_config.json` and the CLI entry point expects the image path to arrive through an environment variable.

## The CLI and Python API for the three scorers

The quick start covers three scorers, each with a command line form and a Python form, and they differ in which flags they take.

The image quality scorer runs through `q_align/evaluate/scorer.py` with an image path passed in through an environment variable, which keeps the path out of the command history.

```shell
export DEFAULT_IMG_PATH=fig/singapore_flyer.jpg
python q_align/evaluate/scorer.py --img_path $DEFAULT_IMG_PATH
```

The aesthetic scorer adds an `--aesthetic` flag and takes a model path explicitly.

```shell
export DEFAULT_IMG_PATH=fig/singapore_flyer.jpg
python q_align/evaluate/scorer.py --img_path $DEFAULT_IMG_PATH --aesthetic --model-path q-future/one-align
```

The video scorer uses the same script with a video file, and the documentation example points it at `fig/baby.mp4`. The `--model-path` argument being available on the aesthetic form is the hint that different tasks can use different fine-tuned checkpoints rather than one model doing everything.

The Python API is the same logic without the shell. `QAlignScorer` and `QAlignAestheticScorer` are the two classes, and both take a list of images and return a tensor, which the samples print with `.tolist()`.

```python
from q_align import QAlignScorer
from PIL import Image

scorer = QAlignScorer()
img_list = [Image.open("fig/singapore_flyer.jpg")] # can be multiple images
print(scorer(img_list).tolist())
```

```python
from q_align import QAlignAestheticScorer
from PIL import Image

scorer = QAlignAestheticScorer()
img_list = [Image.open("fig/singapore_flyer.jpg"), Image.open("fig/boy_colorful.png")] # can be multiple images
print(scorer(img_list).tolist())
```

Batching through a list is the point of the API shape, since a scoring model that you call once per image wastes most of the GPU.

The paper and the project write-ups are linked from the documentation as a technical report PDF, and the repository also carries a `_config.yaml` and a `scripts/` directory, which together with the `playground/` Gradio app are the pieces you would look at to build an interactive demo rather than a batch job.

## What the licences actually cover, and where the project has moved

Two licensing facts sit in this repository and they are not the same thing.

The packaging metadata classifies the project under the Apache Software License, and the project directory listing shows a `LICENSE` file at the root and a separate `S-Lab-LICENSE` file beside it. The S-Lab-LICENSE is the laboratory licence, and it is the one that governs the model weights and the research artefacts rather than the code. Anyone shipping the OneAlign weights in a product needs to read that file rather than assume the Apache-2.0 classification in the metadata covers them. The repository's own licence field is not classified by GitHub, which is consistent with there being two documents rather than one.

This is not legal advice, and the point is simply to read both files before you redistribute anything.

The second fact is about where the project lives now. The recorded release is v1.1.0-happy-finetune-transformers-4.36.1, tagged on 2024-01-04, and the last push to the repository was on 2026-06-24. The repository is not archived, but it has been quiet for a while, and the more important signal is the two successor projects the documentation points to.

Q-ReAlign is the direct successor, described as trained on modern vision-language models and gaining the same performance with 50% fewer parameters, with a retrained version on Qwen3.5-VL in a repository of the same name. Q-SiT and Q-SiT-mini are a separate line, described as the first models able to handle both image quality scoring and interpreting tasks.

The practical reading is that Q-Align is the ICML 2024 artifact, useful if you want to reproduce that work or cite that checkpoint, while a new project built on a current vision-language model is the better starting point if you are starting today. The transformers 4.36.1 pin is the constraint that will make the decision for you, since a modern stack will not install alongside it.

## Conclusion

Q-Align suits a researcher or engineer who needs a quality or aesthetics score for images or video and wants a model rather than a hand-tuned metric, and it suits a team already working with transformers who can live with the pinned dependency set. It does not suit a production service that needs a stable, small, fast scorer, because the published path is a full multimodal model and the newer generation still lives in a separate repository. Verify first that the transformers version you need is compatible, since the project pins 4.36.1 for this line and has already broken compatibility once, and read the licence file directly, because the repository ships a separate S-Lab-LICENSE and the licensing terms for the weights are not the same as the code licence.

## FAQ

### What is Q-Align and what does it do?

It is the Q-Future research release for Q-Align, teaching large multimodal models for visual scoring via discrete text-defined levels, presented at ICML 2024. One model covers image quality assessment, image aesthetics assessment and video quality assessment, and it can be fine-tuned to downstream datasets.

### How do I score an image with Q-Align without installing the repository?

Load the published OneAlign checkpoint from Hugging Face with AutoModelForCausalLM.from_pretrained using trust_remote_code=True, then call model.score with a list of PIL images, task_="quality" or "aesthetics", and input_="image" or "video".

### What dependencies does Q-Align require?

The pinned set includes torch 2.0.1, torchvision 0.15.2, transformers 4.36.1, tokenizers 0.15.0, sentencepiece 0.1.99, accelerate 0.21.0, peft 0.4.0, bitsandbytes 0.41.0, scikit-learn 1.2.2 and gradio 3.35.2. Training additionally needs deepspeed 0.9.5, ninja, wandb and flash_attn.

### What is the difference between Q-Align and Q-ReAlign?

Q-ReAlign is the newer release the documentation points to, trained on modern vision-language models, and the project states that it gains the same performance with 50% fewer parameters. Q-Align itself is the ICML 2024 line built on mPLUG-Owl2 and pinned to transformers 4.36.1.

### How is Q-Align licensed?

The packaging metadata classifies the project under the Apache Software License, and the repository contains both a LICENSE file and a separate S-Lab-LICENSE file, the latter being the laboratory licence that covers the weights and research artefacts rather than the code.

## Sources

- [Issues](https://github.com/Q-Future/Q-Align/issues)
- [Project website](https://q-align.github.io)
- [Q-Future/Q-Align on GitHub](https://github.com/Q-Future/Q-Align)
- [README](https://github.com/Q-Future/Q-Align/blob/main/README.md)
- [Releases](https://github.com/Q-Future/Q-Align/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/q-future-q-align