SINQ quantizes LLMs without calibration by scaling rows and columns separately

Dual scaling is the whole idea

Conventional weight quantization uses one scale per weight dimension, and the project argues that this makes a model vulnerable to outliers, the large weights that distort the scaling and cause significant errors. SINQ's answer is dual scaling, that is, separate scale factors for rows and columns of the weight matrix. The claim in the documentation is that this flexibility redistributes outlier influence and keeps quantization errors smaller and more balanced. On top of that sits the Sinkhorn-normalized optimization, an iterative procedure that rescales rows and columns to balance their variance, inspired by Sinkhorn matrix normalization. The stated effect is a reduction in overall matrix imbalance, which makes the weights inherently easier to quantize and yields more stable behaviour across layers and consistently higher accuracy even at very low bit widths. The documentation goes as far as 3 bit precision, where single-scale errors would otherwise cluster around the outliers.

Calibration-free, and the calibrated A-SINQ variant

The feature comparison in the documentation positions SINQ against two calibration-free methods and two calibrated ones. SINQ is calibration-free, supports symmetric and asymmetric quantization, supports NF4, and is described as roughly twice as fast as HQQ with higher model quality, while HQQ is asymmetric only and has no NF4 support. A-SINQ is the calibrated sibling, also symmetric and asymmetric with NF4 support, and is described as roughly four times faster than AWQ, which is calibrated and lacks NF4 support. Two architectural claims frame the choice. SINQ is model-agnostic and works without knowing the specific LLM architecture, unlike QuaRot, and it is training-free, so it does not require end-to-end training unlike SpinQuant or KurTail. The A-SINQ branch is the one to look at when you have calibration data available: the documentation says it beats AWQ, GPTQ and Hadamard plus GPTQ on quality while quantizing more than four times faster.

The speed and memory figures are self-reported

The numbers in the documentation are the project's own, and they are worth quoting precisely so you can check them against your own setup rather than take them on trust. SINQ is said to quantize Qwen3-14B in about 21 seconds and DeepSeekV2.5-236B in about 5 minutes on a single GPU, and the more than 31 times faster claim is made against AWQ and GPTQ for the calibration-free path. The memory claim is the one most likely to matter to you: running DeepSeekV2.5-236B on a single GPU is said to need about 110 GB of memory instead of about 472 GB, while losing less than one point of perplexity on WikiText2 and C4. These are the authors' comparisons on named benchmarks, not an independent measurement, so treat them as a starting point for your own evaluation rather than a guarantee for your model and your hardware.

The Transformers integration is the practical entry point

SINQ was integrated into Hugging Face Transformers, and the project documents the news entry for 18/02/2026 as making the method usable through a configuration object called SinqConfig that is compatible with AutoModelForCausalLM(). If you already load models through the Transformers stack, that is the shortest path to trying it, and the project links the Transformers documentation page for SINQ rather than re-describing the API. The method was accepted and is to be presented at ICML 2026, with the poster linked, and the paper itself is on arXiv at 2509.22944, which the repository also lists as its homepage. So the documentation surface is split in a useful way: the paper for the mechanism and the error analysis, the Transformers page for the API, and this repository for the code and the model collection.

pre-SINQ is a separate path for GGUF files

GGUF, the format used by the local inference tooling, needs a different entry point, and the project handles that with a pre-SINQ algorithm described as SINQhorn normalization for GGUFs. The documentation explains the intent as combining the strengths of SINQhorn normalization with the advantages of GGUF quantization, and a news entry for 10/02/2026 records the first GGUF model using pre-SINQ, available in a PreSINQ GGUF collection under the huawei-csl organisation on Hugging Face. If your deployment target is a GGUF file for a local runner, pre-SINQ is the branch to follow and plain SINQ is the wrong door. The repository also carries a presinq_models_generation directory, which is where the artefacts for that path are produced, alongside a fused_kernels directory that suggests the quantised path ships custom kernels rather than relying only on reference implementations.

A source distribution with no tagged releases

The packaging is minimal. The repository root holds the sinq package directory, tests/, docs/, imgs/, a fused_kernels directory, presinq_models_generation, a requirements file named req.txt and setup.py. The setup script is short:

from setuptools import find_packages, setup

setup(
    name="sinq",
    version="0.1",
    description="SINQ quantization method",
    packages=find_packages()
)

The package name is sinq and the version field still reads 0.1, which tells you the distribution has not been versioned in step with the method's development. There are no GitHub releases either, so there is nothing to pin, and the last push to the repository was on 2026-05-08. The licence is Apache-2.0 with a LICENSE.txt file at the root, which permits modification and redistribution under its conditions, though that describes the licence rather than settling how its obligations apply to your own distribution. The visible documentation does not print an installation command, so treat setup.py and req.txt as the concrete entry points and check the quantization section of the documentation before assuming a command.

Where SINQ is the wrong tool

Three cases stand out. If you need GGUF output, plain SINQ does not get you there and the pre-SINQ path with SINQhorn normalization is the one to evaluate. If you are already running a calibrated method such as AWQ or GPTQ and have a calibration set that works for your model, the calibration-free argument mostly disappears, and A-SINQ is the branch worth comparing against your current numbers. If your deployment requires a versioned, installable artefact with release notes, the absence of GitHub releases and the 0.1 version in setup.py make this repository a source checkout rather than a dependency you can pin with confidence. Finally, note that the quality and speed claims are comparisons the authors publish themselves, on WikiText2 and C4, with the model and hardware they chose. A quantization method that reduces memory footprint substantially is worth measuring on your own weights before you plan a fleet migration around it.

Editorial conclusion

Adopt SINQ if you need to fit a large open weights model onto a single GPU without running a calibration set or an end to end training job, and if you can accept a method whose quality claims come from the authors' own comparisons. Do not adopt it if your pipeline already depends on GGUF files, since that path runs through pre-SINQ and the SINQhorn normalization, or if you need a released, versioned package, because the repository publishes no GitHub releases and setup.py still carries version 0.1. Before you start, read the quantization section of the project documentation, confirm your GPU memory budget against the figures the README quotes, and check the last push date of 2026-05-08 to see how current the code is.

Frequently asked questions

Official sources

1. huawei-csl/SINQ on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README