# mct-model-optimization starts from a model you have already trained

> MCT is Sony Semiconductor Solutions' toolkit for quantizing and compressing neural networks, covering post training quantization, gradient based GPTQ and quantization aware training across both PyTorch and Keras. Its cost table is the most useful thing it publishes, from about 1 to 10 CPU minutes for PTQ to 12 to 36 GPU hours for QAT, and its sharpest constraint is that it needs a pre-trained floating point model before it does anything.

**SonySemiconductorSolutions/mct-model-optimization** — Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

- Repository: https://github.com/SonySemiconductorSolutions/mct-model-optimization
- Website: https://sonysemiconductorsolutions.github.io/mct-model-optimization/
- Stars: 453 · Forks: 80
- Language: Python
- License: Apache-2.0
- Published: 2026-09-10 · Updated: 2026-09-10 · Language: en
- Canonical page: https://hysenlabs.com/projects/sonysemiconductorsolutions-mct-model-optimization

## It refuses to start without a pre-trained floating point model

The important note in the install section is short and load bearing: to use MCT you need to provide a pre-trained floating point model in PyTorch or Keras form as an input. It is repeated as a required input in the method list. Nothing here trains a network from nothing, and there is no code path that produces the baseline you then compress.

That single constraint decides who this is for. A team without a trained baseline cannot use the toolkit at all, and a team with one has already paid the largest cost in the pipeline before touching MCT. The compression work that follows is comparatively cheap, which is exactly what the cost table argues, but the honest framing is that MCT optimizes an artefact you already own rather than creating one.

It also changes how you evaluate results. Comparing a quantized model against the uncompressed original is the only measurement available, so you need somewhere to put the float baseline and a validation set to run both against. A representative dataset is an optional input, either supplied by you or produced by the Data Generation capability, so the dataset is the one input you are not forced to bring.

## The cost table is the decision axis: CPU minutes to a day and a half of GPU

Three methods are published side by side with complexity and computational cost attached, and those numbers are more useful than the feature list. Post training quantization is low complexity at roughly 1 to 10 CPU minutes. GPTQ, described as parameter fine-tuning using gradients, is moderate at roughly 1 to 3 GPU hours. Quantization aware training is high at roughly 12 to 36 GPU hours.

Read that as a ladder. PTQ needs no training infrastructure at all, which is why it is the right first attempt and why the tutorial examples start there. GPTQ moves to a GPU because it uses gradients to correct quantization error in the weights, and it stays measured in hours rather than days because it is still a post training method rather than a retrain. QAT is the only one that requires training, and at 36 hours it belongs in a schedule rather than in a session.

All three end in the same place, a compressed model, and the README says each flow runs through a quantization core using hardware-aware algorithms and hyper-parameters. You are choosing how much compute to spend to get there, not choosing a different destination.

## Hardware awareness is a file you author, not hardware MCT detects

Behind the hardware-aware claim sits Target Platform Capabilities, or TPC. A TPC describes the constraints of the target hardware that the model is being optimized for, and the quantization core consumes it when running any of the three flows. The documentation for it lives inside the package rather than on the site, at `model_compression_toolkit/target_platform_capabilities/README.md`, which tells you the format is meant to be read by people integrating a new device.

That is the part to understand before adopting. Nothing here sniffs the hardware you are running on. The target is an input you supply, describing what the destination supports, and the optimizer uses it to settle quantization decisions. If your deployment target is a specific NPU or accelerator whose capabilities nobody has written down, the hardware-aware path degrades into ordinary quantization until someone writes that description, and that work is on you.

Per-layer decisions rather than one global bit width follow from that arrangement. Mixed-precision search assigns an optimal bit width per layer for weights and activations, and graph optimizations transform the model so it is better fitted to the quantization process before any of that happens. Reshape first, then choose widths per layer, both against the TPC.

## Threshold search minimizes expected quantization noise, and the docs cut off there

Below mixed precision sit three more mechanisms. Quantization parameter search chooses activation thresholds to minimise expected quantization noise, using methods including MSE, No-Clipping and MAE, each with its own example notebook. Advanced quantization algorithms cover cases the basic path handles poorly, naming shift negative correction, outliers filtering and clustering.

Then the visible text stops. The section heading for data-free quantization is cut off partway through the word, and everything after it, along with the rest of the feature list and the resources and community sections the navigation bar links to, is not present in what was published. So the capability is announced in the navigation and its description is missing, which is an awkward gap for anyone deciding whether they can quantize without a representative dataset.

What you can still evaluate is the structure. Two frameworks are supported with separate API pages for each method, PyTorch and Keras post training quantization, PyTorch and Keras gradient post training quantization, and a QAT section, plus interactive notebooks for hands-on work. The GPTQ and QAT examples are built on MobileNet, which tells you the intended scale of model and the shape of the tutorial path.

## requirements.txt tracks three version lines and excludes one networkx build

The dependency list is where upgrade cost becomes concrete, and it is short enough to read in full:

```text
networkx!=2.8.1,<3.6
tqdm
Pillow
numpy
scikit-image
scikit-learn
tensorboard
PuLP
matplotlib<3.10.0
scipy
protobuf
mct-quantizers~=1.7.0
pydantic>=2.0,<2.12.0
edge-mdt-cl~=1.1.0
```

Three observations. The quantizers live in a separate distribution, `mct-quantizers~=1.7.0`, versioning independently of MCT's own 2.6.1, so there are two release lines to watch rather than one. `edge-mdt-cl~=1.1.0` is a third, and the visible documentation does not explain what it does. And `networkx!=2.8.1,<3.6` excludes one specific build outright without saying why, which is the kind of line that either matters a great deal to you or means nothing.

The bounds are also inconsistent in a way that suggests history rather than policy. `matplotlib<3.10.0` and `pydantic>=2.0,<2.12.0` carry ceilings, most entries carry nothing, and only networkx has an exclusion. PuLP being present is consistent with per-layer bit-width assignment being solved as a problem rather than applied by a fixed rule, which matches what mixed-precision search claims to do.

## setup.py reads requirements.txt and README.md while it builds

Packaging here is old style, and it does its file reading at build time. `setup.py` declares the distribution as `model_compression_toolkit`, and `find_packages` is called with an include list limited to `model_compression_toolkit` and `model_compression_toolkit.*`, so only that tree ships.

Two helpers do the work. `read_install_requires` opens `requirements.txt` and takes the first line of each entry to build the install requirements, and `get_log_description` opens `README.md` and returns its contents as the long description with markdown content type set. So the file you just wrote is the package description, and the file you just edited is the dependency list, both read during the build rather than declared inline.

A third helper is worth knowing about if you ever build from a release tag. `get_release_arguments` parses known arguments, discards them, and rewrites `sys.argv` with only the unknown ones before handing control to setuptools, which is how a release script can pass options through to the build without argparse refusing them first. One small artifact of that hand-written packaging: the log line in `get_log_description` prints "Reading READEME File", misspelled, so a build log with that string is expected rather than a sign of a corrupted checkout.

## The most expensive method has the thinnest documentation

Every row of the methods table carries an API link and a tutorial link, and the allocation is revealing. PTQ and GPTQ each get two dedicated API pages, one PyTorch and one Keras, plus two example notebooks. QAT gets neither. Its API link is an anchor on the documentation index rather than a page of its own, and its single notebook is the Keras one.

That is a strange allocation, because QAT is the method with the largest bill. At 12 to 36 GPU hours it is the one where you would most want to know the expected accuracy cost before committing a day and a half of hardware, and it is the one whose documentation is a shared index page. Anyone planning a QAT run should budget time to read that index rather than assuming the PTQ tutorials transfer.

The Keras file naming is inconsistent too, which is a small thing that costs time. The Keras post training notebook spells the name `example_keras_post-training_quantization` with a hyphen where the PyTorch notebook uses `example_pytorch_post_training_quantization`, so a link or script written from one will not resolve against the other.

Read together with the truncated feature list, the pattern is that documentation depth tracks popularity rather than cost. PTQ is the path most people walk first and it is the best documented. The expensive paths are thinner, which is worth knowing before you budget GPU time on the strength of a table row.

## Patch releases land months apart, and the docs live in six places

Release cadence sets the rhythm of an upgrade. v2.5.1 shipped on 2026-01-05, v2.6.0 on 2026-03-03, and v2.6.1 on 2026-09-17, with the last push on the same date as that patch. Six months between a minor and a patch means you can reasonably plan around the current version rather than tracking a fast-moving main branch, and it also means a regression you hit may sit unfixed for a while.

Documentation is spread rather than consolidated. The README links out to a hosted documentation site with per-method API pages under an `api_docs` path, an installation guide in `INSTALLATION.md`, a FAQ in `FAQ.md`, a quantization troubleshooting note at the repository root, and a tutorials directory with notebooks. The repository also carries `tests_pytest/`, a `network_deployment_tests/` directory and a `test_release/` directory, which is more validation surface than most projects of this size bother with, and `docs/` alongside `docsrc/` for the site build.

Framework-native quantization is the alternative to all of this. PyTorch and TensorFlow both ship their own APIs, which arrive with the install you already have and need no second distribution. What MCT adds is the TPC description, the per-layer search, and one interface across two frameworks, at the cost of an extra install plus a hardware description you have to write yourself.

## Conclusion

Adopt MCT if you have a trained PyTorch or Keras model, a target device whose constraints you can write down as a TPC, and a few CPU minutes to spare for a PTQ baseline you can compare against. Do not adopt it as a training project, because it refuses to start without a pre-trained floating point model, and do not adopt it on a framework-native quantizer you already use without weighing what hardware awareness buys you, since that is the part with a separate install and a file you have to author. Verify three things before committing: that the quantizer package version mct-quantizers~=1.7.0 is compatible with your target, since it versions separately from MCT's own 2.6.1; what networkx 2.8.1 was excluded from, which requirements.txt does not explain; and the PTQ accuracy delta on your own validation set, because every method here trades accuracy for size and the published cost figures say nothing about what you lose.

## FAQ

### What does mct-model-optimization require before it can compress a model?

A pre-trained floating point model in PyTorch or Keras form, which the install section states as an important note and the method list repeats as a required input. A representative dataset is optional, since it can be supplied by you or generated by the Data Generation capability.

### How much compute does each quantization method in MCT cost?

Post training quantization is listed as low complexity at roughly 1 to 10 CPU minutes, GPTQ as moderate at roughly 1 to 3 GPU hours, and quantization aware training as high at roughly 12 to 36 GPU hours.

### What is Target Platform Capabilities in mct-model-optimization?

A TPC describes the constraints of the target hardware that the model is being optimized for, and the quantization core uses it to produce hardware-aware results. Its format is documented inside the package, at model_compression_toolkit/target_platform_capabilities/README.md, and the target is supplied by you rather than detected.

### Does mct-model-optimization support both PyTorch and Keras?

Yes, with separate API documentation for PyTorch and Keras post training quantization, for PyTorch and Keras gradient post training quantization, and a QAT section. Installation expects Python 3.10 or newer with PyTorch 2.3 or newer, or TensorFlow 2.14 or newer.

### What packages does installing mct-model-optimization pull in?

The main package installs with pip install model-compression-toolkit. Its requirements.txt declares networkx, tqdm, Pillow, numpy, scikit-image, scikit-learn, tensorboard, PuLP, matplotlib, scipy, protobuf, mct-quantizers at 1.7, pydantic, and edge-mdt-cl at 1.1.

### How does MCT differ from the quantization APIs already in PyTorch or TensorFlow?

The framework-native APIs arrive with the install you already have. MCT is a separate distribution that adds a Target Platform Capabilities description of the target hardware, per-layer bit-width search through mixed-precision search, and one interface covering both frameworks, at the cost of an extra install and a hardware description you have to write.

## Sources

- [License: Apache-2.0](https://github.com/SonySemiconductorSolutions/mct-model-optimization/blob/main/LICENSE)
- [Project website](https://sonysemiconductorsolutions.github.io/mct-model-optimization/)
- [README](https://github.com/SonySemiconductorSolutions/mct-model-optimization/blob/main/README.md)
- [Releases](https://github.com/SonySemiconductorSolutions/mct-model-optimization/releases)
- [SonySemiconductorSolutions/mct-model-optimization on GitHub](https://github.com/SonySemiconductorSolutions/mct-model-optimization)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/sonysemiconductorsolutions-mct-model-optimization