optimum-intel: exporting Hugging Face models to OpenVINO IR

One interface between four libraries and OpenVINO

Optimum Intel is described in the README as the interface between the Transformers, Diffusers, Sentence Transformers and timm libraries and the different tools and libraries provided by OpenVINO, in order to accelerate end-to-end pipelines on Intel architectures. OpenVINO itself is described as an open source toolkit enabling high performance inference for Intel CPUs, GPUs and special deep learning inference accelerators, and it comes with tools to optimize models with compression techniques such as quantization, pruning and knowledge distillation. Optimum Intel's job is the three steps in between: optimize the model, convert it to the OpenVINO Intermediate Representation format, and run inference with OpenVINO Runtime.

The scope is wider than text generation. The repository topics name asr, tts, vlm, llm, omni and diffusers alongside transformers and timm, which is a list of pipelines rather than a single model family, and the library ships per-task classes to match, from OVModelForCausalLM for text generation to OVModelForSpeechSeq2Seq for speech recognition. The project is Apache-2.0 licensed, the default branch is main, the last push to the repository was on 2026-10-22, and the recent releases include v2.2.0 from 2026-09-17 and v2.1.0 from 2026-08-05.

Installing the package, or the branch you need

The published install is a single pip command.

pip install -U optimum-intel

The README then explains why you might not want it. Optimum Intel is a fast-moving project with regular additions of new model support, so you may want to install from source.

pip install optimum-intel@git+https://github.com/huggingface/optimum-intel.git

That is a genuine trade-off rather than a formality, and the setup.py makes the cost visible. The install requirements are torch>=2.1, optimum~=2.3.0, transformers>=4.51 with an upper bound below 5.11, huggingface-hub, nncf>=3.3.0, openvino>=2026.0, openvino-tokenizers>=2026.0, requests, torchvision and setuptools. So you take torch and OpenVINO into the same environment, pinned fairly tightly, and the test requirements pull in a much longer list, including timm, diffusers-related packages, sentence-transformers, peft, datasets, open_clip_torch and openvino-genai.

The version is derived from a file rather than a literal, and setup.py reads optimum/intel/version.py to find it, then appends the short git commit hash when the version ends in .dev0, so a source install reports exactly which commit it came from.

Exporting a checkpoint with optimum-cli

The export step is a command, and the README's example uses TinyLlama/TinyLlama_v1.1.

optimum-cli export openvino --model TinyLlama/TinyLlama_v1.1 ov_TinyLlama_v1_1

The output name is the second argument, and it becomes a model repository you can load later. For a model hosted on the Hub, the README points at a Hugging Face space for exporting, and after the conversion a repository is pushed under your namespace, which can be either public or private. That last detail is worth pausing on, because a converted model is a derived artefact of a checkpoint whose licence you may or may not have the right to redistribute, and the conversion pipeline is not itself a licence check.

There is also a documented alternative path for inference. The README notes that an exported model can also be inferred using the OpenVINO GenAI framework, which provides optimized execution methods for highly performant generative AI. So you are not locked into the Optimum Intel classes once the export is done, which is a useful property if you want the model to outlive this library.

Inference by swapping AutoModel for OVModel

The inference pattern is a class substitution. Instead of the AutoModelForXxx class, you use the corresponding OVModelForXxx class, and everything else in the pipeline stays the same.

from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer, pipeline

model_id = "ov_TinyLlama_v1_1"
model = OVModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("Hey, how are you doing today?", max_new_tokens=100)

The tokenizer still comes from transformers and the pipeline is still the transformers pipeline, which is the point of the design. A team that has already built a script around pipeline and AutoTokenizer can swap two lines and keep the rest, and the exported model is loaded from a Hub-style repository id, so the artefact is versioned and shared the same way as any other model.

Post-training static quantization with a calibration dataset

Quantization is the other half of the toolkit, and the README's example applies post-training static quantization to a Whisper model using the LibriSpeech dataset for the calibration step.

from optimum.intel import OVModelForSpeechSeq2Seq, OVQuantizationConfig

model_id = "openai/whisper-tiny"
q_config = OVQuantizationConfig(dtype="int8", dataset="librispeech", num_samples=50)
q_model = OVModelForSpeechSeq2Seq.from_pretrained(model_id, quantization_config=q_config)

# The directory where the quantized model will be saved
save_dir = "nncf_results"
q_model.save_pretrained(save_dir)

Two details in that snippet matter in practice. The calibration data is not optional for static quantization, so a dataset name and a sample count have to be supplied, and the sample count here is 50, which is small, so it is a starting point rather than a quality setting. And the quantized result is a directory written by save_pretrained, with the name nncf_results in the example, which points at the NNCF compression toolkit in the dependency list. The README also notes that OpenVINO comes with pruning and knowledge distillation among its compression techniques, and the docs section for optimization is where those are described.

The notebooks, and what the repository layout says about the workflow

The running examples live in a notebooks/ directory, and the README says to check it out to see how Optimum Intel can be used to optimize models and accelerate inference. Each example carries its own requirements file, and the README is explicit that you need to install them for every example.

cd <example-folder>
pip install -r requirements.txt

The top level of the repository confirms the shape of the project: an optimum/ package directory holding the library, notebooks/, docs/, tests/, and the packaging files setup.py, setup.cfg and pyproject.toml. The primary language of the repository is listed as Jupyter Notebook, which is a direct consequence of the examples being notebooks.

The tooling configuration is worth a mention because it tells you what the maintainers expect of a contribution. Black is configured with a line length of 119 and a py311 target, with the notebooks folder excluded, and ruff selects C, E, F, I and W rules with E501 never enforced, which is consistent with the 119 character limit being handled by black instead. The pytest configuration registers a run_slow marker and a gemma4 marker, and the latter carries a note that it requires transformers>=5.5, which tells you the newest architectures are tested against a dependency range that has moved past the one in the install requirements.

Where an OpenVINO export is the wrong answer

Three situations point elsewhere. The first is a deployment that has to run on more than Intel hardware. The exported artefact is OpenVINO Intermediate Representation, so a server that also runs on an NVIDIA GPU or on a cloud TPU needs a second path, and you end up maintaining two exported models. The second is a project that cannot pull torch and OpenVINO into one environment, for instance a training image that already has a carefully pinned torch build, since the install requirements ask for torch>=2.1 alongside openvino>=2026.0 and nncf.

The third is a case where the model is not yet supported. The README describes the project as fast-moving with regular additions of new model support, and the topic list names omni and vlm among the pipeline types, which are the sort of thing that arrives with the underlying architecture rather than with the integration. The marker configuration in pyproject.toml shows the pattern, with the gemma4 marker requiring transformers>=5.5, so the newest architectures are handled in a separate test lane rather than in the default one.

There is a real alternative to consider, and it is not another Hugging Face integration. If your requirement is int8 inference on Intel hardware rather than OpenVINO specifically, the ONNX Runtime path with a quantised ONNX model keeps the artefact portable across runtimes. The difference in approach is that OpenVINO IR is tuned for OpenVINO Runtime and Intel hardware, while ONNX is a neutral interchange format you can run elsewhere, so the choice is about how locked you want the model to be to one vendor's stack.

Editorial conclusion

optimum-intel fits a team already committed to Intel hardware, especially one running Whisper or a text generation model on CPUs today, and that wants the model to live in the Hugging Face ecosystem so the rest of the pipeline keeps working unchanged. It does not fit a team that has to keep the same weights portable across several accelerators, since the exported artefact is OpenVINO IR, and it does not fit a project that cannot take torch and OpenVINO in the same environment. Before you commit, export one of your own models with optimum-cli and measure it, because the README's own framing of the project as fast-moving with regular additions of new model support means the model coverage you need may not be there yet.

Frequently asked questions

Official sources

1. huggingface/optimum-intel on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases