jiwer: the five error rates speech-to-text teams actually report

Five measures over one edit distance

jiwer exists to answer one question: how close is a machine transcript to the human reference? The package supports five measures, word error rate, match error rate, word information lost, word information preserved and character error rate. All of them are computed from the same underlying quantity, the minimum-edit distance between one or more reference sentences and one or more hypothesis sentences. That shared foundation is what makes the metrics comparable with each other, and the README points to the Morris, Maier and Green paper on connected speech recognition for the argument that WER alone is misleading when words carry unequal amounts of information. Match error rate counts correct words rather than word count, WIL and WIP are information-flipped counterparts of each other, and CER works at the character level, which matters for languages where a word is not a reliable unit. jiwer is not a general string similarity library; it is a set of five speech-specific views of one edit-distance computation.

RapidFuzz does the alignment work

The performance story of jiwer is delegated. The README states that the minimum-edit distance is calculated using RapidFuzz, which uses C++ under the hood and is therefore faster than a pure Python implementation. The dependency list in `pyproject.toml` names `rapidfuzz>=3.9.7` and `click>=8.1.8`, and nothing else, so the package is small by design. For a library that people tend to run over every utterance in a test set, this matters: the alignment cost is what decides whether evaluation is a quick check at the end of a training run or a job that queues overnight. The flip side is that the version floor on RapidFuzz is a real constraint, since a pinned floor means an environment with an older RapidFuzz present will not install cleanly, and the underlying alignment semantics are those of another project rather than of jiwer itself. Anyone relying on exact score reproducibility across time should treat the RapidFuzz version as part of their evaluation environment.

Empty references got a defined answer in version 4

This is the sharpest design decision in the package. The README is candid that there is undefined behaviour when you apply an empty reference and hypothesis pair to the WER formula, because you get a division by zero. As of version 4.0, jiwer defines that behaviour and thereby lifts the requirement for reference strings to be non-empty, which the project says allows for testing whether models hallucinate on silent audio. When both sides are empty there is no error, since the system correctly predicted silence:

import jiwer

assert jiwer.wer('', '') == 0
assert jiwer.mer('', '') == 0
assert jiwer.wip('', '') == 1
assert jiwer.wil('', '') == 0

assert jiwer.cer('', '') == 0

When the hypothesis is non-empty, every word or character counts as an insertion:

import jiwer

assert jiwer.wer('', 'silence') == 1
assert jiwer.wer('', 'peaceful silence') == 2
assert jiwer.process_words('', 'now defined behaviour').insertions == 3

assert jiwer.cer('', 'a') == 1
assert jiwer.cer('', 'abcde') == 5

The README also warns that there are usually multiple reference and hypothesis pairs, and that while one or more references may now be empty, most references should still be non-empty to test well.

Installing jiwer with uv or pip

Installation is deliberately short. The README leads with uv, which is the tool the project itself uses for development:

$ uv add jiwer

If you prefer pip and are on Python 3.8 or newer, the same package installs the usual way:

$ pip install jiwer

The `requires-python = ">=3.8"` entry in `pyproject.toml` is the machine-readable form of that same requirement, and the project is built with hatchling. There is no compiled extension of its own to build, since the heavy lifting lives in RapidFuzz, so an install is a wheel fetch plus two dependencies. After installing, the shortest useful call computes the word error rate between two strings:

from jiwer import wer

reference = "hello world"
hypothesis = "hello duck"

error = wer(reference, hypothesis)

A team wiring this into a pipeline can treat that call as the reference point: import a single measure function, pass reference and hypothesis, get a number back.

A CLI and a justfile for the maintainers

jiwer is not library-only. The `pyproject.toml` declares a console script entry point, `jiwer = "jiwer.cli:cli"`, so an installed environment also gives you a `jiwer` command, with `click` as its argument parsing dependency. The README does not document the flags of that command, so the documentation site at jitsi.github.io/jiwer is the place to check what it exposes, and that silence is worth knowing before you rely on it for automated runs. The repository also ships a `justfile` that describes the maintainer workflow, and it is worth reading because it shows how the project is actually exercised. Its quick target deliberately skips the slow tests:

uv run --group dev pytest \
    --ignore=tests/test_large_vocab.py \
    --ignore=tests/test_speed.py

The full run, the lint pass, the formatting pass and the documentation server are all defined as `just` targets, and the docs group uses mkdocs with mkdocstrings for API pages. The presence of `test_speed.py` and `test_large_vocab.py` as separately ignorable files tells you the project treats performance and vocabulary size as first-class concerns rather than afterthoughts.

What jiwer does not do for you

There are three gaps worth naming before you adopt it. The first is normalisation. The README does not document any lowercasing, punctuation stripping or number expansion, so if your references contain "12" and your hypotheses contain "twelve", the edit distance will count them as a substitution. That is a deliberate choice for a low-level metric library, but it means the caller owns tokenisation and normalisation, and that decision must be identical between reference and hypothesis or the score is meaningless. The second is reporting depth. jiwer returns a rate, and `process_words` exposes counts such as insertions, but the README describes no alignment output and no per-utterance breakdown, so explaining why a score moved is still manual work. The third is version coupling: the empty-reference semantics only exist from version 4.0, so a team that pins an older release inherits the division-by-zero ambiguity the project has since removed.

Apache-2.0, released by 8x8

The licence field on the repository says Apache-2.0, and the README states that the package is released under the Apache License, Version 2.0 licence by 8x8, with the `LICENCE` file in the repository root. Apache-2.0 is a permissive licence, which makes jiwer straightforward to pull into a proprietary product or an internal research stack, subject to the usual notice and licence obligations that only a qualified reviewer should interpret. On maintenance, the release history is regular and the tree is current: v3.0.5 in November 2024 with a fix for a SyntaxError in Python 3.12, v3.1.0 in January 2025, and v4.0.0 in June 2025, which is the release that introduced the defined empty-reference behaviour. The last push was on 2026-09-03, so the project is still receiving work, and the version pinned in `pyproject.toml` matches the latest tag.

Editorial conclusion

jiwer suits teams that need a repeatable ASR error metric with one agreed definition of edge cases, particularly those moving to version 4 and relying on empty-reference handling to detect hallucination on silent audio. It does not suit pipelines that need text normalisation rules, confidence weighting or alignment visualisation, because the README documents none of those. Before adopting it, check that your references are tokenised the way your normaliser expects, confirm your Python is 3.8 or newer, and run the empty-hypothesis assertions from the README against your own data to see how insertions are counted.

Frequently asked questions

Official sources

1. Issues
2. jitsi/jiwer on GitHub
3. License: Apache-2.0
4. README
5. Releases