ctc-forced-aligner: word timings from a CTC model and a transcript

The problem: audio and text that do not line up

Forced alignment is the task of taking an audio recording and its transcript and working out where in time each word is said. It is the missing piece in a lot of speech work, because a transcript gives you the words and a recording gives you the sound, and nothing in either tells you the alignment.

The package is described as providing an efficient way to perform forced alignment between text and audio using Hugging Face pretrained models. It uses Wav2Vec2, HuBERT and MMS models, and the project describes the result as a tool for creating speech corpuses.

That framing explains the design. A speech corpus needs timestamps, not just text, and the traditional way to get them is a dedicated forced aligner trained on phoneme-level data. This package takes a different route, using CTC models trained on speech recognition, which are far more widely available, and it does so by adding a star token between words so that the model has somewhere to put the gaps between them.

The audience is a speech or NLP researcher who has recordings and transcripts and needs word level timings, and who wants to use the model ecosystem they already has rather than a separate aligner stack.

The stated advantages are specific. At least five times less memory usage than the TorchAudio forced alignment API, according to the README. Wide language support, with English, Arabic, Russian and German named and 1126 more languages claimed. Flexible alignment granularity at sentence, word or character level. Customizable parameters. GPU acceleration. And JSON output.

How the star token trick makes CTC alignment work

The alignment parameters are the interesting part of the design, and they are documented as CLI options with their defaults.

The star_frequency setting controls the frequency of the `<star>` token insertion, and it takes two values, segment or edges, with edges as the default. The merge_threshold controls segment merging and defaults to 0.00. The split_size setting is the alignment granularity, sentence, word or char, with word as the default.

The mechanism follows from how CTC output works. A CTC model emits one label per frame, and the frames that fall between two words are blanks, which is exactly where the word boundary lies. Inserting a star token into the text at a chosen frequency gives the model a label that can align to those boundary regions, and the segments that come out can then be mapped back onto the original words. Choosing edges rather than every segment means stars are placed at word boundaries only, which is why it is the default for word level work.

The Python API shows the pipeline explicitly. preprocess_text takes the text and a romanize flag and a language code and returns the starred tokens and the starred text. get_alignments takes the emissions, the starred tokens and the alignment tokenizer and returns segments, scores and a blank token. get_spans turns those into spans against the starred tokens, and postprocess_results turns spans into word timestamps against the original text and the stride.

The romanize option is the other half of the design. CTC models are trained on Latin script, so a non-Latin transcript has to be converted before the model can align it, and the README marks --romanize as required when using the default model.

The command line, the model choice and the chunking window

The tool is usable as a single command, and the argument list is long enough that the defaults matter.

ctc-forced-aligner --audio_path "path/to/audio.wav" --text_path "path/to/text.txt" --language "eng" --romanize

Beyond the three required arguments, the notable ones are the alignment_model, whose default is the author's own MahmoudAshraf/mms-300m-1130-forced-aligner on Hugging Face, and the compute_dtype, which defaults to float32, and batch_size at 4, and device, which is cuda if available and cpu otherwise.

The two windowing parameters are what let it handle long recordings. window_size is the chunk length in seconds for audio chunking and defaults to 30, and context_size is the overlap between chunks in seconds and defaults to 2. So a long file is cut into 30 second windows with a two second overlap, and the overlap is what stops a word that straddles a boundary from being lost.

There is also an attention implementation option, defaulting to eager, which is a performance lever on GPUs where the fused implementations are faster.

The examples show the three ways people use it: an English file with romanization enabled, a Russian file with romanized text, a sentence-level alignment with --split_size sentence, and a run against a model with a native vocabulary, jonatasgrosman/wav2vec2-large-xlsr-53-arabic for Arabic, which is the way to avoid the romanization step for a script the model already handles.

The compiled extension and the model dependencies

The build has a native component, and the setup file makes that explicit. A Pybind11Extension named ctc_forced_aligner.ctc_forced_aligner is built from ctc_forced_aligner/forced_align_impl.cpp, with the optimisation flag set to /O2 on Windows and -O3 elsewhere. The build system requires setuptools 42 or newer, wheel and pybind11, and the package requires Python 3.10 or later.

The runtime dependencies are in requirements.txt and are short: uroman, nltk, torch, torchcodec, transformers 4.34 or later, and Unidecode. The romanization stack, uroman and Unidecode, is there for the same reason the --romanize flag exists, to turn a non-Latin transcript into something a Latin-script CTC model can align. nltk is there for the punctuation list, which is shipped as package data alongside the module.

The console entry point is ctc_forced-aligner, mapped to ctc_forced_aligner.align:cli.

One external prerequisite is called out plainly: FFMPEG is needed as a prerequisite to use, which is what the audio loading path depends on to decode whatever container the recording arrived in.

The two install routes are the usual pair. From GitHub, the latest version is installed with pip install git+https://github.com/MahmoudAshraf97/ctc-forced-aligner.git, and locally from source you clone the repository, change into the directory and run pip install -e .[dev], where the dev extra pulls ruff, pre-commit, pytest 7.x and torchaudio.

Licence ambiguity, release cadence and the output format

There is a discrepancy in the licence information that anyone planning to use this should resolve before they build on it. The licence field for the repository shows BSD-2-Clause, and the README carries an MIT badge, while the package metadata in pyproject.toml declares the licence as CC-BY-NC 4.0 in the package metadata. The repository also has a LICENSE file. Those are three statements about the same artefact, and CC-BY-NC is not a software licence in the usual sense, since it forbids commercial use. This review does not give legal advice, and the point to make is procedural: read the LICENSE file and confirm with the author which terms apply to your use before you depend on it, because a non-commercial restriction in a dependency can block a commercial deployment.

The release history is thin. The tagged releases are v0.2 from 2024-06-03 and v0.1 from 2024-05-29, and the last push was on 2026-09-07. So there is recent commit activity with no new release since 2024, which means a pip install from the package index gives you the 2024 code and the git install gives you something newer. The version is dynamic, taken from the package attribute at build time, so the two paths can report different numbers.

The output format is JSON and the documented fields are the aligned text and a list of segments, each holding the start time, the end time and the text of the corresponding segment. The example output shows word level segments with times to three decimal places, which is millisecond resolution, and the whole document is written to a file for analysis and integration.

Where ctc-forced-aligner is the wrong tool

The first limitation is the input assumption. This is a CTC approach, and the method expects the transcript to match the audio. Forced alignment tools built for phoneme-level alignment can often handle a transcript with errors, insertions or deletions, because the model maps phonemes to audio and tolerates mismatch. A CTC model that has to emit a star between each word of your text will align badly if the text is wrong, and it will not tell you it is wrong, it will just produce confident timestamps that point at the wrong place. For a corpus of clean, matched recordings this is fine. For a corpus of found audio with approximate transcripts, it is the wrong instrument.

The second limitation is that the alignment is only as good as the acoustic model, and the default model is a 300 million parameter MMS model. The README offers the alternative of a language-specific model, such as the Arabic XLSR model, which is a better match for a single-language corpus and will be both faster and more accurate than a multilingual model used for one language. That is an optimisation worth making, and it is the kind of thing the option exists for.

The third is that the project is quiet. Two tagged releases, both in 2024, and a Python package with a compiled extension and a moving set of model dependencies is a combination where breakage is likely over time, and the answer is to pin versions and test on a small sample before running a corpus.

The alternative worth comparing is a dedicated forced aligner, such as the Montreal Forced Aligner, which is built for exactly this task, handles imperfect transcripts, and is the tool most speech corpora are actually built with. The difference in approach is the model class. A phoneme-based aligner predicts a phoneme sequence and maps it to the transcript, which is more accurate and more forgiving. A CTC recogniser predicts characters and is given a transcript to align against, which is cheaper, uses models almost everyone already has, and needs the romanization step. For a researcher who already has CTC checkpoints and wants timestamps for an existing corpus, this package is the faster path. For a linguist who needs trustworthy phoneme boundaries on imperfect audio, it is the wrong tool.

Editorial conclusion

ctc-forced-aligner fits a researcher who has audio and a matching transcript and needs word level timestamps without training an aligner, because the model inference, the text preprocessing and the span extraction are all exposed as functions you can call from Python. It is the wrong choice if you need a forced aligner that handles imperfect transcripts, since this is a CTC approach and the package expects the text to match the audio. Before adopting it, check the licence on the release you intend to use, because the licence field for the repository shows BSD-2-Clause while the package metadata in pyproject.toml declares CC-BY-NC 4.0, and confirm your FFMPEG and torch setup before you process a corpus, since the two newest tagged releases are from 2024.

Frequently asked questions

Official sources

1. Issues
2. License: BSD-2-Clause
3. MahmoudAshraf97/ctc-forced-aligner on GitHub
4. README
5. Releases