aeneas: forced alignment between audio and text with a Python CLI

What aeneas actually computes, and who needs that

The task is forced alignment. You already know the words; you do not know when each one is spoken. aeneas takes a list of text fragments plus an audio file containing the narration of that text and produces a synchronization map: for each fragment, a start time and an end time. The README shows the shape of the result on a Shakespeare sonnet, where the first line of the text file maps to [00:00:00.000, 00:00:02.640] and each following line gets its own interval.

That is a different problem from transcription. Speech recognition asks what was said. Forced alignment assumes you have the transcript and asks when. The distinction matters for who this is for: producers of audiobooks and EPUB 3 titles who own the text, subtitle authors working from a script, and researchers who need TextGrid or ELAN files to annotate a recording they have already transcribed. If you do not have the text, aeneas is the wrong tool. If you have the text and a recording that follows it, the alignment problem is well posed.

The output formats are the tell. Audacity AUD, ELAN EAF and TextGrid for annotation work; SMIL for EPUB 3; SRT, SBV, TTML and WebVTT for captioning; JSON, CSV, SSV, TSV, TXT and XML for further processing. A tool that emits SMIL and TextGrid is aimed at publishing and research workflows, not at live captioning.

Text-to-speech, DTW and the C extension inside aeneas

aeneas does not compare the waveform to the text directly. The topics list names the pieces: espeak, espeak-ng, festival, ffmpeg, dtw. The mechanism, as the documentation describes it, is a two-track comparison. The audio file is decoded with FFmpeg and reduced to a sequence of feature vectors. The text fragments are synthesized to speech with a TTS engine, typically eSpeak, and the synthesized audio is reduced to a comparable sequence of feature vectors. The two sequences are then aligned with dynamic time warping.

This design explains both the strengths and the failure modes. Because alignment runs against synthesized speech rather than an acoustic model of the speaker, aeneas does not need a trained model for your language. It needs a TTS engine that can pronounce that language. The task_language parameter in the command line is what selects the pronunciation rules, which is why a wrong language code produces a plausible-looking map with wrong boundaries rather than an error.

The DTW core and other hot paths are implemented as Python C/C++ extensions, which is why the README lists Python headers as optional but strongly recommended. Without them the package falls back to slower pure-Python paths. The README does not state how much slower, so treat the compiled extension as the intended configuration and the fallback as a convenience.

Installing aeneas and aligning your first file

The generic procedure in the README has three prerequisites before any Python package: Python itself, FFmpeg, and eSpeak. After that, the executables espeak, ffmpeg, ffprobe, pip and python must all be callable from your shell. The install order matters, and the README says so explicitly: numpy first, then aeneas.

pip install numpy
pip install aeneas

aeneas ships a diagnostics entry point. Run it before you write any pipeline code, because a missing ffprobe or an eSpeak that is not on PATH will otherwise surface later as an alignment failure.

python -m aeneas.diagnostics

To compute a synchronization map from an MP3 and a plain text file, the README gives this command. The third argument is a pipe-separated task configuration; here it sets the language to English, the output format to JSON, and the text input type to plain.

python -m aeneas.tools.execute_task \
    audio.mp3 \
    text.txt \
    "task_language=eng|os_task_file_format=json|is_text_type=plain" \
    map.json

The README notes the backslashes are only for visual clarity and that the whole command can sit on one line. The result is map.json, a list of fragments with their time intervals. If you prefer to see working inputs before preparing your own, the tool can print live examples that run against files shipped in the repository.

python -m aeneas.tools.execute_task --examples

For multi-step work there is a second entry point, execute_job, which processes a job container rather than a single audio and text pair. The README points to a job container under aeneas/tests/res/container/job in the source tree as a reference for that layout.

Platform support, Python versions and the 2017 release line

The README is unusually blunt here: aeneas has been developed and tested on Debian 64bit with Python 2.7 and Python 3.5, and those are the only supported platforms. Other Linux distributions, Mac OS X and Windows are listed as confirmed to work but not supported, with a PLATFORMS file in the repository holding the details.

The version on PyPI is 1.7.3, dated 2017-03-15. The most recent GitHub release is v1.7.3 from 2017-03-16, preceded by v1.7.2 and v1.7.1. The repository itself is not archived and the last push was on 2026-07-25, so there is activity in the tree, but the release line has not moved since 2017. That gap is the single most important fact for anyone planning a dependency. Python 3.5 reached end of life years ago, and a package whose documented support matrix stops there is a package you will be pinning around.

The practical consequence: on a current Linux distribution you may need to install Python 2.7 or 3.5 alongside your system interpreter, or accept that you are running on an unsupported combination. The README anticipates this and points to aeneas-vagrant, a virtualized Debian image under VirtualBox and Vagrant, for cases where a native install proves difficult. All-in-one installers exist for Mac OS X and Windows, and there is a Bash script for deb-based distributions in the repository. The INSTALL file carries the step-by-step procedures.

Where aeneas breaks down

The TTS-based approach has a real cost: alignment quality depends on how well the synthesizer pronounces your text. Names, numbers, abbreviations and code-switched sentences are the usual casualties. Because the text is synthesized rather than recognized, an unusual pronunciation shifts the boundary estimate for that fragment. The documentation does not offer a confidence score per fragment, so a bad boundary and a good one look the same in the output file.

Long silences and music are another weak point. DTW aligns two sequences; a passage where the narrator pauses for several seconds, or an intro with background music and no speech, gives the algorithm no text to match against. You should expect to inspect and correct the map rather than publish it directly.

The platform matrix is the second limitation. Windows with Python 3 is not in the supported set. If your build environment is Windows and current Python, you are relying on an installer the README describes but does not guarantee.

Finally, the licence. aeneas is AGPL-3.0. If you run it as part of a network service, the AGPL's source-availability condition applies to that service in a way the GPL's does not. For a personal annotation script this is irrelevant; for a hosted product it is a design constraint to raise with whoever handles licensing. This is not legal advice, and the LICENSE file in the repository is the authoritative text.

What to use instead, and how the approach differs

The obvious alternative is a modern neural forced aligner. Tools in that family use an acoustic model trained on speech to produce frame-level or phoneme-level posteriors, then align the known transcript to those posteriors. The difference from aeneas is where the reference comes from: aeneas synthesizes the text with eSpeak or Festival and aligns audio to audio, while a neural aligner aligns audio to a model of speech. Neural aligners generally give finer granularity, down to the phoneme, and they do not depend on a TTS engine's pronunciation rules for your language.

The trade-off runs the other way too. A neural aligner needs a pretrained model for your language and a runtime that can execute it, which is a heavier dependency than espeak plus ffmpeg. aeneas runs on a CPU-only machine with 4 GB of RAM and a 2 GHz 64-bit CPU, per the README's system requirements, and needs no model download. For a one-off alignment of a public-domain audiobook on a small VM, that difference is decisive.

If you only need subtitles and you have no transcript, aeneas is not the right category at all; you want speech recognition, not alignment. Forced alignment assumes the text is already correct.

Maintenance, upgrades and the licence question

Upgrading aeneas is not a routine version bump. The last tagged release is v1.7.3 from 2017, so there is no newer release to move to. The repository's last push was on 2026-07-25, which means the tree sees changes, but the version number has not advanced in the release metadata. If you depend on aeneas, pin the version you install and record the exact Python interpreter and FFmpeg build you validated against, because a future FFmpeg release changing its output is a more likely breakage than a change in aeneas itself.

The dependency list is short: BeautifulSoup4, lxml and numpy, per requirements.txt. lxml is a compiled package, so it inherits the same build-toolchain concerns as aeneas's own C extensions.

On licensing, AGPL-3.0 is a strong copyleft. The README states the licence plainly and the repository carries a LICENSE file and a licenses/ directory. The distinction that matters in practice is between using aeneas internally to produce an alignment map and offering aeneas-backed functionality over a network. The first is unremarkable; the second is the case the AGPL was written for. Read the licence text and get a real opinion if your use is commercial.

Editorial conclusion

Adopt aeneas if you need a synchronization map between a known text and its recording, and you can live with the 2017-era dependency set, the Debian 64bit focus, and AGPL-3.0. Do not adopt it if you need a maintained dependency chain, Windows with Python 3, or per-word timestamps from a modern neural aligner. Verify first that python -m aeneas.diagnostics passes on your machine, that espeak, ffmpeg and ffprobe are all on PATH, and that your output format is on the supported list before you build a pipeline around it.

Frequently asked questions

Official sources

1. License: AGPL-3.0
2. Project website
3. readbeyond/aeneas on GitHub
4. README
5. Releases