WikiPron: mining multilingual pronunciation data from Wiktionary

The gap WikiPron fills for low-resource languages

Most grapheme-to-phoneme toolkits assume you already have a pronunciation lexicon. CMUdict covers English. For everything else, you either pay a vendor or build a lexicon yourself. WikiPron's pitch is that Wiktionary already contains IPA transcriptions for a large number of languages, contributed by editors, and that this material can be extracted programmatically instead of copied by hand.

The README describes it as "a command-line tool and Python API for mining multilingual pronunciation data from Wiktionary, as well as a database of pronunciation dictionaries mined using this tool," and states the database holds over 5 million word pronunciation pairs from 348 languages. That scale is the point. A researcher working on, say, Georgian or Yoruba can get a lexicon without commissioning one.

The audience is narrow but real: computational linguists, speech engineers training G2P models for languages with thin resources, and phonologists who want a corpus to query. It is not aimed at application developers who want a text-to-speech front end. That distinction shows up everywhere in the design.

How the scraping pipeline actually works

The flow is straightforward. A language is identified by a three-letter ISO 639-3 code, so fra selects French. WikiPron fetches the corresponding Wiktionary entries, parses the IPA transcriptions out of the wiki markup with lxml, and emits one word-pronunciation pair per line, tab-separated, with the pronunciation in IPA.

The parsing is where the interesting decisions live. By default WikiPron selects broad transcriptions in angled brackets /like this/; the --narrow flag switches to narrow transcriptions written [like this]. Some languages only have one or the other, and the README notes Russian only has narrow. Dialects are filtered with --dialect, which accepts a pipe-separated union such as 'General American | US', and transcriptions with no dialect label are always kept regardless of the flag.

Segmentation is the part that distinguishes WikiPron from a naive regex scraper. It uses the segments library to split transcriptions on whitespace in a way that keeps IPA diacritics attached to their parent symbol, so [kʰæt] becomes kʰ æ t rather than k ʰ æ t. That matters if you are training a model over symbols, because a detached aspiration mark is a different token. The --no-segment flag disables this.

Parentheses get their own treatment. English A&E is transcribed /eɪ.ən(d)ˈiː/ on Wiktionary, and --parens decides what happens: expand generates every variant, skip drops the parenthesized material, and show leaves it in place. Expand is the default, which means a single Wiktionary entry can produce several output lines. That inflates row counts in a way that is easy to misread as lexical coverage.

Installing WikiPron and scraping French in one command

Installation is a single pip command, and the README gives Python 3.11 or later as the floor via pyproject.toml's requires-python field.

pip install wikipron

That puts a wikipron executable on your PATH. The basic invocation takes only a language code:

wikipron fra

By default the output goes to the terminal, so a first run against French will scroll past. Redirect it to a file to keep it:

wikipron fra > fra.tsv

The result is tab-separated, one pair per line, with the pronunciation segmented by spaces. A few lines of the French output look like this:

accrémentitielle	a k ʁ e m ɑ̃ t i t j ɛ l
accrescent	a k ʁ ɛ s ɑ̃
accrétion	a k ʁ e s j ɔ̃

If you would rather stay inside Python, the README's workflow builds a Config object and iterates over a generator:

import wikipron

config = wikipron.Config(key="fra")  # French, with default options.
for word, pron in wikipron.scrape(config):
    ...

Run wikipron -h for the full option list; the README points there rather than enumerating every flag.

Network dependence, uneven coverage, and the platform caveat

The first constraint is that WikiPron scrapes a live website. There is no bundled offline dump of the source entries in the package. Every run depends on Wiktionary being reachable and on the page structure being what the parser expects. If Wiktionary's markup changes, the parser breaks, and you find out when your output is empty or malformed. A pre-mined snapshot exists in the repository's data/ directory, which is the safer input for reproducible work, but it is a snapshot and will drift from the live site.

Coverage is uneven by design. The README points readers to Wiktionary's own category listing for terms with IPA pronunciation by language as the authoritative list of what can be scraped. Nothing in the tooling guarantees a given language has enough entries to be useful, and the 348-language figure says nothing about per-language volume. A language with forty entries is technically covered.

The dependency list carries a platform caveat that is easy to miss. pynini is declared with a linux-only environment marker in pyproject.toml, so the dependency set is not identical across operating systems even though the classifiers claim Operating System :: OS Independent. If your pipeline touches pynini, plan for Linux.

Finally, the tool produces data. It does not decide whether that data is good. Wiktionary transcriptions come from volunteer editors with varying conventions, and WikiPron's dialect and bracket filters are blunt instruments. A lexicon scraped with default settings mixes whatever the editors wrote.

WikiPron is not Phonemizer, and that is the point

The common confusion is treating WikiPron as a phonemizer. Phonemizer, and similar libraries, take arbitrary text at runtime and produce phonemes using a backend such as eSpeak or a trained model. WikiPron does none of that. It extracts pairs that already exist on Wiktionary. If a word is not on Wiktionary, WikiPron has nothing to say about it.

The practical difference is where the work happens. With a phonemizer you get a black box that handles unseen words and returns something plausible. With WikiPron you get a training set. The README explicitly points to a separate repository, kylebgorman/wikipron-modeling, for grapheme-to-phoneme models and modeling software. That separation is deliberate: WikiPron supplies the data, and the modeling repository trains on it. If your goal is a model that generalizes to new words, WikiPron is the first step, not the whole pipeline, and you will need a G2P training setup on top of it.

If your goal is a quick runtime lookup for a handful of languages, a phonemizer will get you there faster. If your goal is a lexicon for a language no phonemizer supports well, WikiPron is the more direct route, because you are not depending on someone else's model quality.

Release cadence, upgrade cost, and the data licence split

The project is not archived and the last push was on 2026-07-23, the same day as the v2.2.0 release. Before that came v2.1.0 on 2026-07-01 and v2.0.0 on 2026-05-23. Two releases in the two months before the last push suggests an active release line, and the README points to CHANGELOG.md for notable changes. Read that file before upgrading, because the jump from v1 to v2 is not described in the README and the changelog is the only place the project documents it.

Upgrade cost is mostly about output stability. Because the tool scrapes a live wiki, a new version can change segmentation, parenthesis handling, or dialect matching, and your downstream TSV will shift accordingly. Pin the version in your environment and re-scrape deliberately rather than letting a pip install change your corpus underneath you. The requires-python floor of 3.11 also means Python 3.10 and older are out.

The licensing is split and worth reading carefully. The code is Apache 2.0, per LICENSE.txt and the pyproject.toml license field. The mined data in data/ is not covered by that. The README states plainly that Wiktionary data in the data/ directory "has its own licensing terms" and links to Wiktionary's copyright page. So the permissive terms you get for the software do not automatically extend to the lexicons. Check the Wiktionary terms for your use case; this is not legal advice, and the two licences need to be evaluated separately.

Editorial conclusion

Adopt WikiPron if you need IPA transcriptions for many languages and can accept Wiktionary's uneven coverage and its attribution terms. Skip it if you need a runtime phonemizer for TTS or ASR, or if your target language has no Wiktionary pronunciation entries. Before building on it, run wikipron fra and inspect the segmentation, then check the data/ directory's separate licensing terms and the ISO 639-3 code for your language.

Frequently asked questions

Official sources

1. CUNY-CL/wikipron on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases