lingua-py: language detection for short and mixed-language text
The most accurate natural language detection library for Python, suitable for short text and mixed-language text
At a glance
- What is it?
- lingua-py is the Python binding to Lingua, a Rust language-detection library that targets short snippets and mixed-language input. It installs from PyPI, supports 75 languages, and needs no network call at runtime.
- Who is it for?
- Adopt lingua-py when your input is short, noisy or multilingual and you need a local, offline detector: install lingua-language-detector from PyPI, restrict the language set with the from_languages constructor, and check the accuracy-reports directory for the language pairs you care about before trusting the output.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 60 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 19, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What lingua-py is for, and where the other detectors fall short
The README states the task plainly: the library tells you which language some text is written in, and it is meant as a preprocessing step for NLP work such as text classification and spell checking. The author also names two use cases outside NLP: routing e-mails to a geographically located customer service desk, and any pipeline where language is an input to a later decision.
The reason the project exists is a specific complaint about the alternatives. The README lists CLD 2, CLD 3, Langid, Simplemma and Langdetect, and says that except for Langdetect they share two drawbacks: detection only works with lengthy text fragments, and accuracy falls as more languages enter the decision. That second point is the more interesting one. A detector asked to choose among five languages has an easier job than one asked to choose among seventy-five, and most libraries do not let you control that trade-off. Lingua does, through a restricted language list passed at construction time.
The intended audience is a Python developer who does not want to pull in a full machine learning framework or learn one, and who wants the detector to run locally. The README is explicit that no neural networks and no word dictionaries are used, and that no external API or service is contacted. Once installed, it works offline.
Rule-based and Naive Bayes, with no network and no dictionaries
Lingua draws on both rule-based and statistical Naive Bayes methods. The rule-based half is what makes single words and short phrases tractable: character n-grams and script information survive where a word-level model has almost nothing to work with. The statistical half handles the longer inputs where frequency evidence accumulates. Neither half calls out to a service, and neither depends on a dictionary of words, which is what allows the library to cover 75 languages without shipping 75 lexicons.
The architecture changed substantially at version 2.0.0. The README describes the history directly. The original pure Python implementation stored language models in dictionaries, which was fast but consumed more than 3 GB of memory. Moving the models to NumPy arrays cut memory to roughly 800 MB but dropped CPU performance significantly. Neither approach satisfied the author. From 2.0.0 onward, the pure Python code was replaced by compiled Python bindings to the native Rust implementation, which the README says delivered both quick performance and a small memory footprint.
That is the trade-off to understand before adopting. You are not installing a Python package in the usual sense. You are installing a wheel that wraps a compiled Rust library, and the repository layout reflects it: the build badge and codecov badge in the README point at the lingua-rs workflow and repository, not at lingua-py. The Python repository is the binding surface.
Installing lingua-py and running a first detection
The package name on PyPI is lingua-language-detector, not lingua-py. That mismatch between the repository name and the distribution name is the first thing to get right. The README badge links to the PyPI page for that distribution and states version 2.2.0.
pip install lingua-language-detectorThe README badge states the supported Python versions as 3.12 and above. The README does not include a full Python usage example, so the API surface described here is limited to what the README names: a LanguageDetector built through a builder, the from_languages constructor, and the confidence-value call. Passing a restricted language set is the mechanism the README credits for accuracy on short text, because fewer candidates mean fewer chances to guess wrong. Build the detector once and reuse it; the language models are the expensive part, and constructing a detector per request is the wrong pattern for a service.
Where lingua-py gets things wrong, and when to pick something else
The README's accuracy section is careful about its own evidence, and that care is worth reading before you trust the numbers. Both the language models and the test data come from separate documents of the Wortschatz corpora from Leipzig University. Training used data crawled from news websites, one million sentences per corpus. Testing used corpora from arbitrarily chosen websites, ten thousand sentences each, with a random subset of 1000 single words, 1000 word pairs and 1000 sentences extracted per language.
That is a clean train/test split, but it is a news-text split. Short social messages, product titles, chat logs and code comments are not news text, and the README does not claim the reported accuracy transfers to them. If your inputs are user-generated, the published figures are a starting point, not a guarantee.
The comparison methodology has a second caveat that the README states itself: languages not supported by the other detectors are ignored for them during detection. So Lingua is measured over 75 languages while the others are measured over fewer. That is a fair way to compare on a shared subset, but it means the headline accuracy numbers are not measuring every detector on identical inputs.
The wrong-tool case is narrower than it first appears. If your text is long, monolingual and drawn from a handful of languages, a smaller detector will do the job and cost less to ship. Lingua's advantage is concentrated in the short and mixed cases. The other clear boundary is the environment: the README says there are environments that do not support native Python extensions, and names Juno as an example. For those, version 2.x will not work at all.
The pure Python branch, and how it differs from the 2.x line
The alternative to lingua-py 2.x is not another library. It is lingua-py 1.x, the pure Python implementation kept on a separate branch in the same repository. The README says that branch will be kept up to date in subsequent 1.* releases and that both 1.* and 2.* versions remain available on PyPI.
The difference in approach is exactly the memory-versus-speed trade-off described above. The 1.x line runs anywhere Python runs, including environments that reject native extensions, at the cost of the memory and CPU behaviour the author spent two major versions trying to fix. The 2.x line gets the Rust performance and the smaller footprint, at the cost of a compiled wheel that has to exist for your platform.
If you compare lingua-py with FastText instead, the distinction is different in kind. FastText is a general text classification toolkit that you train on your own labelled data; Lingua ships pretrained models for a fixed set of 75 languages and is not trained on your corpus. That makes Lingua far quicker to adopt and useless if your labels are not languages, or if your languages are not in the supported list. The README's own comparison set is CLD 2, CLD 3, Langid, Simplemma and Langdetect, and the claimed differentiator against all of them is short-text and many-language accuracy.
Licence, release cadence and what upgrading costs
The project is licensed under Apache-2.0, and pyproject.toml carries the matching classifier and the license field. Apache-2.0 includes an explicit patent grant and permits commercial use, modification and redistribution provided the licence and notices are preserved. That is a permissive arrangement, but it is not legal advice, and if you redistribute the wheel inside a product you should have someone check the notice requirements rather than take this paragraph as clearance.
The release history in the repository shows v2.2.0 on 2026-03-09, v2.1.1 and v1.4.2 both on 2025-05-27. The last push to the default branch was 2026-07-20, which is recent, and the repository is not archived. The project is maintained, though the gap between 2.1.1 and 2.2.0 was roughly nine months, so do not plan around frequent releases.
The upgrade cost is concentrated in the 1.x to 2.x boundary. That move replaced the implementation language underneath the API and changed the packaging from pure Python to compiled bindings. If you are already on 1.x, moving to 2.x is a platform-compatibility decision before it is a code decision: check that a wheel exists for your target before you plan the migration. Within the 2.x line, the README does not document a breaking-change policy, and the RELEASE_NOTES.md file in the repository is where that information would live.
Editorial conclusion
Adopt lingua-py when your input is short, noisy or multilingual and you need a local, offline detector: install lingua-language-detector from PyPI, restrict the language set with the from_languages constructor, and check the accuracy-reports directory for the language pairs you care about before trusting the output. Skip it if you need a pure Python wheel for an environment without native extensions, since version 2.x ships compiled bindings to the Rust implementation; the 1.x line on the pure-python-impl branch is the fallback there. Before rollout, verify two things yourself: how the detector behaves on your own shortest inputs, since the bundled test corpora are drawn from news text, and whether your target platform has a prebuilt wheel, because the README does not document a source-build fallback.
Frequently asked questions
How do I install lingua-py?
Install it from PyPI with pip install lingua-language-detector. The distribution name differs from the repository name, and the README badge lists Python 3.12 and above as supported.
Which languages does lingua-py support?
The README lists 75 supported languages, from Afrikaans to Zulu, including Chinese, Japanese, Korean, Arabic, Hindi and most European languages. The project states its focus is quality over quantity, so the set is smaller than some alternatives.
Does lingua-py work offline, and does it need a model download?
The README states that the library needs no connection to any external API or service and can be used completely offline once downloaded. It also states that it uses no neural networks and no dictionaries of words.
Can I use lingua-py on a single word or a very short phrase?
That is the case the project targets. The README says it yields accurate results on both long and short text, even on single words and phrases, and the bundled test data includes a list of single words with a minimum length of 5 characters.
What changed between lingua-py 1.x and 2.x?
From version 2.0.0 the pure Python implementation was replaced with compiled Python bindings to the native Rust implementation. The pure Python version remains available on a separate branch and in subsequent 1.* releases.
How do I restrict lingua-py to a smaller set of languages?
Build the detector with LanguageDetectorBuilder.from_languages and pass the languages you want considered. The README identifies the number of languages in the decision as a factor in accuracy, so a restricted set is the intended way to improve results.
Community notes