Detoxify: Jigsaw toxicity models behind a three-line Python API

What Detoxify actually predicts, and who needs it

The library wraps trained models for three separate Kaggle challenges: Toxic Comment Classification (2018, Wikipedia comments), Unintended Bias in Toxicity Classification (2019, Civil Comments) and Multilingual Toxic Comment Classification (2020). Each challenge produced a model name you pass to the constructor: original, unbiased and multilingual. The README also lists two smaller Albert-based variants, original-small and unbiased-small, which trade some accuracy for size; the README reports original-small at a mean AUC of 98.28 against 98.64 for the full original model.

The audience is narrow and the README says so. The intended use is described as research purposes, fine-tuning on carefully constructed datasets, or helping content moderators flag harmful content faster. If you are building a comment queue, a dataset labelling pass, or an evaluation harness for a larger moderation system, this fits. If you want a drop-in service that decides what gets published, it does not, and the README's own limitation section explains why: comments containing swearing or profanity are likely to be scored toxic regardless of tone or intent, which the authors note can bias results against already vulnerable groups.

How the prediction pipeline works

The public surface is one class, Detoxify, imported from the detoxify package. You construct it with a model name and optionally a device, then call predict with either a single string or a list of strings. The README shows both forms, and the list form is the one that matters for throughput because it lets the underlying PyTorch model batch the inputs rather than running one comment at a time.

The dependencies in pyproject.toml tell you what happens underneath: transformers, torch and sentencepiece. The repository layout confirms the training side is separate from inference, with train.py, configs/, preprocessing_utils.py and model_eval/ at the top level, and detoxify/ as the installed package. Training additionally needs the Kaggle API to download the competition data, which is why the README lists it separately from the inference dependencies. The multilingual model was retrained on translated data from the second Jigsaw challenge plus the first, and the release notes report a test AUC of 92.11 against 89.71 before that update. That model returns the same categories as the unbiased model, and all models were standardised so the original model's identity_hate label became identity_attack.

Installing Detoxify and scoring your first comments

Installation is a single pip command. The package requires Python 3.9 up to but not including 3.13, so a 3.12 environment is fine and a 3.13 environment is not.

pip install detoxify

Once installed, construct a model and score one string. The README's quick prediction section uses exactly this shape, and the return value is a mapping of label to score.

from detoxify import Detoxify

results = Detoxify('original').predict('example text')
print(results)

For a batch, pass a list. This is the form you want in any real pipeline, because it avoids paying model overhead per comment.

results = Detoxify('unbiased').predict(['example text 1', 'example text 2'])

To run on a GPU, pass the device explicitly. The README notes that the default is cpu and that any torch.device input is accepted.

model = Detoxify('original', device='cuda')

The multilingual model accepts a list mixing the seven languages it was trained on. The README gives this example directly.

results = Detoxify('multilingual').predict(['example text', 'exemple de texte', 'texto de ejemplo', 'testo di esempio', 'texto de exemplo', 'örnek metin', 'пример текста'])

If you want readable output rather than a raw dict, the README suggests pandas, which is an optional install, not a runtime dependency of the library itself.

Where the multilingual model is weakest

The README publishes a per-language breakdown for the multilingual model, and it is worth reading before you assume "multilingual" means uniform quality. Turkish scores 97.19 and Spanish 92.74, while Italian sits at 89.18 and French at 89.61. That is a spread of roughly eight AUC points across languages in the same model. If your traffic is mostly Italian or French, the headline multilingual number is not the number you will experience.

The README is explicit that the multilingual model should only be tested on English, French, Spanish, Italian, Portuguese, Turkish or Russian. Anything outside that set is out of scope, and the documentation does not describe how the model behaves on unsupported languages. There is also no per-language model selection guidance: you pick multilingual and accept the subgroup scores as published.

The research-only boundary and the profanity trap

The most important sentence in the README is the limitation about profanity. Words associated with swearing, insults or profanity make a comment likely to be classified as toxic regardless of tone or intent, and the authors give humorous or self-deprecating use as the example. This is a documented failure mode, not a bug you can configure away. A community that swears affectionately will generate false positives at a rate the library cannot tell you in advance.

The README links three papers on bias in toxicity detection, which is a signal that the authors treat this as a known research problem rather than a solved one. It also states the intended use is research, fine-tuning, or moderator assistance. That framing matters for procurement: there is no uptime commitment, no rate limit, no moderation dashboard, and no per-tenant isolation described anywhere in the repository. The API mentioned in the project description is accessed by emailing contact@unitary.ai, so it is a conversation, not a self-serve endpoint. If you need a service with an SLA, this library is not it.

Detoxify versus calling a hosted moderation API

The practical alternative for most teams is a hosted content-moderation endpoint from a cloud provider. The difference in approach is structural rather than a matter of accuracy. A hosted API gives you a network call, provider-side model updates you do not control, request quotas and a bill that scales with volume. Detoxify gives you model weights that run locally, no per-request cost, and full control over when the model changes, at the price of managing the environment yourself.

That control cuts both ways. With a hosted API, a provider can improve the model and you inherit the change. With Detoxify, upgrading from version 0.5.2 to 0.5.3 is your decision and your regression test. The flip side is data residency: comments scored locally never leave your infrastructure, which for moderation of sensitive user content is often the deciding factor. If you are scoring millions of comments a day and already run GPU capacity, local inference is the cheaper path. If you score a few thousand a month and have no ML infrastructure, a hosted endpoint removes an entire dependency chain of torch, transformers and sentencepiece.

Maintenance, releases and the Apache-2.0 licence

The repository is not archived and the last push was on 2026-07-06. The most recent release is v0.5.3 from 2026-03-26, following v0.5.2 in 2024 and v0.5.1 in 2022. That release cadence is uneven: a two-year gap between 0.5.1 and 0.5.2, then a shorter gap to 0.5.3. Plan for the possibility of long stretches without updates, and pin your version rather than tracking the latest tag in production.

The upgrade cost is mostly environmental. pyproject.toml pins torch to version 2 or above, transformers to 3 or above, and sentencepiece to 0.1.94 or above, with a Python ceiling below 3.13. A major torch or transformers release can change inference behaviour even when the Detoxify version does not, so a lockfile is the honest way to keep scores reproducible. The licence is Apache-2.0, which is permissive and includes an explicit patent grant; the classifier in pyproject.toml confirms the OSI-approved Apache Software License. The trained weights are distributed through the same package, and the README does not add a separate weights licence, so treat the code and the weights as arriving under the same terms. That is a summary of what the repository states, not legal advice.

Editorial conclusion

Adopt Detoxify if you need a fast, offline toxicity baseline for English comments, or a multilingual classifier covering English, French, Spanish, Italian, Portuguese, Turkish and Russian, and you can accept that outputs are probabilities from a research model rather than moderation decisions. Do not adopt it if you need a hosted API with an SLA, per-language coverage outside those seven, or a model trained on your own community's demographics. Before you commit, check three things: whether the class names your code expects match the standardised names (identity_attack rather than identity_hate), whether Python 3.12 is enough for your runtime or you need 3.9 to 3.12 only, and whether the pip package version 0.5.3 matches the weights you intend to ship.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. unitaryai/detoxify on GitHub