# Chinese-offensive-language-detect: a harmful and safe paired dataset for Chinese text

> This repository holds a research dataset for detecting six categories of harmful Chinese text, together with the code that fine tunes a Chinese BERT model on it and an ensemble of three detectors with a weighted vote. The pairs are generated from keywords with a large language model and a jailbreak technique, and a pinyin conversion step exists to catch text that hides profanity in homophones.

**royal12646/Chinese-offensive-language-detect** — 本项目旨在通过数据生成技术、模型微调技术、加权投票技术来检测以下6种有害文本：涉黄有害文本、辱骂/谐音辱骂有害文本、地域、性别、种族、职业攻击有害文本

- Repository: https://github.com/royal12646/Chinese-offensive-language-detect
- Stars: 878 · Forks: 59
- Language: Python
- License: not declared
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/royal12646-chinese-offensive-language-detect

## Six categories, and the data is generated rather than collected

The project detects six kinds of harmful Chinese text, and the categories are specific rather than a single notion of abuse. They are sexual content, abusive text including abbreviations, and four kinds of discriminatory text aimed at geography, gender, ethnicity and occupation. What is unusual is how the data was made. The authors use a large language model to generate a keyword for each theme, and then combine the keyword with a model jailbreak technique to generate the data. Each keyword produces ten harmful texts and ten safe texts, which is the pairing that makes the dataset useful, because a keyword alone would let a model learn to spot the keyword rather than the behaviour. The readme says explicitly that the harmful texts in the data are all artificially generated or selected from public corpora, that they do not represent the authors' views, and that the dataset is prohibited for spreading, imitating or inciting hatred or any non-research purpose. Users are asked to read and accept that statement before downloading the data.

## Three datasets, and why the safe half matters

The result is three datasets. SexHarmSet covers the sexual content category. AbuseSet covers the abusive and homophone abuse category. BiasSet covers the four discrimination categories, that is geographic, gender, ethnicity and occupation. Each of them is built on the same keyword plus pair structure, and the design intent is stated in the readme as preventing the model from falling into the trap of the keyword, since the same keyword appears in both the harmful and the safe halves, and a model that learns to key on the word alone gets a fifty percent score on data it has never really understood. The safe examples are written to be genuinely safe uses of the same term, referring to anatomy, to a sports context, to a literary context or to a figurative usage, so the discrimination between the two is about intent and framing rather than vocabulary. The dataset is published as ZHateBench, with a citation in the readme giving the author, the year, the publisher and a DOI on Zenodo, and a plain text form of the same reference for anyone who prefers to paste it.

## Homophone evasion is handled by rewriting the text before detection

Detecting abuse written in homophones is a specific Chinese problem, because the abusive word can be spelled with characters that sound the same but are not the same characters, which defeats a string match and often defeats a model trained on the written form. The solution described is a normalization step that runs before detection. The text is first converted into its pinyin form, and then a Pinyin2Hanzi method converts that pinyin back into characters that do not contain the homophone, so that the text the detector sees is the standard spelling of the abusive word. The Demo directory carries a Pinyin2Hanzi directory, which is where that conversion code lives, and the pypinyin and jieba libraries in the requirements are the pinyin conversion and the Chinese word segmentation. It is a pragmatic normalisation rather than a semantic one, so it will not catch every homophone construction, but it moves a class of evasion back into the range of the classifier.

## Three fine tuned detectors combined by a weighted vote

The detection method starts from a cost argument. Considering training and inference cost, the authors chose to fine tune the open source model hfl/chinese-macbert-base, and from that base model and the three self-built datasets they built three detectors. D1 is the sexual content detector, D2 is the abuse detector, and D3 is the geographic, ethnic and gender bias detector. Each detector handles its own category, and then an ensemble learning step combines them into one general detector. The combination is a weighted vote with learnable parameters. The text goes into each of the three detectors, which each return the probability that the text is harmful, giving p1, p2 and p3, and those probabilities are each multiplied by a learnable parameter alpha1, alpha2 and alpha3. The results are summed, and if the sum is greater than 0.5 the text is judged harmful, otherwise harmless. The learnable weights are the point of the design, since a category that is rare in the data should not get an equal vote with a common one.

## The demo is a Flask service with a separate front end

The Demo directory holds a small application that shows the detectors in use. The backend is a Flask app served on port 5000, and the front end is a separate directory with its own package.json, a Vite configuration and a public directory, so the demo is a client and server pair rather than a single page. The demo instructions are a sequence of shell steps, and the firewall rule for the port comes first because the service is bound locally and the front end has to reach it:

```bash
cd ./Chinese-offensive-language-detect/Demo
sudo ufw allow 5000/tcp
python Flask/app.py
```

The front end is started separately after activating the environment, and the two are in different terminals, which is what you would expect from a Vite development server talking to a Flask backend:

```bash
conda activate off-detect
cd ./Chinese-offensive-language-detect/Demo/
npm run dev
```

The requirements list confirms the shape of the stack, with Flask, flask_cors for the cross origin calls between the two halves, transformers and torchkeras for the model, scikit-learn and tabm for the training, evaluate for the metrics, and dashscope, which is the model service the generation step uses. jieba and pypinyin are there for the segmentation and pinyin work described earlier.

## Setting up the environment, and a warning about torchvision

The environment is a conda environment on Python 3.10, and the setup is three commands:

```bash
conda create -n off-detect python=3.10
conda activate off-detect
pip install -r requirements.txt
```

The instruction after that is worth reading carefully. The PyTorch site is where you configure PyTorch for your machine, and because some of the code has problems, the recommendation is not to install torchvision. The requirements file is pinned exactly, and two of those pins are worth noticing before you install. transformers is pinned to 5.0.0rc2, which is a release candidate rather than a final release, and torch is not pinned at all because you are expected to install the build that matches your hardware from the PyTorch site. The remaining pins are the data stack, with datasets, pandas and evaluate, the model stack with transformers and torchkeras, and the training helpers with scikit-learn, tabm and accelerate. Training and testing are then single scripts, python train.py and python test.py, and the repository layout is small, with a Dataset directory, the Demo directory, the two scripts and the requirements file at the root.

## Conclusion

This repository suits a researcher building Chinese content moderation or testing the robustness of a classifier against homophone evasion, since the paired structure gives you the negative examples that most safety datasets lack. It does not suit a team that wants a ready to run moderation service, because the deliverable is a dataset and a training script rather than a service, and the demo is a Flask and Vite application that has to be run by hand. Before you use it, read the terms of use at the top of the readme, which restrict the dataset to research and engineering purposes and forbid using it for any non-research purpose, and check the pinning in requirements.txt, which includes a release candidate of transformers.

## FAQ

### What categories of harmful text does this dataset cover?

Six: sexual content, abusive text including abbreviations, and discriminatory text aimed at geography, gender, ethnicity and occupation. The last four are grouped into the BiasSet dataset, while sexual content is SexHarmSet and abuse is AbuseSet.

### How was the data generated?

A large language model was used to generate a keyword for each theme, and the keywords were combined with a model jailbreak technique to produce the text. Each keyword yields ten harmful texts and ten safe texts, so the safe examples use the same keyword, which is what stops a model from learning to spot the keyword instead of the behaviour.

### How are the three detectors combined into one result?

The text goes into each of the three fine tuned detectors, which return the probability that it is harmful, and each probability is multiplied by a learnable weight. The weighted results are summed, and a sum above 0.5 means the text is classified as harmful.

### How does the project handle abuse written with homophones?

Before detection, the text is converted into pinyin and then converted back into characters without the homophone through the Pinyin2Hanzi method, so that the detector sees the standard spelling of the word. The conversion code is in the Demo directory, and pypinyin and jieba are the libraries used for the pinyin work and word segmentation.

## Sources

- [Issues](https://github.com/royal12646/Chinese-offensive-language-detect/issues)
- [README](https://github.com/royal12646/Chinese-offensive-language-detect/blob/main/README.md)
- [royal12646/Chinese-offensive-language-detect on GitHub](https://github.com/royal12646/Chinese-offensive-language-detect)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/royal12646-chinese-offensive-language-detect