Chinese-ELECTRA: HFL's Chinese Pre-trained ELECTRA Models for PyTorch and TensorFlow
Pre-trained Chinese ELECTRA（中文ELECTRA预训练模型）
At a glance
- What is it?
- A review of ymcui/Chinese-ELECTRA, the HFL repository that publishes Chinese ELECTRA discriminators from 12M to 324M parameters, how to load them through Hugging Face Transformers, and where the small checkpoints stop being the right tool.
- Who is it for?
- Adopt Chinese-ELECTRA if you need a Chinese encoder that runs on modest hardware and you have labelled task data: the 12M-parameter small discriminator is the reason to pick this family over BERT-base. Do not adopt it if you need a generative model or if you have no fine-tuning set, because the repository distributes weights, not a ready task model.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 150 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What Chinese-ELECTRA actually ships
The repository is a model distribution, not a training framework you are meant to run from scratch. HFL trained ELECTRA on large Chinese corpora and published the resulting discriminators at several sizes. The model list is explicit: ELECTRA-large at 24 layers, 1024 hidden units, 16 heads and 324M parameters; ELECTRA-base at 12 layers, 768 hidden, 12 heads and 102M parameters; ELECTRA-small-ex at 24 layers, 256 hidden, 4 heads and 25M parameters; and ELECTRA-small at 12 layers, 256 hidden, 4 heads and 12M parameters. There are three data regimes. The 180g line was trained on 180G of data including CommonCrawl additions; the base line used about 20G; and a legal line was trained on large-scale judicial documents. The training description states 5.4B tokens for the general models, matching the RoBERTa-wwm-ext series, and the vocabulary follows Google's original BERT WordPiece scheme.
The intended user is an engineer with a Chinese classification, reading-comprehension or tagging task who wants an encoder that fits on one GPU. The README makes the core claim directly: ELECTRA-small is comparable to BERT-base and other models of similar scale while using one tenth of BERT-base's parameters. That ratio, not the large model, is the reason this repository exists.
Generator and discriminator: why only half the model survives
ELECTRA's pretraining framework has two parts. A small masked language model, the generator, predicts the original token at [MASK] positions and uses those predictions to replace some words in the input. A discriminator then judges, for every token in the sentence, whether it was replaced. This is the Replaced Token Detection task, and it replaces BERT's Masked Language Model objective. The README also notes that Next Sentence Prediction is not used here.
The consequence for anyone downloading weights is the part people miss: after pretraining, only the discriminator is kept as the base model for downstream fine-tuning. That is why every Hugging Face link in the repository ends in `-discriminator`. You are not getting a generator to fill in blanks. You are getting an encoder whose training signal came from dense, per-token supervision rather than the sparse 15 percent masking of BERT. The repository's own configuration directory carries both `discriminator.json` and `generator.json`, and the archive layout shows both files alongside `vocab.txt`, but the downstream path uses the discriminator config.
Getting the weights: Hugging Face, Baidu Netdisk and PaddleHub
There is nothing to compile. The repository's quick-loading section points at three routes: Hugging Face Transformers for PyTorch users, PaddleHub for PaddlePaddle users, and the Baidu Netdisk TensorFlow archives. The README does not print a pip command or a loading snippet for any of them; it gives the download table and the file layout, and the checkpoint names are what you carry into whichever loader you already use.
Every 180g entry in the table has a Hugging Face link of the same shape. The small one is:
https://huggingface.co/hfl/chinese-electra-180g-small-discriminatorThe base, small-ex and large entries follow the same pattern with `base`, `small-ex` and `large` in place of `small`, and the 20G-data line drops the `180g` segment entirely. The legal line uses the `chinese-legal-electra-*-discriminator` naming. That string is what you pass to a Transformers loader; the README states only that PyTorch users can load the models through Transformers, so the loader call itself comes from that library's documentation rather than from this repository.
The TensorFlow archives live on Baidu Netdisk, each with its own extraction password listed in the table. The README advises users inside mainland China to prefer the Baidu links and users outside to prefer the Google links. Unpacking the small archive gives this layout:
chinese_electra_small_L-12_H-256_A-4.zip
|- electra_small.data-00000-of-00001 # 模型权重
|- electra_small.meta # 模型meta信息
|- electra_small.index # 模型index信息
|- vocab.txt # 词表
|- discriminator.json # 配置文件：discriminator（若没有可从本repo中的config目录获取）
|- generator.json # 配置文件：generator（若没有可从本repo中的config目录获取）If a config file is missing from an archive, the README says it can be copied from the `config` directory in the repository. Note what is not in that list: no fine-tuned task head, and no tokenizer files beyond the raw `vocab.txt`.
The small checkpoint is the whole argument, and it has a ceiling
The 12M-parameter small model is the strongest reason to choose this family. It is roughly a tenth of BERT-base's parameter count while the README describes its performance as comparable to BERT-base and peers of that scale. For inference on CPU, for edge deployment, or for a team that wants to fine-tune several task heads without renting a large GPU, that is a real difference in cost.
The ceiling is on the other side of the same trade. A 12M-parameter encoder has limited capacity for tasks that depend on long-range reasoning over long documents, and the repository does not publish task-specific fine-tuned checkpoints. Everything here is a base model. If your task needs a model that already answers questions or extracts entities out of the box, this repository does not provide it; you supply the labelled data and the training loop. The size ladder is also not smooth: small and small-ex share 256 hidden units but differ in depth (12 versus 24 layers), while base jumps to 768 hidden. Choosing between small-ex and base is a real decision, and the README's baseline tables are where that comparison has to be made rather than guessed.
One more boundary worth stating plainly: the repository is not archived and its last push was on 2026-04-19, so the code has seen recent activity, but the model releases described in the news section date back to 2020. The weights are stable; do not expect new pretraining runs.
Chinese-BERT-wwm and MacBERT as the alternatives
The natural comparison is HFL's own Chinese-BERT-wwm, which the README cites as using the same 5.4B-token training scale. The difference is the objective. BERT-wwm predicts masked tokens, with whole-word masking, and keeps the encoder as the only component. Chinese-ELECTRA replaces that with Replaced Token Detection, training a discriminator to label every token as original or replaced. The practical effect is that ELECTRA's training signal is denser per example, which is the mechanism behind the claim that a much smaller discriminator can match a larger BERT.
MacBERT, also from the same group, is the other sibling worth knowing. It stays within the masked-language-model family but changes what gets masked, using similar-word substitution instead of the [MASK] token. If your pipeline already depends on a BERT-shaped checkpoint and you only want a better Chinese encoder, MacBERT is a drop-in swap. If your constraint is model size at inference, Chinese-ELECTRA's small discriminator is the more direct answer, because that is the axis it was designed around. The three repositories share tooling and vocabulary conventions, so moving between them is mostly a checkpoint-string change.
Licence, upgrade cost and what to check before you commit
The repository is licensed under Apache-2.0, and the README carries the standard Apache badge. That is a permissive licence, but it governs the repository's code and configuration, not automatically every pretrained weight you download from a third-party host. The weights are distributed through Hugging Face and Baidu Netdisk, and the terms attached to those checkpoints are the ones that matter for commercial use. Read the model card for the specific checkpoint you plan to ship. This is not legal advice; if the distinction matters to your organisation, get it reviewed.
Upgrade cost is low in one sense and non-zero in another. Because the models load through the standard Transformers interfaces that the README points to, a Transformers version bump is the only moving part, and the checkpoint format is the widely supported one. The non-zero part is the vocabulary. The archives ship their own `vocab.txt`, and the README states the vocabulary follows Google's original BERT WordPiece scheme. If you tokenize with a different vocabulary than the one the weights were trained against, fine-tuning results will be quietly wrong rather than loudly broken. Use the `vocab.txt` from the same archive as the weights.
The repository also includes training and evaluation scripts at the top level, including `run_pretraining.py`, `run_finetuning.py`, `configure_pretraining.py`, `configure_finetuning.py` and `cmrc2018_drcd_evaluate.py`, so pretraining from scratch and CMRC2018/DRCD evaluation are both supported paths if you want to reproduce rather than consume.
Editorial conclusion
Adopt Chinese-ELECTRA if you need a Chinese encoder that runs on modest hardware and you have labelled task data: the 12M-parameter small discriminator is the reason to pick this family over BERT-base. Do not adopt it if you need a generative model or if you have no fine-tuning set, because the repository distributes weights, not a ready task model. Before committing, verify on the Hugging Face model card which checkpoint matches your domain, confirm the discriminator config loads in your Transformers version, and check that your tokenizer treats the vocab.txt shipped in the archive as the one the weights expect.
Frequently asked questions
What is Chinese-ELECTRA 180g?
It is the version of HFL's Chinese ELECTRA models trained on 180G of data, which the README says includes high-quality CommonCrawl additions on top of the earlier corpus. The 180g line covers large, base, small-ex and small discriminators, and each has its own Hugging Face checkpoint name.
How do I load Chinese-ELECTRA with Hugging Face Transformers?
The README states that PyTorch users can load the models through Transformers, and the download table gives the checkpoint names, such as hfl/chinese-electra-180g-small-discriminator. The loading call itself comes from the Transformers documentation rather than from this repository.
Why are all the Chinese-ELECTRA checkpoints named discriminator?
ELECTRA pretraining uses a generator to replace some tokens and a discriminator to detect which tokens were replaced. The README states that after pretraining only the discriminator is used as the base model for downstream fine-tuning, so the published weights are discriminators.
Community notes