Two Chinese XLNet checkpoints, best-of-ten numbers, and a badge pointing elsewhere

The license badge points at a repository that is not this one

The badge at the top of the Chinese README links to `ymcui/Chinese-PreTrained-XLNet/blob/master/LICENSE`, while the repository you are reading is `ymcui/Chinese-XLNet`. So the one element on the page that is supposed to point at the license text points at a neighbouring name instead. Two other details sit in the same header. The page offers a language pair, a Chinese `README.md` and an English `README_EN.md`, and everything substantive in this article is written from the Chinese one. The declared homepage is `http://xlnet.hfl-rc.com`, plain HTTP rather than HTTPS. The sibling-model strip near the top lists the lab's other work, from LERT and PERT through MacBERT, Chinese-ELECTRA, Chinese-BERT-wwm, TextBrewer and TextPruner, and one of its eight links points back at Chinese-XLNet itself. The repository tree holds seven entries, of which the interesting ones are `src/`, `pics/` and the two READMEs.

Two checkpoints, two channels, and the PyTorch weights come second

There are two models. `XLNet-mid` is 24-layer, 768-hidden, 12-heads, 209M parameters; `XLNet-base` is 12-layer, 768-hidden, 12-heads, 117M parameters. Both were trained on Chinese Wikipedia plus general data that the page says totals 5.4B words, the same corpus as the lab's BERT-wwm-ext. Each is offered twice. HuggingFace hosts PyTorch weights under `hfl/chinese-xlnet-mid` and `hfl/chinese-xlnet-base`, and a Baidu netdisk link hosts the TensorFlow checkpoint, with the extraction password written into the URL itself: 2jv2 for mid and ge7w for base. The PyTorch route is the derived one. The page tells you to convert the checkpoint yourself with a Transformers conversion script, or to download the PyTorch weights from the HuggingFace site by opening a model, scrolling to the bottom, clicking List all files in model, and taking the bin and json files. The TensorFlow archive is about 800M and unpacks into five files: `xlnet_model.ckpt`, `xlnet_model.meta`, `xlnet_model.index`, `xlnet_config.json` and `spiece.model`.

Every score is the best of ten seeds, and the page expects your run to match

The measurement protocol is stated plainly, and it is worth reading before any table. For a given model the authors ran ten trials with different random seeds and report the maximum and the mean, with the mean in parentheses. Two consequences follow. The headline figure in each cell is a maximum over ten draws, which is a higher bound on the model's quality than a single run would give, and the spread between the two numbers is large enough to matter: on the CMRC 2018 challenge set, XLNet-base reads 25.0 outside and 22.7 inside, a gap of 2.3 points on one metric. The page then says that unless something unexpected happens, your own result will very probably land inside this interval. That does not follow from reporting maxima, and readers who benchmark against these numbers with one seed should expect to sit below them. The comparison has a second layer: the BERT, BERT-wwm and BERT-wwm-ext figures were taken from the Chinese-BERT-wwm project rather than measured here.

On the sentiment task XLNet ties BERT at the top and loses on the mean

ChnSentiCorp is the shortest table and the most revealing. It classifies text as 积极 or 消极, and it reports accuracy. The BERT row reads 95.0 (94.7) on the test set, BERT-wwm reads 95.4 (95.0), and XLNet-mid reads 95.4 (94.9). The two 95.4 values are the same number and both are marked as best results, while the means underneath them point the other way: 95.0 for BERT-wwm against 94.9 for XLNet-mid. So on the one task where the headline ties, the average run of BERT-wwm is ahead. The table also has a hole in it. The XLNet-base row carries no development figure and no test figure at all, so a reader looking for the smaller model's sentiment result finds an empty line between BERT-wwm and XLNet-mid, and the accompanying note says only that time and energy were limited and that more task categories were not covered.

On CMRC 2018, XLNet-mid wins F1 and loses exact match

CMRC 2018 is extractive reading comprehension scored with EM and F1, and it is where the two metrics disagree. On the test set XLNet-mid reports 69.3 (68.5) EM against 87.7 (87.0) for BERT-wwm-ext, while the F1 column reads 89.2 (88.8) for XLNet-mid against 87.7 (87.0) for BERT-wwm-ext. In other words, XLNet-mid extracts spans that overlap the answer more often and match the answer string exactly less often, and BERT-wwm-ext wins the exact-match column by two points on both the maximum and the mean. The development set repeats the pattern at smaller size, with BERT-wwm-ext's EM maximum of 67.1 above XLNet-mid's 66.8. The challenge set is where the ordering flips hard: XLNet-mid reaches 29.1 (27.1) EM and 55.8 (54.9) F1 against BERT-wwm-ext's 24.0 (20.0) and 47.3 (44.6). On traditional Chinese, DRCD, XLNet-mid leads both columns of both splits.

The 32k vocabulary is the official recipe with one number changed

The tokenizer is a SentencePiece model of 32000 units, and the page is explicit that every other parameter is the default from the official XLNet example. The command is reproduced in full:

spm_train \
	--input=wiki.zh.txt \
	--model_prefix=sp10m.cased.v3 \
	--vocab_size=32000 \
	--character_coverage=0.99995 \
	--model_type=unigram \
	--control_symbols=\<cls\>,\<sep\>,\<pad\>,\<mask\>,\<eod\> \
	--user_defined_symbols=\<eop\>,.,\(,\),\",-,–,£,€ \
	--shuffle_input_sentence \
	--input_sentence_size=10000000

Two of those lines are worth pausing on. `character_coverage=0.99995` keeps all but a fraction of a percent of character types, and the user-defined symbol list carries punctuation, a hyphen, an en dash and two currency signs, which tells you what the raw corpus looked like. The model prefix follows the naming of the English vocabulary, and the input file is a single `wiki.zh.txt`, so reproducing this means rebuilding the corpus the same way first. There is no published vocabulary file in the repository; the trained `spiece.model` arrives inside the checkpoint archive.

Pretraining is twenty passes over 512-token segments, bidirectional

The corpus is turned into training records by `data_utils.py`, with the raw text shaped as in the original tutorial: one sentence per line, and a blank line marking the end of a document. The invocation is given as a template, with `num_task` and `task` left for the reader to set according to how many slices the data was cut into:

SAVE_DIR=./output_b32
INPUT=./data/*.proc.txt

python data_utils.py \
	--bsz_per_host=32 \
	--num_core_per_host=8 \
	--seq_len=512 \
	--reuse_len=256 \
	--input_glob=${INPUT} \
	--save_dir=${SAVE_DIR} \
	--num_passes=20 \
	--bi_data=True \
	--sp_path=spiece.model \
	--mask_alpha=6 \
	--mask_beta=1

The numbers describe a large but ordinary setup: 32 sequences per host across 8 cores, 512-token segments where the second half reuses the first 256 tokens, and twenty passes over the data with `bi_data=True`, which is the permutation-language-model setting rather than the masked one. The listing ends there, mid-argument, so whatever flags follow it in the original recipe are not on this page. `mask_alpha=6` and `mask_beta=1` set the span lengths for the corruption schedule.

The news section stopped in 2023 while commits continued

The chronology at the top of the page ends at 2023/3/28 with the announcement of the Chinese LLaMA and Alpaca models. Before that sit entries for LERT in October 2022, PERT in March 2022, TextPruner in December 2021, CINO in October 2021, a book published in July 2021, and a 2021/1/27 note that all models support TensorFlow 2 and should be reached through the Transformers library. Further back, in the collapsed history, 2019/12/19 marks the day these models were wired into Huggingface-Transformers and 2019/9/5 the day XLNet-base became downloadable. The default branch's most recent commit is dated April 19, 2026, and the repository publishes no GitHub releases, so there is no versioned artifact to point at. The quick-load recipe is pinned to a much older library than any of this: it is written against Huggingface-Transformers 2.2.2 and consists of an `AutoTokenizer.from_pretrained` line and an `AutoModel.from_pretrained` line, with the model name set to `hfl/chinese-xlnet-mid` or `hfl/chinese-xlnet-base`.

Editorial conclusion

Take these checkpoints if you specifically need a Chinese permutation-language-model encoder, and load them through the HuggingFace identifiers, because the netdisk route is a TensorFlow checkpoint that you would have to convert yourself. Treat every number in the tables as an upper bound: each cell is the best of ten random seeds, the BERT baselines were copied from a different project, and on ChnSentiCorp the XLNet-mid mean is below BERT-wwm's even though both are marked best at 95.4, with no XLNet-base figure at all. Before you build on the sentencepiece vocabulary, note that it was trained with the official defaults apart from its size, and that the pretraining command in the documentation stops mid-flag. The default branch's last commit is dated April 19, 2026, and the news section stops in 2023.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. ymcui/Chinese-XLNet on GitHub