Fengshenbang-LM: IDEA-CCNL's Chinese Model Family and the Fengshen Training Framework

The gap Fengshenbang-LM was built to fill

The README opens with an argument about language coverage rather than about architecture. It states that foundation models are dominated by the English-speaking community, and that Chinese, despite having the largest number of native speakers, lacks systematic research resources, which it says puts Chinese work behind English work. The project positions itself as an answer to that: a Chinese-driven foundation ecosystem covering pretrained models, task-specific fine-tuned applications, benchmarks and datasets, announced at an IDEA conference on 2021-11-22.

The audience follows from that framing. It is not aimed at teams who want an API endpoint. It is aimed at people who need a Chinese checkpoint they can download and adapt, and at researchers who want the training and fine-tuning scripts that produced those checkpoints. The README also states an explicit contribution model: pick the closest open model, continue training, then release the result back into the system, so that each participant spends less compute than starting from scratch.

How the model families are organized

The repository sorts its models into named families, each tied to a task type and a parameter band. Jiangziya (姜子牙) is the general-purpose large model family, above 7 billion parameters, covering translation, coding, text classification, information extraction, summarization, copywriting, commonsense question answering and arithmetic. Taiyi (太乙) covers multimodal work in the 80 million to 1 billion range, including text-to-image generation, protein structure prediction and speech-text representation. Erlangshen (二郎神) handles language understanding from 90 million to 3.9 billion parameters, and the README claims it was the largest open Chinese BERT at release, with first place on FewCLUE and ZeroCLUE in 2021. Wenzhong (闻仲) handles generation from 100 million to 3.5 billion, Randal (燃灯) handles text-to-text transformation such as translation and summarization from 70 million to 5 billion, and Yuyuan (余元) covers domain models from 100 million to 3.5 billion, including what the README calls the largest open GPT2 medical model at the time.

That table is the most useful page in the project. A reader deciding what to download can match a task type to a parameter budget in one pass. The trade-off is that the families are described in aggregate, and the per-model documentation lives outside the README, in a Read the Docs site and on Hugging Face. The README is an index, not a reference.

The Fengshen framework and its DeepSpeed dependency

Underneath the models sits a training framework, referred to as 封神框架. The dependency list in setup.py is a compact statement of its design: transformers at 4.17.0 or newer, datasets at 2.0.0 or newer, pytorch_lightning at 1.5.10 or newer, deepspeed at 0.5.10 or newer, and jieba plus jieba-fast for Chinese tokenization.

That combination tells you what kind of codebase this is. PyTorch Lightning supplies the training loop abstraction; DeepSpeed supplies the sharding and memory optimizations needed to train models in the billions of parameters. It is a distributed-training stack, and the README's topics list confirms the intent with entries for distributed-training and pytorch. The practical consequence is that the framework is not lightweight. If you only want to run inference on a downloaded checkpoint, you do not need most of this, and the README points such readers to Hugging Face, where it says every Fengshenbang model has been converted and synced so that a few lines of code are enough to use them.

The framework also exposes a command-line entry point, fengshen-pipeline, registered in setup.py as a console script pointing at fengshen.cli.fengshen_pipeline. The README's table of contents lists a Pipelines section under the framework heading, but the visible portion of the README does not reproduce its contents.

Installing from setup.py and running a first model

The package name is fengshen, not fengshenbang, and the version declared in setup.py is 0.0.1. There is no PyPI release mentioned in the README, so the realistic path is installing from the repository itself. Clone it and run pip against the checkout:

git clone https://github.com/IDEA-CCNL/Fengshenbang-LM.git
cd Fengshenbang-LM
pip install -e .

The -e flag installs in editable mode, which is what you want if you intend to modify the training scripts. After this, setup.py declares that the fengshen-pipeline console script becomes available, alongside the six dependencies listed above. Expect the install to pull DeepSpeed and PyTorch Lightning, which are substantial downloads.

For a first real use, the README's own recommendation for the Jiangziya models is to follow the Ziya-LLaMA-13B-v1 model card on Hugging Face rather than the repository README, because the README defers to it. The model card is where the loading snippet lives. What the README does give you is the list of checkpoints to choose from:

Ziya-LLaMA-13B-v1.1
Ziya-LLaMA-13B-v1
Ziya-LLaMA-7B-Reward
Ziya-LLaMA-13B-Pretrain-v1
Ziya-BLIP2-14B-Visual-v1

Those names are copied from the README's Jiangziya section. The v1.1 checkpoint is the one the README links first, and the README states that the v1 and v1.1 general models completed three training stages: large-scale pretraining, multi-task supervised fine-tuning, and human feedback learning. For fine-tuning your own data, the README points to the ziya_llama directory under fengshen/examples; for quantized inference, it points to a separate inference example whose path is cut off in the README text. If you cannot find that directory in the checkout, the README is the wrong place to look.

Where Fengshenbang-LM is the wrong choice

The clearest limitation is temporal. The README's news section, which is the project's own record of activity, ends with an entry dated 2023-06-05 announcing the multimodal Ziya release and the first volume of the Jiangziya vertical capability series. The repository's last push was on 2026-06-08, so the code has moved since the news stopped, but the README no longer describes the current state of the project. Anyone reading it as a status document will be working from a description that is roughly three years behind.

A second limitation is the packaging. The repository's LICENSE file is Apache-2.0, and the README's badge says Apache 2, but setup.py declares license="MIT Licence". Those are different licences with different patent and attribution terms, and the discrepancy is unresolved in the README. That is a reason to read LICENSE directly rather than trust either the badge or the setup metadata.

Third, the framework is heavy for what many users actually need. If your goal is to call a Chinese model, the DeepSpeed and PyTorch Lightning dependencies are dead weight, and the README itself directs you to Hugging Face for that case. The framework only earns its install cost if you are training or fine-tuning at scale.

Fengshenbang-LM against a single-model release

The obvious alternative approach is to skip the umbrella project entirely and use one model card. A team that needs a Chinese generative model can load Ziya-LLaMA-13B-v1.1 from Hugging Face with transformers alone, at version 4.17.0 or newer, and never install fengshen, DeepSpeed, or PyTorch Lightning. The difference is not quality; it is scope. Fengshenbang-LM is an aggregation project. Its value is that it collects model families across understanding, generation, transformation, multimodal and domain tasks under one naming scheme and one documentation index, and that it ships the training code that produced them.

That aggregation is also its cost. A single model card has one maintainer, one dependency set and one update cadence. An umbrella project inherits the release rhythm of every family inside it, and the README shows those families were released at different times: Erlangshen's benchmark results are from 2021, the Chinese Stable Diffusion announcement is from 2022-11-02, and the Jiangziya vertical series is from 2023. If you only need one of those, the umbrella adds surface area without adding capability.

Licence and the cost of staying current

The repository is Apache-2.0 per its LICENSE file and the README badge, which permits commercial use and requires attribution and notice retention. The conflicting MIT declaration in setup.py means you should treat the LICENSE file as the governing text and have your own counsel confirm, since this article is not legal advice. Apache-2.0 also includes an explicit patent grant, which MIT does not, so the distinction is not cosmetic for a company shipping a product.

Upgrade cost is the harder question. The dependency floors in setup.py are minimums, not pins: transformers >= 4.17.0, datasets >= 2.0.0, pytorch_lightning >= 1.5.10, deepspeed >= 0.5.10. Installing today will resolve to current versions of all four, and PyTorch Lightning in particular has changed its API across major versions since 1.5. The repository declares version 0.0.1 and the README shows no releases, so there is no changelog to consult when something breaks. Budget time for pinning your own versions rather than relying on the declared floors.

Editorial conclusion

Adopt Fengshenbang-LM if you need Chinese-language checkpoints you can pull from Hugging Face and fine-tune, or if you want a PyTorch Lightning plus DeepSpeed training scaffold to read and adapt. Do not adopt it expecting a maintained product: the last push to the repository was on 2026-06-08, and the README's own news list stops at 2023-06-05. Before you commit, check that the specific model you want still exists on the IDEA-CCNL Hugging Face organization, and read the LICENSE alongside the MIT declaration in setup.py, since the two do not agree.

Frequently asked questions

Official sources

1. IDEA-CCNL/Fengshenbang-LM on GitHub
2. Issues
3. License: Apache-2.0
4. README