PaddlePaddle/ERNIE: ERNIE 4.5 Models and the ERNIEKit Training Toolkit

What ERNIE 4.5 and ERNIEKit actually cover

The repository holds two things that are easy to confuse. The first is the ERNIE 4.5 model family: ten variants described in the README as Mixture-of-Experts models with 47B and 3B active parameters, a largest model at 424B total parameters, and a 0.3B dense model. The family splits into text-only LLMs (ERNIE-4.5-300B-A47B, ERNIE-4.5-21B-A3B and their Base versions), vision-language models that accept text, image and video input (ERNIE-4.5-VL-424B-A47B and ERNIE-4.5-VL-28B-A3B), and dense models. All of them list a 128K context window.

The second thing is ERNIEKit, the development toolkit in the erniekit/ directory. It is a Python package installed as erniekit, with a console entry point defined in setup.py as erniekit = erniekit.cli:main. The README frames it as an industrial-grade toolkit for training and inference workflows, and the release notes trace its growth: v1.0 in June 2025, then v1.1 adding SFT and LoRA for the ERNIE-4.5-VL series, v1.2 adding WebUI support for 28B and 424B VL models, v1.3 and v1.4 adding SFT and function-call training for the 21B-A3B-Thinking model and PaddleOCR-VL-0.9B, and v1.5 adding the same for ERNIE-4.5-VL-28B-A3B-Thinking.

Who this is for: teams that have already chosen PaddlePaddle and want to fine-tune or serve an ERNIE 4.5 checkpoint without exporting it to another framework. It is not a drop-in inference server for people who just want an API endpoint.

How the toolkit is put together

The top-level layout tells most of the story. ernie/ holds model code, erniekit/ holds the toolkit, data_processor/ handles data preparation, examples/ carries runnable configurations, docker/ carries container definitions, and docs/ carries the training documentation that the README links to as ./docs/erniekit.md. The Makefile defines the check directories as cookbook, data_processor, ernie, erniekit, examples, tools, tests and requirements, which is a fair map of what is maintained.

ERNIEKit's training path is configuration-driven. The examples/configs/ and examples/pre-training/ directories exist for that purpose, and the release notes describe a dataflow component that gained a padding-free strategy in v1.4: packing data within a batch into a sequence to avoid padding, which the notes say reduces GPU memory use and speeds up training. That is a real mechanism, not marketing, and it matters when you are packing 128K-token sequences.

Parallelism is handled by an AutoParallel component. The v1.2 bug fixes mention use_intermediate_api with pp, recompute and moe, and a separate checkpoint saving fix, which tells you the parallel path is under active repair rather than frozen. Hardware support is not CUDA-only: pyproject.toml declares test paths for tests/gpu, tests/iluvatar_gpu, tests/npu and tests/xpu, and the v1.2 notes record iluvatar GPU support arriving in the command-line tool.

Installing ERNIEKit and running a first training job

The repository's own install path is the Makefile target, not a bare pip install. It removes any existing paddlepaddle-gpu, then installs prebuilt PaddlePaddle GPU and FastDeploy wheels from Baidu-hosted URLs, installs requirements/gpu/requirements.txt, and finally installs the package itself in editable mode.

make install

Read that target before running it. It pins a specific CUDA 12.6, cuDNN 9.5, TensorRT 10.5, Python 3.10 wheel, so a machine on a different Python minor version or driver branch will need the URLs adjusted rather than reused. The same target runs pip uninstall paddlepaddle-gpu -y first, which will remove a working installation if you run it on a box you care about.

Once installed, the console script defined in setup.py is available under the name erniekit. Training itself is driven from configuration files rather than long flag lists. The examples/ directory is where the repository keeps those configs, with examples/configs/, examples/data/ and examples/pre-training/ as the named subdirectories, and the README points to ./docs/erniekit.md for the training documentation. Start there: the README does not reproduce a full training command, so the config schema and the checkpoint paths have to come from the docs and the example configs rather than from the front page. For container-based runs, docker/ exists in the tree, but the README does not document the image names or tags.

Where ERNIEKit will disappoint you

The install target is the first wall. It assumes a Linux GPU host with CUDA 12.6, cuDNN 9.5 and TensorRT 10.5, and it fetches wheels from paddle-qa.bj.bcebos.com URLs that are not on PyPI. If those URLs move or your environment does not match, the target fails partway through and leaves you with an uninstalled paddlepaddle-gpu, because the uninstall step runs first. There is no documented rollback in the README.

The second wall is framework lock-in. Everything here is built on PaddlePaddle. The README states that all models are trained with PaddlePaddle and that this enables high-performance inference and deployment, which is also the boundary: if your serving stack is vLLM, TensorRT-LLM or another PyTorch-centric runtime, ERNIEKit's training output is not aimed at you. The README's deploy link points to PaddlePaddle/FastDeploy rather than to a neutral format.

The third is documentation depth. The README is a release-notes feed plus a model table. It does not document the training config schema, the checkpoint conversion path, or how to resume an interrupted run. The docs/ directory is referenced but its contents are not reproduced, so you are reading the repository, not the README, to learn the toolkit. For a 424B-parameter model, that gap is expensive to discover late.

Finally, the model sizes themselves are the limitation. A 424B-total, 47B-active MoE variant is not a single-GPU workload, and the README offers no hardware sizing table. The 0.3B dense model is the only one that reads as laptop-adjacent, and the README gives it the same 128K context claim as the rest without qualification.

The alternative you are probably comparing it to

The obvious comparison is a PyTorch-native fine-tuning stack such as Hugging Face Transformers with PEFT. The difference is not quality, it is where the engineering effort sits. Transformers and PEFT let you load a checkpoint, attach a LoRA adapter and train with a trainer class, and the ecosystem around them is broad because the framework is the default one. ERNIEKit instead assumes PaddlePaddle end to end, with its own AutoParallel layer, its own dataflow packing strategy and its own CLI. You get a tighter integration with ERNIE 4.5 checkpoints and the FastDeploy serving path, and you give up the portability of a PyTorch checkpoint and the surrounding tooling.

A second alternative is not fine-tuning at all. If you only need ERNIE 4.5 inference, the README points at ERNIE Bot and the AI Studio model overview rather than at this repository, and the Hugging Face baidu organization hosts the weights. Cloning this repo to call a hosted model is the wrong use of it. The repository is for people who need to change the weights or run them on their own PaddlePaddle stack.

Licence, maintenance and what an upgrade costs

The repository is Apache-2.0, and the README states that all models are publicly accessible under Apache 2.0. That is permissive for both research and commercial use, but the licence covers the code and the stated model terms, not the third-party wheels the Makefile pulls from Baidu's build servers, which carry their own terms. Nothing here is legal advice; check the wheel sources if redistribution matters to you.

Maintenance is visible and recent. The last push to the default branch release/v1.5 was on 2026-07-24, and the release cadence in the README runs from ernie-4.5 in June 2025 through ERNIEKit v1.5 in November 2025, with v1.1 through v1.4 landing in a single September 2025 window. That is a fast-moving toolkit, and the bug-fix entries in v1.2 (AutoParallel checkpoint saving, LoRA 128k training, the pp+recompute+moe path) are a fair warning that early versions had rough edges in exactly the areas you would hit at scale.

The upgrade cost is the install target. Because make install pins specific wheel URLs and uninstalls paddlepaddle-gpu first, moving between ERNIEKit versions is not a pip upgrade; it is a re-run of a build script whose URLs encode a CUDA, cuDNN and TensorRT combination. Budget for a rebuild, not a patch.

Editorial conclusion

Adopt it if you are already on PaddlePaddle and need to fine-tune an ERNIE 4.5 checkpoint, especially a VL variant, because ERNIEKit is the only supported path the repository documents for that work. Stay away if your stack is PyTorch-native or your hardware is outside the CUDA, XPU, NPU and iluvatar targets the tests and Makefile name, since the install target pulls prebuilt PaddlePaddle and FastDeploy wheels from Baidu-hosted URLs. Before committing, check the requirements/gpu/requirements.txt file against your driver and CUDA version, and confirm whether the checkpoint you want has a documented SFT recipe in docs/ rather than only a Hugging Face listing.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. PaddlePaddle/ERNIE on GitHub
3. Project website
4. README
5. Releases