Paddle Lite: a C++ inference runtime for Arm, NPU and edge accelerators
PaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge (飞桨高性能深度学习端侧推理引擎）
At a glance
- What is it?
- Paddle Lite is Baidu's Apache-2.0 inference engine for phones, embedded Linux boards and edge accelerators. The interesting part is not the runtime itself but the opt tool that rewrites a Paddle model before it ever reaches the device.
- Who is it for?
- Adopt Paddle Lite if your model already comes out of PaddlePaddle, or if you are willing to run it through X2Paddle first, and your target is an Arm CPU, an Android or iOS device, or one of the NPUs listed in the README. Do not adopt it if you need a runtime that ingests ONNX directly, or if you want an accelerator backend that is not on that list.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 142 days ago.
- What is it written in?
- Mainly C++, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem Paddle Lite exists to solve
Training frameworks produce graphs full of operators that no phone can execute efficiently. Paddle Lite sits at the other end of that pipeline. It is a C++ inference engine whose stated targets are mobile, embedded and edge hardware, and the README lists Android, iOS, embedded Linux, Windows, macOS and Linux hosts as supported platforms. The intended user is an engineer who has a trained model and needs it running on a device with a fixed memory budget and no Python interpreter.
The project is not a general-purpose runtime that accepts anything. The README is explicit that Paddle Lite directly supports models produced by PaddlePaddle, specifically those saved through the save_inference_model API. Models from Caffe, TensorFlow or PyTorch are handled by converting them to PaddlePaddle format with X2Paddle first. That single sentence defines the audience: teams already inside the Paddle ecosystem, or teams willing to enter it at the conversion step.
Model optimization happens offline, in the opt tool
The design choice that separates Paddle Lite from a plain interpreter is that most of the work is done before deployment. The README describes an opt tool that performs quantization, subgraph fusion and kernel selection, producing a smaller model that uses fewer resources and runs faster. The same tool also prints the operator inventory of a model and reports which operators are supported on a given hardware platform.
That offline step is the real interface of the project. If opt cannot handle an operator, you find out on your workstation rather than on a device in the field. The trade-off is a two-artifact workflow: the model you trained is not the model you ship, and the optimized file is tied to the optimization decisions you made. The README points to a dedicated model optimization page for downloading and using the tool, so the exact flags are not reproduced in the repository front page. Anyone evaluating this should read that page before assuming a default optimization level is appropriate.
Getting a model onto a device: the four documented steps
The README lays out a four-step flow. First, produce a model with save_inference_model, or convert an existing one with X2Paddle. Second, run the opt tool to optimize it. Third, obtain the runtime, either by downloading a prebuilt prediction library for Android, iOS, x86 or macOS, or by compiling from source. Fourth, write the integration against the C++, Java or Python API, using the per-platform examples as a starting point.
For compilation, the README recommends a Docker-based unified build environment rather than assembling the toolchain by hand, while noting that per-architecture instructions exist for host and target combinations. The prebuilt libraries are described as the preferred route. The repository organizes examples by backend, with separate guides for Arm CPU, x86, OpenCL, Metal, Huawei Kirin NPU, Huawei Ascend NPU, Kunlunxin XPU and XTCL, Qualcomm QNN, Cambricon MLU, Verisilicon TIM-VX, Android NNAPI, MediaTek APU, Imagination NNA, Intel OpenVINO and Eeasytech NPU. The API surface is small enough that the examples are the practical documentation.
The backend matrix is the most useful page in the project
The continuous integration table in the README is more informative than the feature list above it. It shows which combinations are built: CPU 32-bit and 64-bit across x86 Linux, Arm Linux, Android and iOS; OpenCL on Android only; Metal on iOS only; Kirin NPU on Android; Ascend NPU, Kunlunxin XPU and XTCL on x86 and Arm Linux; Qualcomm QNN, Android NNAPI and MediaTek APU on Android; Cambricon MLU on x86 Linux; Verisilicon TIM-VX on Arm Linux and Android; Imagination NNA on Arm Linux; Intel OpenVINO on x86 Linux.
Read that as a support contract, not a marketing list. An accelerator backend that is only built for Android is not a Linux option, and a backend built only for x86 Linux is a server-side deployment, not an edge one. The blank cells are the answer to most feasibility questions, and they are stated plainly in the repository rather than buried in release notes.
Where Paddle Lite is the wrong tool
The clearest limitation is the model format. Paddle Lite consumes PaddlePaddle inference models. If your production pipeline is ONNX-first, or your models come from a framework whose exporter you do not want to replace, the conversion step through X2Paddle is an extra stage that can fail on unusual operators, and each failure costs debugging time on a graph you did not write.
A second limitation is the accelerator list itself. It is long, but it is a fixed list. If your target is a chip that is not named in the README, there is no generic plugin path documented on the front page. You are on the CPU path, or you are writing a backend.
A third point worth stating plainly: the release cadence visible in the repository is uneven. The most recent release listed is v2.14-rc from July 2024, marked as a release candidate, preceded by v2.13-rc in March 2023 and the v2.12 stable release in November 2022. The develop branch shows later activity, but anyone planning around tagged releases should look at the actual release history rather than assume a steady stable cadence.
How this differs from ONNX Runtime and TensorFlow Lite
The nearest alternatives approach the problem from the opposite direction. ONNX Runtime is built around the ONNX graph format as the interchange point, which means a PyTorch or TensorFlow model can often be exported straight into it without passing through a second training framework. Paddle Lite instead treats PaddlePaddle's own inference format as the input and offers X2Paddle as the bridge for everything else.
TensorFlow Lite has a comparable story on the other side: it is the deployment path for TensorFlow models, with its own converter and its own delegate mechanism for accelerators. The architectural difference that matters is where the conversion boundary sits. Paddle Lite puts an optimization tool between the training framework and the runtime, and that tool is the component that decides quantization, fusion and kernel choice. ONNX Runtime and TensorFlow Lite both have graph optimizers, but they are oriented around their own format being the common one. If your models are already Paddle models, Paddle Lite removes a conversion hop that the alternatives would require.
Licence and maintenance cost
Paddle Lite is licensed under Apache-2.0, which the repository states in its badge and LICENSE file. That is a permissive licence with a patent grant, and it does not impose copyleft obligations on the surrounding application. This is a factual description of the licence identifier, not legal advice; teams with distribution or patent questions should have counsel review the actual LICENSE text.
Maintenance cost is dominated by the offline optimization step rather than the runtime. Every model change means re-running opt and re-validating accuracy, because quantization and fusion alter the numerics of the graph. The runtime itself is a library you link once per platform, but the backend you select constrains which build you download or compile, and switching backends later is not a configuration flag alone. The prebuilt libraries reduce the build burden; the Docker environment reduces it further when you do need to compile.
Editorial conclusion
Adopt Paddle Lite if your model already comes out of PaddlePaddle, or if you are willing to run it through X2Paddle first, and your target is an Arm CPU, an Android or iOS device, or one of the NPUs listed in the README. Do not adopt it if you need a runtime that ingests ONNX directly, or if you want an accelerator backend that is not on that list. Before committing, run the opt tool against your actual model and read the operator report it prints, because that report, not the platform table, tells you whether the model will run on the hardware you own.
Community notes