# ESP-DL: a neural network inference framework for Espressif chips > ESP-DL is Espressif's lightweight inference framework for its own SoCs, with a FlatBuffers-based model format, mixed 8-bit and 16-bit quantization, a static memory planner and dual-core scheduling for convolution operators. It suits developers porting a model onto an ESP32-S3 or ESP32-P4, and the operator support list is the first thing to check before planning a port. **espressif/esp-dl** — Espressif deep-learning library for AIoT applications - Repository: https://github.com/espressif/esp-dl - Stars: 1,211 · Forks: 232 - Language: Assembly - License: MIT - Published: 2026-10-19 · Updated: 2026-10-19 · Language: en - Canonical page: https://hysenlabs.com/projects/espressif-esp-dl ## An inference framework, not a training framework ESP-DL is described as a lightweight and efficient neural network inference framework designed specifically for the ESP series chips, and the goal is that you can develop AI applications on Espressif's SoCs without leaving the vendor ecosystem. The framework offers APIs to load, debug and run AI models, and it is designed to integrate with other Espressif SDKs, which means it is a component in a larger firmware tree rather than a standalone program. The primary language recorded for the repository is Assembly, which reflects the hand-optimised kernels at its core. The licence is MIT, the project is not archived, and the last push was on 2026-10-19. The published releases are v3.2.0 on 2025-10-23, v3.0.0 on 2024-12-20 and v2.0 on 2023-07-31, and the news entries in the README run well past the v3.2.0 tag, so the development line has moved on since that release. ## The .espdl format, and why FlatBuffers The ESP-DL standard model format is similar to ONNX but uses Flatbuffers instead of Protobuf, which makes it more lightweight and supports zero-copy deserialization, with the file extension `.espdl`. The zero-copy property is the one that matters on a microcontroller, because it means the runtime can point at the tensor data in the file rather than parsing it into a heap allocation, and heap is the scarce resource. The same reasoning explains a later feature: `.espdl` models can be visualized in netron, which facilitates model debugging, added in January 2026. A format that is not readable by the standard ONNX tooling would otherwise be a debugging dead end, and being able to open the file in netron removes that problem. The trade-off is that you now have two formats to reason about, since the model is quantized into `.espdl` rather than being run straight from ONNX. ## Mixed quantization is the design decision ESP-DL supports 8-bit (w8a8), 16-bit (w16a16) and mixed-precision (w8a16) quantization, and the important word is that these schemes can be freely combined within a model to preserve accuracy while maximizing inference efficiency. That is a different proposition from a framework that applies one scheme to the whole network. A vision model on an ESP32-P4 can keep its first layer at 16-bit because quantisation error there is visible in the output, and quantise the deep layers to 8-bit where the savings matter more than the last fraction of accuracy. The news list records the timeline of this feature, with w8a16 mixed quantization fully supported starting from ESP-DL v3.3.11 in September 2026. The tool on the other side is ESP-PPQ, which quantizes models from ONNX, Pytorch and TensorFlow and exports them into the ESP-DL standard model format. ## Operators, the memory planner and dual-core convolution The runtime implements common AI operators such as Conv, Gemm, Add and Mul, and the full list lives in `operator_support_state.md`, which the documentation tells you to read before quantizing your own model. The performance data for some operators on the ESP32-S3 and ESP32-P4 is in `benchmark_report.md`. Two runtime features are worth understanding on their own. The static memory planner automatically allocates different layers to the optimal memory location based on the user-specified internal RAM size, which is a firmware-level concern rather than a library one, and it is the difference between a model that fits and one that does not. And dual core scheduling lets computationally intensive operators use both cores, with Conv2D and DepthwiseConv2D currently supported for it. The activation functions are implemented with an 8-bit lookup table, with all of them except ReLU and PReLU handled that way, so you can use any activation function and their computational complexity remains the same. ## Setting up the toolchain ESP-DL runs on ESP-IDF, and the documentation asks you to use ESP-IDF `release/v5.3` or above, with the ESP-IDF Programming Guide for the details. The quantization tool is a separate install. ESP-PPQ is a quantization tool based on ppq, and the documented method is: ``` pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu pip install esp-ppq ``` The example installs the Linux PyTorch CPU version, with a note telling you to install the appropriate PyTorch version for your actual needs. There is a naming detail that will bite you if you follow older tutorials: the ESP-PPQ package was renamed from ppq to esp-ppq to avoid conflicts with ppq, and the note says to pay attention to modifying the package name in the quantization script. The framework itself is distributed through the Espressif Component Registry as `espressif/esp-dl`, so in an ESP-IDF project it arrives as a managed component rather than something you vendor by hand. ## The examples directory shows the range of tasks The repository carries a wide set of example projects, and the list is the fastest way to see what the framework is being used for. There is object detection in several flavours, `cat_detect`, `dog_detect`, `hand_detect`, `human_face_detect`, `pedestrian_detect` and `person_reid`, plus `yolo11_detect`, `yolo11_pose`, `yolo11_seg` and `yolo26_detect`. There is classification with `mobilenetv2_cls`, face recognition with `human_face_recognition`, hand gesture recognition, speaker verification, colour detection and motion detection. And there is OCR with `pp_ocr_v6`, added in July 2026, which recognises Chinese, English and 46 Latin-script languages on-device on the ESP32-P4, with the model in `models/pp_ocr_v6/` and the example under `examples/pp_ocr_v6/`. The `examples/tutorial/` directory holds the how-to material, including a walkthrough for quantizing a YOLO26 model, and the `models/` directory holds the pre-quantized `.espdl` files. ## Agent tooling and per-channel accuracy upgrades A recent direction in the project is tooling for code agents, and it is unusual enough to call out. An ESP-DL Agent Tool v1.0 was released in November 2025, designed for code agent systems to assist with implementing and optimising operators for the framework, and it was later evolved into an `espdl-operator` skill to make it easier for agents to implement operators. A separate `espdl-quantize` skill, introduced in May 2026, automatically iterates to search for optimal ESP-PPQ quantization strategies, reducing manual tuning costs and improving quantized model accuracy, and there is also AutoQuant support from April 2026, which automatically explores optimal quantization strategies and parameters. Two more skills arrived in June 2026, `esp32s3-pie-simd` and `esp32p4-pie-simd`, to help a code agent rewrite code into a PIE-accelerated version. The accuracy story matters just as much: in April 2026, Conv and Gemm on the ESP32-P4 were upgraded to per-channel quantization, which requires ESP-PPQ 1.2.10 or newer and ESP-DL 3.3.1 or newer, though the release stays backward compatible and existing per-tensor models are unaffected. ## Conclusion Adopt ESP-DL if you are deploying a quantizable model onto an ESP32-S3 or ESP32-P4 and want a runtime, quantization path and operator set that Espressif maintains together. Skip it if your model needs operators the support list does not cover, because ESP-DL implements a defined set rather than a general graph executor, and read the operator support state before anything else. Before you commit, note that ESP-DL requires ESP-IDF `release/v5.3` or above, that per-channel Conv and Gemm quantization on ESP32-P4 needs ESP-PPQ 1.2.10 and ESP-DL 3.3.1 or newer, and that the model schema changed in v3.1.0, where previous models still load but new models do not work with older ESP-DL versions. ## FAQ ### What is the ESP-DL standard model format? It is similar to ONNX but uses FlatBuffers instead of Protobuf, making it more lightweight and supporting zero-copy deserialization. The file extension is .espdl, and the models can be visualized in netron to facilitate debugging. ### Which quantization schemes does ESP-DL support? 8-bit (w8a8), 16-bit (w16a16) and mixed-precision (w8a16). The schemes can be combined within a single model, and w8a16 mixed quantization is fully supported starting from ESP-DL v3.3.11. ### What do I need to install to quantize a model for ESP-DL? ESP-IDF release/v5.3 or above for the framework, and ESP-PPQ for quantization. The documented install is `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu` followed by `pip install esp-ppq`. ### How do I check whether my model can run on ESP-DL? Refer to the operator support state document in the repository before quantizing, and check the benchmark report for operator performance data on the ESP32-S3 and ESP32-P4. ### Which operators support dual-core scheduling in ESP-DL? Conv2D and DepthwiseConv2D currently support dual-core scheduling, which lets computationally intensive operators use the dual-core computing power of the chip. ## Sources - [espressif/esp-dl on GitHub](https://github.com/espressif/esp-dl) - [Issues](https://github.com/espressif/esp-dl/issues) - [License: MIT](https://github.com/espressif/esp-dl/blob/master/LICENSE) - [README](https://github.com/espressif/esp-dl/blob/master/README.md) - [Releases](https://github.com/espressif/esp-dl/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/espressif-esp-dl