LiteRT: Google's Successor to TensorFlow Lite Targets NPU and GenAI on Device

What LiteRT Actually Replaces and Why

LiteRT is the direct successor to TensorFlow Lite, and the README says it continues that legacy. The problem it solves is running ML and GenAI models on devices where cloud round trips are too slow or too private. The intended users are mobile and embedded developers who need CPU, GPU, or NPU inference without shipping a server. The key shift is that LiteRT V2 introduces a Compiled Model API that automates accelerator selection. That means you no longer explicitly pick a delegate, as you did with TensorFlow Lite. Instead, the runtime decides which accelerator to use. This is a meaningful change in workflow, not just a rename.

The Compiled Model API and Asynchronous Execution

The Compiled Model API is the centerpiece of LiteRT V2. The README lists automated accelerator selection, true asynchronous execution, easy NPU distribution, and efficient I/O buffer handling as its features. The practical effect is that you write model loading and inference code once, and the runtime maps it to whatever hardware is present. The asynchronous execution is notable because TensorFlow Lite's classic API was mostly synchronous at the high level. The README also mentions ML Drift for GPU acceleration, which is a new buffer interoperability layer that reduces latency between different GPU buffer types. This is aimed at GenAI workloads where memory copies dominate. The trade-off is that this API is new, so you are betting on it maturing.

Unified NPU Access: A Promise with Caveats

LiteRT claims broad NPU support through a single API. The platform table shows Android with Qualcomm, MediaTek, Google Tensor, Intel, and Broadcom. Linux gets Broadcom and Intel. Windows gets Intel. iOS and macOS list ANE as coming soon, and Web lists WebNN as coming soon. That is a real limitation. If you target Apple devices, you cannot rely on NPU acceleration yet. The README points to a separate LiteRT NPU page for details, but the table is clear: the unified API exists, but the hardware coverage is uneven. For a framework that markets unified NPU acceleration, the missing Apple Neural Engine and WebNN are gaps you must plan around.

From PyTorch to On-Device: The Conversion Pipeline

The README shows a conversion flow from PyTorch and Hugging Face transformers to .tflite or .litertlm files. The LiteRT Torch Converter handles classic .tflite conversion, while the Generative Torch API produces .litertlm for LLMs. After conversion, the AI-Edge Quantizer optimizes the model, and then the LiteRT Runtime runs it on CPU via XNNPack, GPU via ML Drift, or supported NPUs. This is a concrete pipeline: you start with a PyTorch model, convert, quantize, and deploy. The README does not give exact commands for the converter, but it links to the litert-torch repository. This means the conversion path is real but requires a separate tool. The .litertlm format is specific to LiteRT-LM, so if you want to run an LLM, you need that companion project.

Getting Started: The LiteRT-CLI and Python Environment

The quick setup in the README is for the LiteRT-CLI, not the runtime itself. It assumes you have uv installed. The commands are: create a virtual environment with Python 3.13 using 'uv venv --clear --python=3.13 --seed', activate it, then install 'litert-cli-nightly' with 'uv pip install litert-cli-nightly', and finally run 'litert --help'. The README notes that setting the UV_INDEX_URL environment variable can help resolve dependency errors. This is a nightly package, not a stable release. For the runtime, you need to use the main repository's build system, which is C++ and uses CMake for Android and Linux. The README does not provide a single install command for the runtime, so expect to build from source or use platform-specific packages.

Platform Support and the Reality of Web and IoT

The platform table lists Android, iOS, Linux, macOS, Windows, Web, and IoT. Every platform has CPU support. GPU support varies: Android gets OpenCL and OpenGL, iOS gets Metal, Linux gets WebGPU, macOS gets WebGPU and Metal, Windows and Web get WebGPU. NPU support is sparse, as noted. The IoT row lists Raspberry Pi as coming soon. This is a wide net, but many entries are marked as coming soon. The README also mentions LiteRT.js for browser inference via WebGPU and WASM, which is a separate project. If you need a reliable NPU path today, Android with Qualcomm or MediaTek is the safest bet. On other platforms, you are mostly on CPU or GPU.

Maintenance, Licensing, and the Migration Question

The project is licensed under Apache-2.0, which is permissive and allows commercial use. The release cadence is 6-8 weeks for stable releases, with nightly builds. The last push was August 2026, and the latest release is v2.2.0. This suggests active maintenance. However, the README explicitly says there is a migration guide for TensorFlow Lite users. That implies breaking changes. The Compiled Model API is different from the classic API, so existing TensorFlow Lite code will not work without modification. The maintenance cost is real: you must track releases every two months and potentially update your code. The README does not mention a long-term support version, so you cannot assume stability.

Alternatives: The Trade-Offs You Should Know

The most direct alternative is ONNX Runtime Mobile. It also runs on device, has CPU, GPU, and NPU execution providers, and supports PyTorch and TensorFlow models through conversion. The difference is that ONNX Runtime does not have a single compiled model API with automatic accelerator selection; you usually configure execution providers explicitly. Another alternative is ExecuTorch from PyTorch, which is designed for on-device inference and has a different authoring model. LiteRT's advantage is its integration with Google's ecosystem, including the AI-Edge Quantizer and LiteRT-LM. The disadvantage is that you are tied to Google's roadmap. If you need a more vendor-neutral runtime, ONNX Runtime is worth considering. The choice comes down to whether you want automatic accelerator selection or explicit control.

Editorial conclusion

Adopt LiteRT if you are already on TensorFlow Lite and need a path to GenAI or NPU acceleration, or if you target Android, iOS, or Linux with a need for a unified runtime. Do not adopt it if you require stable NPU support on iOS or Windows, or if you cannot handle the 6-8 week release cadence and the migration from V1.x APIs. Before committing, verify that your specific models convert cleanly through the Torch Converter or the AI-Edge Quantizer, and check the migration guide for breaking changes from TensorFlow Lite. The project is active, but its value depends on the V2 API stabilizing; the next release should clarify that.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes