# flutter_edge_ai runs local models across five engines, if you accept the FFI and plugin overhead

> DenisovAV/flutter_edge_ai splits on-device inference into a small core package plus fourteen opt-in add-ons, covering LiteRT-LM, MediaPipe, llama.cpp, ONNX, operating system models, embeddings, RAG and speech. The architecture is unusually disciplined, but the package sprawl is a real cost for small teams.

**DenisovAV/flutter_edge_ai** — On-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.

- Repository: https://github.com/DenisovAV/flutter_edge_ai
- Website: https://flutteredge.ai
- Stars: 644 · Forks: 176
- Language: Dart
- License: MIT
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/denisovav-flutter-edge-ai

## A core package that only installs models and hands them over

The design decision that defines this project is where the responsibility boundary sits. The package documentation describes the core package as small: it installs models and hands them to engines. Sessions and chat live in the same place. Everything else, every runtime, store and feature, is a separate opt-in package, so an application ships only what it uses.

That is a deliberate departure from the usual pattern of one large SDK where a two-line demo pulls in a megabyte of unused transitive dependencies. Here, the dependency list is the API surface. If an app needs text-to-speech and a vector store but no image understanding, it takes flutter_edge_ai_speech and one store package, and nothing else. The cost is that a developer has to know which package owns which capability before writing the first line of app code.

The engine registry is the seam that makes this work. Instead of each add-on exposing its own incompatible interface, the core holds a registry that engines plug into. Sessions and chat sit above that registry, so application code can talk to the core while the actual inference happens in a native library reached through dart:ffi or through a platform plugin. Whether that abstraction always holds up under six platforms is a question only running the code would answer.

## Five inference backends and five different model file formats

The engines are not interchangeable skins over one runtime. They disagree about model files, about which platforms they cover, and about what happens when hardware is missing.

flutter_edge_ai_litertlm runs LiteRT-LM files with the .litertlm extension through dart:ffi on the five native platforms plus web, and the docs name CPU, GPU and NPU as supported paths. It also supplies the LiteRT embedding backend. flutter_edge_ai_llama runs llama.cpp .gguf files through llamadart on Android, iOS, macOS, Windows, Linux and web, with native LoRA support. flutter_edge_ai_onnx covers ONNX Runtime, doing GenAI inference and embeddings natively and falling back to Transformers.js and onnxruntime-web on web. flutter_edge_ai_mediapipe handles MediaPipe .task files, but only on Android, iOS and web, and the docs mark Google's LLM Inference API inside it as maintenance-only, pointing new applications at .litertlm instead.

That last note is the kind of detail worth catching early. A team that picks the MediaPipe engine for a new application is entering a path the project itself describes as maintenance-only.

Platform coverage is uneven across the set. Qdrant Edge, for example, is native platforms only, while the SQLite vector store using sqlite-vec is described as available on every platform including web. Any plan that mixes an engine, a store and a platform has to be checked against that table rather than assumed.

## The built-in engine that downloads nothing at all

flutter_edge_ai_builtin_ai is the odd one out in a project otherwise centred on shipping model weights to a device. The README describes it as the OS model with nothing to download: Gemini Nano on Android, Apple Foundation Models on iOS and macOS, Windows AI Foundry on Windows, and the browser Prompt API on web.

The appeal is obvious. No download step, no storage budget, no model version to pin, and no risk of a user on a metered connection stalling on a large file. For a summarisation feature or a short rewrite tool, this may be the only part of the stack you need.

The trade-off is that you are not choosing the model. You inherit whatever the operating system exposes, which varies by hardware, by OS release and by whether the vendor has enabled the feature at all. The README does not document a fallback path or a capability check for devices where the OS model is missing, so an application built on this engine alone should verify on hardware it actually supports before shipping. The project is explicit that this engine can run any model the operating system already ships, which is a ceiling as much as a floor.

## RAG is split into a contract and three separate stores

The retrieval part of the stack shows the same decomposition logic taken further. flutter_edge_ai_rag provides the on-device pieces: RagIndex, embedding profiles, filters and what the docs call the vector-store contract. The contract is the part that matters architecturally. Because application code talks to an interface rather than to a specific database, the same index logic can sit on top of different stores.

Two store packages implement it. flutter_edge_ai_sqlite uses SQLite with sqlite-vec and is described as working on every platform, web included. flutter_edge_ai_qdrant targets Qdrant Edge on native platforms only. The tokenizer side is separated again into flutter_edge_ai_embeddings, which covers Gemma SentencePiece and BERT WordPiece tokenizers for .tflite and .onnx embedders.

That three-way split (tokenizers, index logic, store) means a RAG feature pulls in at least two packages before you write a query. It also means you can mix a Gemma SentencePiece tokenizer with an ONNX embedder, which is exactly the flexibility the package layout is built to allow. Index migration between stores is not documented, so treating the store as replaceable is a design intention rather than a documented procedure.

## Getting the packages and running the example

This project publishes to pub.dev rather than shipping a binary, so installation happens through the Dart package manager and there is no separate installer to describe. The docs link the package listings on pub.dev for each package, including flutter_edge_ai itself, flutter_edge_ai_litertlm, flutter_edge_ai_llama, flutter_edge_ai_onnx, flutter_edge_ai_mediapipe and flutter_edge_ai_builtin_ai, and points new users to the core package docs under packages/flutter_edge_ai and to the Getting started page on flutteredge.ai.

The docs do not print a shell command for adding any of these packages, so there is nothing to copy here without inventing a flag. Take the package names from the pub.dev listings as they are written.

The quickest way to see the whole thing work is the example app that ships inside the repository at packages/flutter_edge_ai/example, which carries integration tests. The repository also holds codelabs/ with step-by-step apps behind the published codelabs, and website/ with the source of the documentation site. The top-level layout includes a .fvm directory and a .fvmrc file, which is the FVM setup used to pin the Flutter and Dart versions for this workspace. If your project targets a different Flutter version than the workspace pins, that is the first thing to reconcile, because the native dylibs and the FFI surface have to agree.

## Agents that execute SKILL.md files and MCP tools

flutter_edge_ai_agent extends the stack past answering into acting. Agent skills are written in SKILL.md files that the model picks and runs, with text, JavaScript, native intents and MCP tools as the available action types. The other packages in the same family are flutter_edge_ai_speech, covering speech-to-text, text-to-speech and a VoiceSession voice loop, and flutter_edge_ai_diagnostics, which reads process memory on Android, iOS and macOS.

Letting a model select and execute code changes the risk profile of the whole project. A text model that only generates tokens has a failure mode you can mostly show to the user. A model that chooses among scripts and native intents can take actions, and the security of that depends on what the shipped SKILL.md files are permitted to do on the device.

The repository root includes an AGENTS.md file and a skills_lint.yaml file, the latter suggesting that the skill definitions are checked by a lint step. Neither file is described in detail in the docs, so a team shipping this package should read them directly before deciding which skills to include in a release build. The concern here is concrete: an agent that can run JavaScript on device needs a review process for the files it can reach.

## Frequent native rebuilds and a moving release train

The maintenance picture is a mix of reassurance and work. The repository is not archived and the last push was on 2026-10-20, which is recent. The release history shows three native dylib builds inside a nine day window: native-v0.18.0-f on 2026-10-18, native-v0.18.0-e on 2026-10-12 and native-v0.18.0-d on 2026-10-09. Those are labelled as native dylibs, which points to the native artifacts being republished on their own cadence rather than only on package releases.

Two readings are possible and both are supported by the listing. Either the native binaries are versioned independently and you must track them separately from the Dart packages, or the maintainers are iterating quickly and the naming simply reflects build iterations. Either way, the practical consequence for an adopter is that an upgrade touches more than one artifact, and a pinned dependency set has to include the dylib version rather than only the pub.dev version.

The licence is MIT, which covers commercial use without a copyleft obligation. That is a low-friction position for a commercial app, and it applies to the package set as published. Separate components such as the models themselves carry their own terms, and the README does not discuss model licensing, so any model you install deserves its own review.

## Conclusion

Adopt flutter_edge_ai when your product needs inference on the device, when the OS already ships a capable model, or when shipping data to a server is unacceptable. Do not adopt it for a quick prototype that only needs one text generation call, since the package selection alone will cost more than the code it saves. Before committing, verify the engine that matches your model file format against your target platform matrix, check that the example app in packages/flutter_edge_ai/example runs on your minimum supported OS version, and confirm the native dylib release you depend on is compatible with your Flutter and Dart versions.

## FAQ

### What is flutter_edge_ai and what does it replace?

It is a set of Dart packages that run AI models on the device, with no server and no cloud calls. It covers text, vision, audio, function calling, embeddings, RAG and speech across Android, iOS, web, macOS, Windows and Linux.

### Which package do I add to a Flutter project to run a local LLM?

Pick the engine that matches your model file: flutter_edge_ai_litertlm for .litertlm files, flutter_edge_ai_llama for .gguf files through llama.cpp, flutter_edge_ai_onnx for ONNX Runtime, or flutter_edge_ai_mediapipe for .task files on Android, iOS and web. The core flutter_edge_ai package installs models and holds sessions and chat.

### Can flutter_edge_ai run a model without downloading it?

Yes, through flutter_edge_ai_builtin_ai, which targets Gemini Nano on Android, Apple Foundation Models on iOS and macOS, Windows AI Foundry and the browser Prompt API. The docs describe this engine as the OS model with nothing to download.

### How do I build on-device RAG with flutter_edge_ai?

Use flutter_edge_ai_rag for RagIndex, embedding profiles, filters and the vector-store contract, then add a store such as flutter_edge_ai_sqlite for sqlite-vec on every platform or flutter_edge_ai_qdrant for Qdrant Edge on native platforms. Tokenizers for .tflite and .onnx embedders live in flutter_edge_ai_embeddings.

### Is flutter_edge_ai suitable for production apps?

The repository is not archived and the last push was on 2026-10-20, with native dylib releases on 2026-10-09, 2026-10-12 and 2026-10-18. The example app with integration tests sits at packages/flutter_edge_ai/example, which is the fastest way to check the stack on your target devices before shipping.

## Sources

- [DenisovAV/flutter_edge_ai on GitHub](https://github.com/DenisovAV/flutter_edge_ai)
- [License: MIT](https://github.com/DenisovAV/flutter_edge_ai/blob/main/LICENSE)
- [Project website](https://flutteredge.ai)
- [README](https://github.com/DenisovAV/flutter_edge_ai/blob/main/README.md)
- [Releases](https://github.com/DenisovAV/flutter_edge_ai/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/denisovav-flutter-edge-ai