# llama4micro: a language model and a voice on a microcontroller

> llama4micro puts a small language model and a text to speech model on a Coral Dev Board Micro, with a camera and an object detector deciding what the story is about, and the whole point is that it fits in 64MB of RAM and runs on a microcontroller class CPU.

**maxbbraun/llama4micro** — A "large" language model running on a microcontroller

- Repository: https://github.com/maxbbraun/llama4micro
- Stars: 561 · Forks: 37
- Language: C++
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/maxbbraun-llama4micro

## The question the project was built to answer

llama4micro is a large language model running on a microcontroller, and the readme quotes the author asking whether it is possible to fit a non-trivial language model on a microcontroller at all, with the answer being some version of yes.

That framing is worth keeping in mind, because it tells you what the project is. It is an experiment with a size constraint, not a product and not a chat application. The words large in the title are in quotes for a reason.

The readme also carries a warning that explains how the project grew. Later, things got a bit out of hand and now the prompt is based on objects detected by the camera, and a text to speech model reads the story out loud. So the final device is a small camera that recognises something, writes a short story about it, speaks it and prints it to the serial port.

That is a much better demonstration than a text prompt would have been, and it is also a much larger build, because it pulls in an object detector, a language model, a speech synthesiser and the board's own camera. The languages and frameworks in the tree are the evidence for that: the C++ code, the Python tooling used to flash the board, and a set of vendored third party projects.

The project is MIT licensed, the default branch is main, the last push was on 2026-10-18, and the repository publishes no GitHub releases.

## The hardware: 64MB of RAM and a spare CPU core

The target is the Coral Dev Board Micro with its FreeRTOS toolchain, and the readme is explicit about why that board: it has a number of useful hardware features, and most importantly for this purpose it has 64MB of RAM.

The comparison the readme makes is the right one. 64MB is tiny for language models, which are typically measured in the gigabytes, and it is comparatively huge for a microcontroller. Everything about the design follows from that arithmetic.

The compute allocation is also documented. Language model inference and text to speech generation run on the 800MHz Arm Cortex-M7 core. Camera image classification uses the Edge TPU, with a compiled YOLOv8n model. And the board has a second 400MHz Arm Cortex-M4 core which is currently unused.

That unused core is a good detail to notice, because it means the software is not yet using everything the hardware offers, and it also tells you where the next piece of work would go.

The YOLOv8n choice is a small one, and the nano variant is the smallest model in that family, which is the right call when the accelerator has to classify whatever is in front of the camera in real time while a language model generates a story on the other core.

## The model: llama2.c adapted, trained on TinyStories

The language model implementation is an adaptation of llama2.c, and the checkpoints are the tinyllamas ones trained on the TinyStories dataset.

That is a sensible pairing for the constraint. TinyStories is a dataset of simple children's stories, so a model trained on it produces short, simple prose, which is what a microcontroller with 64MB of RAM can hold and what the story pipeline needs.

The readme is honest about the result: the quality of the smaller model versions isn't ideal, but good enough to generate somewhat coherent, and occasionally weird, stories. That parenthesis does a lot of work, and it is the sentence to read before you decide what this project is for. The point is not literary quality, it is that a transformer can run end to end on a microcontroller alongside a vision accelerator and a speech synthesiser.

The speech side comes from a separate third party project, sanoTTS, which the readme lists among the submodules, alongside the lite synthesis engine from festvox. The other submodules are karpathy/llama2.c for the model, google-coral/coralmicro for the board support, and ultralytics/ultralytics for the detector.

The pre-trained models live in the models directory of the repository, with a models/README.md that documents how to download and convert them. The conversion step exists because the checkpoints you download are not in a format the microcontroller build can use directly, and that instruction file is the part of the setup you need to read before the rest.

## Wiring the speaker, building the image and flashing it

The setup has three distinct physical and software steps, and the readme covers them in order.

The first is hardware. You need a PAM8302 amplifier and a four to eight ohm speaker, and the wiring is given as a table between the amplifier pins and the board pins.

| Amplifier pin | Coral pin |
|---|---|
| A+ | DAC_OUT / A2 |
| A− | GND |
| SD | Not connected |
| VIN | VSYS |
| GND | GND |

The second is the source tree. The repository has to be cloned with its submodules, because the model, the board support, the detector and the speech engines are all vendored.

```bash
git clone --recurse-submodules https://github.com/maxbbraun/llama4micro.git

cd llama4micro
```

The third is the build, which is a standard CMake build in a build directory.

```bash
mkdir build
cd build

cmake ..
make -j
```

Flashing is a separate step with its own Python environment, because the tool that writes the image to the board is a Python script from the coralmicro support tree.

```bash
python3 -m venv venv
. venv/bin/activate

pip install -r ../third_party/coralmicro/scripts/requirements.txt
```

The last command in that sequence is the flash itself, pointing at the ELF that the build produced.

```bash
python ../third_party/coralmicro/scripts/flashtool.py \
    --build_dir . \
    --elf_path llama4micro
```

That the flashing step is a Python virtual environment and a separate tool is a good sign of how much of the work is toolchain rather than code.

## The runtime loop, once the board is flashed

The usage section describes a loop with four states, and it is worth following because the timings are part of the experience.

The models load automatically when the board powers up, which takes around eight seconds, and the green light turns on when the board is ready. That load time is the price of a 64MB memory budget, since the weights have to be read into RAM before anything can happen.

You point the camera at an object and press the button, and the green light turns off. The camera takes a picture and detects an object. The model then generates a story that starts with a prompt based on that object, the sentences are spoken using text to speech while the story is being generated, and the story is also streamed to the serial port.

To cancel, you hold the button for one second, which stops generation and playback, and you then go back to step two. Generation otherwise stops after the end token or the maximum number of steps, playback finishes, the green light comes on again, and the loop repeats.

The serial port output is the detail that makes this useful to a developer, because it gives you the generated text in a terminal while the board speaks it, so you can see what the model actually produced instead of only hearing it.

## Where llama4micro is the wrong tool

Four limits, and the first is about the output rather than the hardware.

The generated text is not something to build on. The readme says the quality of the smaller model versions isn't ideal, good enough to generate somewhat coherent and occasionally weird stories. A TinyStories model on a microcontroller is a demonstration of feasibility, and a product that needs sentences a customer would pay for is a different project with a different memory budget.

The second is the hardware specificity. The build targets the Coral Dev Board Micro with its FreeRTOS toolchain, and the camera path depends on the Edge TPU with a compiled YOLOv8n model. Porting the language model to another board is conceivable, but the vision and audio halves are tied to this board's accelerators, and the wiring table is for its pins. This is a project for one device, and it is not a portable framework.

The third is the third party surface. Five submodules have to be cloned and kept in step, including the model implementation, the board support, the detector and two speech projects, and the pre-trained models have to be downloaded and converted using the instructions in models/README.md. Every one of those is a place where the build stops working when an upstream moves, and there is no release process here to coordinate them, since the repository publishes no GitHub releases.

The fourth is the unused Cortex-M4 core. It is described as currently unused, which means there is headroom in the hardware and a corresponding amount of work still to do, and that is a good thing to know before you plan an ambitious extension on top of it.

## Conclusion

llama4micro is the right project for someone who wants to understand what running a language model actually costs in memory and compute, and for a maker who wants a self-contained device that looks at an object and tells a story about it without a network connection. It is the wrong foundation for an application, since the model is a TinyStories checkpoint whose smaller versions are described as producing somewhat coherent and occasionally weird stories, and the hardware is a single specific board. Before attempting the build, confirm the Coral Dev Board Micro and its FreeRTOS toolchain are what you have, read models/README.md for the model conversion step, and expect to spend the time on the flash and wiring steps rather than on the model.

## FAQ

### What hardware does llama4micro run on?

The Coral Dev Board Micro with its FreeRTOS toolchain, which has 64MB of RAM. Inference and text to speech run on the 800MHz Arm Cortex-M7 core, camera classification uses the Edge TPU with a compiled YOLOv8n model, and the second 400MHz Arm Cortex-M4 core is currently unused.

### Which language model does llama4micro use?

An adaptation of llama2.c using the tinyllamas checkpoints trained on the TinyStories dataset. The readme notes that the quality of the smaller model versions isn't ideal, but good enough to generate somewhat coherent and occasionally weird stories.

### How do I build and flash llama4micro?

Clone the repository with its submodules, then configure and build with CMake in a build directory using cmake .. and make -j. Flashing uses a separate Python virtual environment, installing the coralmicro scripts requirements and then running flashtool.py with the build directory and the llama4micro ELF path.

### What does llama4micro do when you press the button?

The camera takes a picture and detects an object, and the model generates a story starting with a prompt based on that object. The sentences are spoken through text to speech while the story is being generated, the story is also streamed to the serial port, and holding the button for one second cancels generation and playback.

## Sources

- [Issues](https://github.com/maxbbraun/llama4micro/issues)
- [License: MIT](https://github.com/maxbbraun/llama4micro/blob/main/LICENSE)
- [maxbbraun/llama4micro on GitHub](https://github.com/maxbbraun/llama4micro)
- [README](https://github.com/maxbbraun/llama4micro/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/maxbbraun-llama4micro