Hysen Labsneedle
14MB foundation model for tiny devices; phones, wearables, smart home, and robots.
Needle, a 14MB model for tool calling
Needle 2 is an open 45 million parameter model for tool calling and structured extraction, packed into a single 14MB binary that runs in about 28MB of RAM.
The size claim
GitHub calls it a 14MB foundation model for tiny devices, from phones and wearables to smart home and robots. The README says Needle 2 is an open 45 million parameter model for tool calling, device use, and structured extraction, compressed to 2 bit with Cactus Quants and baked into its own engine, running a full session in about 28MB of RAM.
What the package holds
This repository is the Python package: inference, LoRA fine tuning, and export. You install it, describe your tools, and call them from Python, with the inference engine fetched once from Hugging Face and cached, plus offline setup covered for air gapped devices.
Design features
Tool calls come back as structured data, text in and JSON out, with a byte level grammar compiled from schemas constraining every token. Responses carry a calibrated confidence score, a retrieval head surfaces only the top five tools per turn, and a sliding window keeps memory near 28MB however long the conversation runs.
The architecture
Needle 2 is a Simple Attention Network, a dense small model recipe with a Hadamard MLP in place of the feed forward network, GQA attention, engram key value memory, and multi lane hyper connections. The README references a paper for the design and ablations.
Benchmark framing
On the benchmarks cited, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M, and Apple FM, at 5 to 70 times smaller and at 2 bits against their f16. The positioning is explicitly about devices with limited memory and compute.
Community notes