Model or dataset
cactus-compute/needle avatar
cactus-compute/needle

Needle by Cactus Compute: a single-file tool-calling model that refuses to guess

14MB foundation model for tiny devices; phones, wearables, smart home, and robots.

7 stars3 forksPythonLicense varies

At a glance

What is it?
Needle is an 8 to 29 MB on-device model built for tool calls, structured extraction and embeddings, with the Whistle speech model sharing its engine. It gives up general chat to do those jobs on phones, wearables and microcontrollers.
Who is it for?
Needle fits developers who need an app or device to turn user requests into function calls or typed fields locally, without a server, and who can accept that it is not a chat model. Teams that need open-ended conversation should look elsewhere.
Can I use it commercially?
Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
Is it still maintained?
Yes. The repository last received commits 10 days ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 13, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • cactus-compute/needle7 ★ · -7,419
Star history
Repository2026-08-182026-11-13
cactus-compute/needle7.4k7

-7,419 Stars in the last 87 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

Chat capacity traded away for tool calls and extraction

Needle is not a small general assistant. Cactus Compute describes the whole model as a single 8 to 29 MB binary built on its Simple Attention Network, and states plainly that it trades general chat capacity for narrow jobs. The claims are that it beats models ten times its size on mobile tool calls and matches models two to three times bigger on extraction, with tool calling scored as exact-match accuracy and extraction as field micro-F1 on full test splits.

The jobs are four: picking functions your app exposes and filling their arguments, extracting typed fields from messy text against a declared shape, returning a sentence embedding for local search and routing, and accepting speech through Whistle. If your product needs free-form conversation, Needle does not cover it.

A decorated Python function becomes the tool schema

The Python package installs from PyPI:

sh
pip install cactus-needle

The signature of a function decorated with needle.tool supplies argument types, the docstring becomes the tool description, and run() executes the chosen function and returns its results:

python
import needle

@needle.tool
def get_weather(city: str):
    "Get the current weather for a city."
    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
print(agent.run("what's it like in Lagos right now?")["results"])
# [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]

The base install depends only on huggingface_hub and requires Python 3.9 or newer. Weights and the platform engines are hosted on Hugging Face under Cactus-Compute/needle3.

An empty list instead of a guess

Every turn returns one JSON object with function_calls, the model's reasoning and a calibrated confidence score from a learned head. A request no tool covers comes back as an empty list rather than an invented call, and a request for two things yields two calls in order. A byte-level grammar compiled from your schemas constrains every token, so extraction output always parses.

The consequence for app code is that an empty list is a normal result you must handle, not an error. Existing deployments can stay on the previous generation with needle.Needle(tools=[...], generation=2).

Whistle shares the .cact container and C++ engine

Whistle is the speech-to-text model: one 16.9 MB file on the CPU, taking 16 kHz mono audio of any length, one pass up to 30 seconds and windowed beyond that, in English, German, French, Spanish, Italian, Dutch and Polish.

python
import needle

print(needle.transcribe("clip.wav")["text"])
# turn off the kitchen lights

word_timestamps=True adds per-word start, end and probability; keywords=[...] favours your product names and jargon; language="de" forces a language. needle.stream(chunks) transcribes live. Because both models load into one engine, a single command goes from audio to tool calls:

sh
needle --model needle3.cact --model whistle.cact --tools tools.json --audio clip.wav

The speech fields in that JSON result are prefixed audio_.

Music and noise can still turn into text

Silence and steady noise return an empty transcript, but other sounds, music especially, can still come out as text. For an always-on microphone that means false transcripts, which then reach your tools as requests. Cactus Compute's own advice is to put a voice activity detector or a wake word in front of it.

Whistle is also not ahead everywhere. Against Whisper base it leads on LibriSpeech test-clean and test-other, SPGISpeech, Earnings-22 and the FLEURS average, while Whisper base is ahead on TED-LIUM, AMI and the MLS average. Check your own audio domain against those benchmarks before replacing Whisper.

Fine-tuning pulls in JAX through an optional extra

Training lives behind the train extra, which adds numpy, jax, jaxlib, flax, optax, safetensors, sentencepiece, whisper-normalizer and soxr. Whistle fine-tuning trains LoRA adapters on the attention projections with the base frozen, merged at export at 4-bit, from JSONL files whose lines hold audio, text and language:

sh
pip install "cactus-needle[train]"
needle whistle finetune train.jsonl validation.jsonl test.jsonl --tokens tokens.txt --out adapter.safetensors
needle whistle build --lora adapter.safetensors --out tuned.cact

Other extras are gpu (jax[cuda12]), metal (which pins jax and jaxlib to 0.4.38 plus jax-metal), mic, compare (openai-whisper and moonshine-voice) and mlx. Because the metal extra pins exact JAX versions, installing it next to another JAX project in one environment forces both onto 0.4.38.

No GitHub releases and a community JavaScript port

The repository has no GitHub releases. The package version is set in pyproject.toml, currently 3.2.1. The last push was on November 2, 2026, so work is ongoing, but you should pin the exact package version you validate.

For Node.js and browsers, a separate cactus-needle npm package runs Needle and Whistle through WebAssembly. It is community-maintained by a different author, not by Cactus Compute:

sh
npm install cactus-needle

That means its release timing and support are independent of the Python package.

Editorial conclusion

Needle fits developers who need an app or device to turn user requests into function calls or typed fields locally, without a server, and who can accept that it is not a chat model. Teams that need open-ended conversation should look elsewhere. Before adopting it, pin the PyPI version you test against, since the repository publishes no GitHub releases, and if you add Whistle to an always-on microphone, put a voice activity detector or wake word in front of it.

Frequently asked questions

What is Cactus Compute Needle used for?

Needle is an on-device model for tool calls, structured extraction and text embeddings on phones, wearables, robots, smart home devices, cars and microcontrollers. It trades general chat capacity for those tasks.

How do I install Needle?

Run pip install cactus-needle; it needs Python 3.9 or newer. Fine-tuning needs the train extra, installed with pip install "cactus-needle[train]".

What does Needle return when no tool matches?

It returns an empty list of function calls rather than a guess. Each turn is one JSON object with function_calls, reasoning and a calibrated confidence.

Which languages does Needle's Whistle speech model support?

Whistle transcribes English, German, French, Spanish, Italian, Dutch and Polish. Fine-tuning with a language code outside those seven adds that language.

Official sources

  1. Official documentation
  2. Official README
  3. Project repository
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/cactus-compute-needle.svg)](https://hysenlabs.com/projects/cactus-compute-needle)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/cactus-compute-needle.svg)](https://hysenlabs.com/projects/cactus-compute-needle)

Was this useful?