# Jeff 0.8B Decision Model: Local Choices with Calibrated Confidence

> Jeff is an MIT-licensed 0.8B classification model for fast local decisions, with swappable LoRA adapters and optional fallback to a larger model. Its narrow task format suits routing, triage, classification, and agent decisions, but the available documentation does not define the full quick-start workflow.

**firelex/jeff** — Millisecond decisions, any domain: a 0.8B open "System 1" model that picks between your options with calibrated probabilities. One base, swappable LoRA adapters, on your own hardware.

- Repository: https://github.com/firelex/jeff
- Website: https://jeffhub.ai
- Stars: 1,590 · Forks: 83
- Language: Python
- License: MIT
- Published: 2026-10-16 · Updated: 2026-10-16 · Language: en
- Canonical page: https://hysenlabs.com/projects/firelex-jeff

## Jeff's Decision Path and Intended Users

Jeff addresses a specific part of an AI system: choosing among a defined set of options quickly. It is presented as a 0.8B open "System 1" model, while a larger model such as Qwen3.8-27B remains responsible for writing and planning. That division is useful for developers building local agents, classification services, request routers, or moderation pipelines where every decision would otherwise invoke a much larger model.

The important abstraction is a probability distribution over options, rather than free-form generated text. Jeff can select customer intent, tool choice, urgency, sentiment, prompt-injection status, clause type, navigation target, or another adapter-defined label. Its documentation also describes grounded-answer decisions, emotion classification, spam and phishing detection, and coding-agent actions. The adapter decides both what the candidate labels mean and how Jeff is specialized for that task.

The current README supports both local execution and a hosted API; its current SDK also offers Node, Rust, C# and Java clients. The serving dependencies include PyTorch, Transformers, FastAPI, Uvicorn, and Hugging Face Hub. Optional groups cover NVIDIA and Apple backends, chess, games, and LoRA training. This makes the package relevant to Python developers who want task-specific inference without sending every small decision to a remote API. It is not positioned as a chatbot or an open-ended reasoning model.

## Base Model, Readout, and Adapter Architecture

The repository describes Jeff as an adapter-first classification architecture. A common base supplies the core model, while a LoRA add-on specializes it for one decision domain. Each released adapter was trained for one epoch according to the README. The package description calls this "one-pass decisions with full-weight supervised training," while the adapter section identifies LoRA as the mechanism for creating and serving multiple specializations.

This arrangement avoids loading a separate copy of a 0.8B model for every domain. The README reports a base footprint of 1.74 GB on the stated RTX PRO 6000 setup, with 2.09 GB reported after loading the base with all 15 adapters. A three-adapter configuration was reported as 1.83 GB. Those are specific deployment measurements, not guarantees for every hardware combination.

Version compatibility is strict. The v1.3 adapters work only with the v1.3 base, while earlier v1.2 models remain available under their old names. This matters when building a serving image, swapping adapters, or upgrading from v1.1 or v1.2. Loading a mismatched pair is not presented as a supported configuration.

Jeff also supports GGUF files for llama.cpp and lists Apple serving support through `JEFF_BACKEND=mlx`. The optional NVIDIA packages are described as fast kernels for Qwen3.5, while `transformers` provides a slower fallback. Repository layout also includes `src/`, `clients/`, `scripts/`, `examples/`, `docs/`, and `tests/`, indicating that this is a Python package rather than a collection of notebook examples.

## Calibration and Fallback Control

Jeff does more than return the highest label. Its purpose is to produce calibrated probabilities so a caller can decide whether to accept its answer or pass the case to a larger model. Each task has an adapter-specific fallback threshold. In the comparison described by the README, thresholds were selected on separate calibration rows before held-out test rows were scored.

The documented selection rule is unusually concrete. The project chose the fastest threshold that beat Qwen3.8-27B by at least one accuracy point on the calibration set. When no threshold met that condition, as reported for the `ground` task, it used the threshold with the best calibration accuracy. This turns confidence calibration into an application policy rather than a fixed universal score.

The measured routing pattern illustrates the intended use. For `ground`, the 27B was consulted on 11% of decisions. The other listed adapters used 0% fallback in that comparison. Those percentages apply to the documented samples and thresholds, not every deployment. A custom adapter will still need its own calibration data, threshold selection, and evaluation rows.

For agent systems, the coding example applies the same idea around a larger model. Jeff handles routine information gathering and decides whether Qwen needs to think on a particular turn. The README reports that this was not equivalent to disabling reasoning for every turn, which would have reduced quality. Jeff is therefore best understood as a policy layer for selective computation.

## Installing Jeff and Launching the Server

The current README requires Python 3.13 or later for the Python SDK. Install `jeff-sdk` for hosted API access, or add a runtime extra for local execution:

```bash
pip install jeff-sdk
pip install jeff-sdk 'jeff-runtime[mac]'  # Apple silicon
pip install jeff-sdk 'jeff-runtime[cuda]' # NVIDIA GPU
```

The current local quick start uses `jeff.Jeff.local(["guard"])`; the first run downloads the base model and adapter. Consult the project's current SDK documentation for connection, platform and model requirements. The other SDKs do not themselves require Python for hosted API calls.

## Python Commands, Training, and Evaluation Surface

The project exposes a broad command-line interface through `pyproject.toml`. These entry points cover data preparation, generation, grounding, latency measurement, training, evaluation, serving, panels, dashboards, model mixing, voice, benchmark utilities, and sampled evaluation. The names indicate intended workflows, but the available documentation does not provide their argument grammars, so inventing flags would be misleading.

LoRA support is optional and uses `peft==0.21.1`. The file specifically associates `jeff-train --lora-rank` with adapter training and `JEFF_ADAPTERS` with serving several adapters over one base. This suggests two operating modes: train a specialized add-on, then select one or more adapters at serving time.

The repository also contains an `examples/adapter-kit/` directory and a chess example. The chess group installs `chess==1.11.2` and `zstandard==0.25.0` for Python chess and Lichess `.zst` readers. The package comments also identify `python -m jeff.games` as the entry point for game tests and their videos, with dependencies declared under `games`.

These capabilities make Jeff more than a one-command inference demo. The cost is a larger operational surface. Users must distinguish serving from training, select compatible base and adapter versions, provide calibration data, and build task-specific evaluation. The `uv.lock` file can pin the declared environment, but the README does not explain an upgrade procedure or migration path between model generations.

## Limits of the Reported Comparison

The README's results are promising within a defined setup, but they should not be read as universal model rankings. The timing comparison used an Apple M4 Max with 128 GB, MLX, an 8-bit Qwen3.8-27B, direct answering, and step-by-step reasoning disabled. It compared fixed held-out samples, generally 300 rows per task, with 500 rows for emotion and legal clauses. Memory figures came from an RTX PRO 6000 with released adapter files.

The task results also reveal why Jeff is not a general replacement for a larger model. On the listed `ground` task, Jeff plus the larger-model fallback reached 98.0%, compared with 96.7% for Qwen3.8-27B alone and 95.0% for Jeff alone. That task needed fallback on 11% of cases. In several other rows Jeff's measured accuracy was sufficient without invoking the 27B, but this remains tied to each adapter's labels, data, and threshold.

Emotion classification exposes another boundary. The README reports 59.6% for choosing the strongest of 27 emotions or neutral, versus 35.6% for the larger model. The project leaves this task out of its averages because the short-comment human labels often disagree. Even when Jeff is faster, weak label agreement can make an apparently objective accuracy score unsuitable for production policy.

The documented memory number also includes all 15 adapters in one deployment. Loading every domain into a single process is optional, not mandatory. The adapter architecture permits narrower loading, but the README does not provide measured latency or memory for every adapter count and backend.

## Maintenance, License, and Upgrade Boundaries

The repository is not archived, and its last push was on 2026-10-07. Recent releases include v1.2 on 2026-10-01 and v1.1 on 2026-09-29. The README also announces Jeff v1.3, although v1.3 is not included in the release list shown by the repository. That discrepancy should be checked against the project's release page before selecting a package or model artifact.

The repository uses the MIT License. That permits use, modification, distribution, and sublicensing under the license's stated conditions, including preservation of its copyright and permission notice. This is a summary of the license category, not legal advice. Adapter files and converted GGUF artifacts may have separate hosting or distribution terms, so deployment teams should inspect the terms attached to every downloaded artifact.

Upgrade cost is driven mainly by model compatibility. The README explicitly says v1.3 adapters cannot run on the v1.2 base. Existing calibration thresholds and evaluation results should not automatically be carried across versions. A migration may require downloading the new base, replacing adapters, rebuilding serving configuration, and rerunning calibration and held-out evaluation.

For Jeff-Code, the repository reports a 14% total-time reduction across all tasks despite a 47% per-task time reduction because roughly 5% of tasks ran more than 30 minutes longer when Jeff continued where Qwen alone stopped. That is a concrete operational caution: optimizing routine turns can alter stopping behavior on difficult tasks.

## Conclusion

Developers should evaluate Jeff when repeated, clearly labeled decisions can be expressed as adapter-specific classification tasks and low local latency matters. It is a poor fit for open-ended generation, general reasoning, or applications whose option labels cannot be defined consistently. Before adoption, verify the exact v1.3 base and adapter pair, run the relevant held-out evaluation, and confirm that confidence thresholds work for the intended fallback policy.

## FAQ

### What is Jeff used for?

Jeff is a 0.8B local model for selecting among defined options with calibrated probabilities. The documented uses include tool routing, support-intent classification, urgency and sentiment detection, grounded-answer checks, navigation, spam detection, and coding-agent decisions.

### How do I install and serve Jeff?

Jeff requires Python 3.12 or newer. The repository documents `uv sync` for the default environment and `uv sync --no-default-groups` for serving-only dependencies, then registers `jeff-serve` as the server command.

### Can Jeff adapters be used with any Jeff base version?

No. The README states that v1.3 adapters work only with the v1.3 base. Earlier v1.2 models remain under their old names, so base and adapter versions must be selected together.

## Sources

- [firelex/jeff on GitHub](https://github.com/firelex/jeff)
- [License: MIT](https://github.com/firelex/jeff/blob/main/LICENSE)
- [Project website](https://jeffhub.ai)
- [README](https://github.com/firelex/jeff/blob/main/README.md)
- [Releases](https://github.com/firelex/jeff/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/firelex-jeff