Model or dataset
magnitudedev/magnitude avatar
magnitudedev/magnitude

Magnitude: a local inference engine that tunes kernels per model on your own chip

Open source local inference engine. It runs models on the hardware you already have, whether that's a Mac, an NVIDIA or AMD GPU, or just a CPU. Plug it into Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.

7,846 stars570 forksTypeScriptApache-2.0

At a glance

What is it?
magnitudedev/magnitude is an Apache 2.0 desktop app and CLI that runs open models locally and wires them into coding agents such as Codex, Claude Code and Cline. Its speed pitch rests on per-model kernel tuning at download time, and the headline comparison with Ollama is scoped more narrowly than the tagline suggests.
Who is it for?
Magnitude suits developers who want open models running locally behind an agent like Codex, Claude Code, OpenCode or Cline, on a Mac or NVIDIA machine where its own Ollama comparison applies. AMD, Intel and CPU-only users can run it, but should measure speed themselves rather than lean on the 2x figure.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 1 day ago.
What is it written in?
Mainly TypeScript, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • magnitudedev/magnitude7,846 ★ · +3,023
Star history
Repository2026-09-222026-11-15
magnitudedev/magnitude4.8k7.8k

3,023 Stars in the last 54 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

Kernel tuning happens when you download a model

Magnitude's central mechanism is automatic kernel tuning. When you download a model, it spends up to a minute on your actual chip trying different ways to split up the math: how many GPU threads team up, how big a chunk each one takes, and how far ahead to load data. It keeps the fastest combination that does not change the model's output.

The project contrasts this with llama.cpp, Ollama and LM Studio, which it says ship one-size-fits-all kernels. The practical consequence is a one-time cost per model per machine. Every new model you pull pays up to a minute of tuning before it is ready, and the result is specific to the chip it ran on. Moving to a different machine means the tuning is done again there.

The 2x claim is scoped to Mac and NVIDIA

The tagline says Magnitude is over 2x faster than Ollama. The fuller statement is narrower: 2x faster on average across popular local models, on Mac and NVIDIA. Full benchmark charts are published at magnitude.dev/benchmarks.

Magnitude also runs on AMD and Intel GPUs and on a plain CPU, but the speed comparison is not stated for those. If you are on an AMD card or a CPU-only server, the headline number does not cover your setup, and you need to compare against your current engine yourself. Part of the advantage also comes from optimized kernels written for the most popular open-weight families, so a model outside those families should not be assumed to get the same gain. The supported list lives at magnitude.dev/models.

Built-in drafters, quantized KV and shared prefix caches

Beyond kernel tuning, three runtime features target agent workloads. Each model ships with what the project calls the fastest speculative decoding drafter for it, so you do not pick or configure a draft model yourself. The KV cache is quantized, which halves memory per agent, and that memory is freed when agents stop. Concurrent sessions share prefix caches so that running several sessions does not slow each one down.

The hardware floor is open ended. There is no fixed minimum: smaller machines run smaller models, and more memory allows larger ones. What that does not tell you is whether a given model fits on your machine, so memory is the number to check before downloading a large model.

Installing pipes a remote script into your shell

Magnitude comes as a desktop app plus a CLI. The terminal install on macOS and Linux is:

sh
curl -fsSL https://magnitude.dev/install.sh | sh

On Windows PowerShell:

powershell
irm https://magnitude.dev/install.ps1 | iex

Both run a script fetched from magnitude.dev directly, so if your environment does not allow that, download the script and read it first, or use a packaged installer. Installers exist for macOS on Apple silicon and Intel as dmg, Windows x64 as exe, Linux x64 as .deb, .rpm and .pkg.tar.zst, and Linux ARM64 as .deb and .rpm. There is no Windows ARM installer and no Arch package for ARM64 in that list. Remote server and Docker setups are covered at docs.magnitude.dev.

One click covers eight agents; everything else goes through the API

The one-click integrations are Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi and Cline. Any other tool has to talk to Magnitude through its OpenAI-compatible API, which means configuring the endpoint and model yourself in that tool.

The repository reflects this split: there is an integrations/ workspace directory next to cli/, desktop/, inference/ and web/. Running Magnitude as a service other computers use is a separate setup, described under Remote server in the docs, so the default install is a single-machine arrangement.

Private after download, but the download needs a network

Magnitude's privacy claim is that prompts, files and models stay on your machine, with no token costs and no internet needed once a model is downloaded. The qualifier matters. Fetching a model, and the tuning pass that follows it, happen on first use, so an air-gapped machine needs models brought in some other way. The installers and install scripts also come from magnitude.dev.

Local state lives under ~/.magnitude. The repository's package.json has scripts that read ~/.magnitude/logs/events.jsonl, including events:tail, which runs tail -f on that file. If you are auditing what the engine records, that log is the place to look.

A Bun monorepo shipping CLI releases days apart

The codebase is TypeScript in a Bun workspace, pinned to [email protected], with workspaces for cli, desktop, web, inference, integrations/* and packages/*, and Turbo for builds. Contributors run scripts such as bootstrap, dev and build; build regenerates a version file and then calls turbo build. The repository also carries AGENTS.md and CLAUDE.md.

The project is under active development, with the last push on 2026-11-02. CLI releases arrive fast: @magnitudedev/cli 0.2.14 on 2026-10-29, 0.2.15 on 2026-10-30 and 0.2.16 on 2026-11-03. The version is still below 1.0. If you script against the CLI or depend on its behavior in a team setup, pin the version and read the changesets before upgrading.

Editorial conclusion

Magnitude suits developers who want open models running locally behind an agent like Codex, Claude Code, OpenCode or Cline, on a Mac or NVIDIA machine where its own Ollama comparison applies. AMD, Intel and CPU-only users can run it, but should measure speed themselves rather than lean on the 2x figure. Before adopting it, check magnitude.dev/models for your model family, and pin a CLI version, since the 0.2.x line was shipping new releases days apart.

Frequently asked questions

How to use Magnitude to run local models with an agent?

Install Magnitude with the install script or a packaged installer, download a model, which it tunes for your chip for up to a minute, then connect an agent. Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi and Cline connect in one click; other tools use the OpenAI-compatible API.

Is Magnitude faster than Ollama on every machine?

Magnitude claims over 2x faster than Ollama on average across popular local models, on Mac and NVIDIA. That comparison is not stated for AMD, Intel or CPU-only machines.

What hardware and operating systems does Magnitude support?

Magnitude runs on Apple Silicon, NVIDIA, AMD and Intel GPUs, or on a CPU alone, with no fixed minimum. It supports macOS, Linux and Windows.

Does Magnitude send prompts to the cloud?

No. Prompts, files and models stay on your machine, and no internet connection is needed once a model is downloaded.

Official sources

  1. License: Apache-2.0
  2. magnitudedev/magnitude on GitHub
  3. Project website
  4. README
  5. Releases
Community

Where developers are discussing it

Posts on Hacker News, dev.to and Lobsters that link to this repository, found by our scan of those communities. 0 in the last 30 days; newest first.

Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/magnitudedev-magnitude.svg)](https://hysenlabs.com/projects/magnitudedev-magnitude)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/magnitudedev-magnitude.svg)](https://hysenlabs.com/projects/magnitudedev-magnitude)

Was this useful?