Magnitude: a local inference engine that picks the model for your machine
Open source local inference engine. It runs models on the hardware you already have, whether that's a Mac, an NVIDIA or AMD GPU, or just a CPU. Plug it into Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.
At a glance
- What is it?
- Magnitude profiles your hardware, ranks catalog models by speed, accuracy, intelligence and memory, then downloads and serves the one you pick to an agent harness. Here is what the README and repository show, and what they leave open.
- Who is it for?
- Adopt Magnitude if you want a local model behind an agent you already run and you would rather have the machine profiled for you than tune quantization by hand. Skip it if you need multi-user serving, a documented headless install path, or a stable CLI: the newest release is @magnitudedev/cli@0.1.0-alpha.0, dated 2026-09-17, and the README documents the desktop app on macOS, Windows and Linux, not a server deployment.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 17, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem Magnitude targets: choosing and tuning a local model
Running a model locally is not one decision, it is a chain of them. Which quantization fits in memory, whether the GPU backend is supported, what tokens per second you can expect, and how the agent you already use reaches the server once it is up. Magnitude's README frames the whole chain as its scope: it profiles your chip, memory and bandwidth, estimates fit and tok/s for every model in the catalog, and ranks them by speed, accuracy, intelligence and memory. The intended user is someone who already runs a coding agent (Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi or Cline) and wants that agent pointed at a model on their own hardware instead of a hosted API. The README's pitch is concrete on cost and privacy: no token costs, API keys or rate limits, and prompts, files and models stay on the machine. That is the audience. It is not a serving stack for a team, and nothing in the README describes multi-user access or remote clients.
How the profiling, catalog and model lifecycle fit together
Three mechanisms are visible in the README and the repository layout. First, hardware profiling: Magnitude reads chip, memory and bandwidth, then produces a fit and throughput estimate per catalog model. Second, ranking: the catalog is ordered by speed, accuracy, intelligence and memory, so the recommendation is a ranking problem rather than a single hardcoded default. Third, lifecycle: models are loaded on request and unloaded when idle or when memory fills, which the README describes as requiring no management after setup. The repository layout matches that split. There are separate top-level directories for inference, cli, desktop, integrations and packages, with a Bun workspace root (packageManager bun@1.4.2) and turbo.json for builds. The integrations directory is where the harness adapters live, and the README says Connections in the desktop app configures a harness without launching it. Tuning is described as end to end, with speculative decoding named as one example, applied for the detected hardware rather than exposed as a knob. The README does not document what happens to an in-flight request when a model is unloaded because memory filled.
Installing Magnitude from the download page and running a first model
The README does not give a package-manager install. It points to the download page for macOS, Windows and Linux installers, and states that the desktop app includes the magnitude CLI, so there is no separate installation step. The three-step flow is: install and open the app, choose a recommended model in Discover and download it, then connect your agent in Connections. The CLI is documented separately at docs.magnitude.dev/reference, so treat that page as the source for exact subcommands. The README does state one capability worth knowing before you start: your agent can install or switch models through the CLI at any time. The README lists no CLI invocation to copy, so the first run happens in the app itself: Discover for the model, then Connections for the harness. Once a model is downloaded, the README says no internet connection is needed.
Where Magnitude is the wrong tool
The README's own answers set the boundaries. There is no fixed minimum hardware, but more memory is what lets you run larger models, and the recommendation is generated from a profile rather than a published compatibility matrix. That means a machine that profiles poorly gets a weak recommendation, and you cannot look up in advance whether a specific model will run well. The release history is the second boundary. The newest entry is @magnitudedev/cli@0.1.0-alpha.0 on 2026-09-17, following @magnitudedev/cli@0.0.15 the day before, so the CLI is still moving between alpha and patch releases. Pinning a version for a fleet is not something the README supports. Third, the product is a desktop application. If you need a headless server, containers, or several users sharing one GPU, nothing in the README describes that path; the repository has a .dockerignore, but a Dockerfile or container documentation is not present in the repository listing. Finally, the README does not document rollback, so if a model switch through the CLI goes wrong, the documented recovery is not stated.
Magnitude against llama.cpp and Ollama
The difference is where the decision lives. llama.cpp gives you a runtime and a quantization toolbox; you choose the model file and the flags, and the quality of the outcome tracks your own knowledge of your hardware. Ollama gives you a model registry and a pull-and-run command, with the model tag as the main choice. Magnitude moves the choice into a profile: the README says it assesses fit and tok/s for every model in the catalog and ranks them, then tunes the result, with speculative decoding named as an example. So the trade is explicit. You give up the direct control over quantization and flags that llama.cpp exposes, and you give up the simple named-tag workflow that Ollama users expect, in exchange for a recommendation generated from your machine. If you already know which quantization your card handles and you want to set it yourself, Magnitude's value proposition is smaller. If you do not, the profile is the feature.
Maintenance, release cadence and the Apache-2.0 licence
The repository is not archived, and the last push was on 2026-09-17, the same day as the @magnitudedev/cli@0.1.0-alpha.0 release. Activity is therefore current, but the version numbers say more than the date does: an alpha of 0.1.0 arriving after a run of 0.0.x patches means the CLI interface is not yet settled, and scripts such as simulate:distribution, test:release-bootstrap and cache:clear in package.json suggest the release and caching machinery is still being exercised. Budget for re-reading docs.magnitude.dev/reference after upgrades rather than assuming flags are stable. The licence is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant; the repository ships the LICENSE file at the root. That is a permissive licence, but it is not legal advice, and if you redistribute a modified build you should read the notice and attribution requirements yourself. The desktop installers are distributed from magnitude.dev, so the licence covers the source in this repository, not necessarily the packaging around it.
Editorial conclusion
Adopt Magnitude if you want a local model behind an agent you already run and you would rather have the machine profiled for you than tune quantization by hand. Skip it if you need multi-user serving, a documented headless install path, or a stable CLI: the newest release is @magnitudedev/cli@0.1.0-alpha.0, dated 2026-09-17, and the README documents the desktop app on macOS, Windows and Linux, not a server deployment. Verify first that your harness appears among Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi and Cline, and that the download page lists an installer for your OS.
Frequently asked questions
What is Magnitude?
It is an open source local inference engine. According to the README, it profiles your machine, recommends the right models for it, then downloads, tunes and runs them, and you plug it into the agent you already use.
How does Magnitude know what my machine can run?
The README states that it profiles your chip, memory and bandwidth, then estimates fit and tok/s for every model in the catalog, ranking them by speed, accuracy, intelligence and memory.
Which agent harnesses work with Magnitude?
The README lists Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi and Cline, and says Connections in the desktop app configures your harness without launching it.
Is Magnitude private and does it need an internet connection?
The README says prompts, files and models stay on your machine, and that once a model is downloaded no internet connection is needed.
Do I need to manage Magnitude after setup?
The README says no: it runs in the background, loads models when your agent needs them, and unloads them when idle or when memory gets tight.
What hardware and systems does Magnitude support?
macOS, Windows and Linux, per the README, which points to the download page for installers and system requirements. There is no fixed minimum hardware; more memory lets you run larger models.
Community notes