hipfire: an RDNA-native LLM inference engine in Rust, and what Redline actually does
RDNA-native LLM inference engine in Rust.
At a glance
- What is it?
- hipfire targets AMD GPUs directly through HIP with no Python in the hot path, and its Redline dispatch layer replays a validated kernel graph. The interesting question is not whether it is fast, but which AMD cards and which model files it will accept.
- Who is it for?
- Adopt hipfire if you have an RDNA1 through RDNA4 Radeon card and want an Ollama-shaped workflow that never enters Python, and if you are willing to verify two things before trusting it: that your exact gfx target appears in the GPU support documentation, and that the model you need exists in the live registry rather than only in the README table.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Rust, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem hipfire addresses: AMD inference without a Python runtime
Most local inference stacks assume an NVIDIA card or assume that a Python process can sit between the client and the GPU. hipfire takes the opposite position on both. It is a Rust engine that talks to AMD hardware through HIP, and the README states plainly that there is no Python in the hot path. The interface is deliberately familiar: pull a model, serve it as a daemon, chat against it. The daemon exposes an OpenAI-compatible API on 0.0.0.0:11435, which means existing clients can point at it without a custom SDK.
The audience is narrower than "anyone with a GPU". The README describes the project as RDNA-first with CDNA supported, and the performance tables only cover three RDNA parts: gfx1100 (Radeon RX 7900 XTX), gfx1151 (Radeon 8060S / Strix Halo) and gfx1201 (Radeon AI PRO R9700). If your card is not in that family, the material here does not tell you what to expect. That is a real constraint, not a footnote, because the engine's optimisation work is explicitly architecture-specific.
Redline records a kernel graph and replays it, or falls back to plain HIP
Redline is the part of hipfire that is not just a HIP wrapper. The README describes it as an in-tree dispatch and retained-replay substrate: it records the actual kernel graph, derives resource dependencies, retains invariant command state, and lowers validated paths through public ROCr queue interfaces. In plain terms, the engine watches what a model actually executes, checks that the recorded graph is still valid, and then replays it with less per-step decision-making.
The design is fail-closed, and the README lists the conditions that trigger a fallback to ordinary HIP dispatch: unsupported graphs, failed shadow validation, ABI mismatches, queue faults, and model changes. That list is the most informative sentence in the whole document, because each entry is a case where the optimised route cannot be trusted. A model swap invalidating a recorded graph is expected behaviour rather than a bug. The README also notes that optimised routes remain architecture- and workload-specific, which means a route validated on gfx1201 is not automatically a route for gfx1151.
The performance claims sit on top of this. The README reports ordinary autoregressive decode with Q8 KV, no MTP, no DFlash, no speculative acceptance, no reduced-output benchmark and no manual clock pinning, then gives TG128 figures of 253.3 tok/s on gfx1100, 115.1 tok/s on gfx1151 and 203.9 tok/s on gfx1201, along with 8-turn averages of 191.0, 92.2 and 169.5 tok/s. Short-context ranges are also published, and they are tight: the gfx1100 range is 253.04 to 253.48 tok/s. A range that narrow across repeated runs is the kind of detail that suggests a sealed fixture rather than a best-case sample. The README attributes the gfx1201 result to a campaign that raised decode from roughly 110 tok/s, and links a report at docs/perf-checkpoints/2026-07-13-redline-mq4r-110-to-204.md. Those numbers are the project's own, and this article has not reproduced them.
MQ4R is a speed SKU, and the README says so before you pull it
Quantisation in hipfire is not a single dial. The registry lists MQ2, MQ3, MQ3P, MQ4P, MFP4, MQ4R, MQ5 and MQ6 variants for the same base model, and MQ4R is explicitly the performance-oriented SKU. The README describes it as combining uniform MQ4 attention and gate-side weights with graded routed experts and a fused gate path.
The trade-off is stated without hedging: MQ4R prioritises throughput, and the README advises using the default MQ4P model, or MFP4, MQ5 or MQ6, when quality matters more than maximum decode speed. That is a useful piece of honesty. It also means the headline 253.3 tok/s figure is not a general statement about hipfire. It is a statement about one quantisation format on one GPU, and the same model at MQ6 will behave differently.
The size constraint is concrete. The README gives the MQ4R Qwen 3.6 35B-A3B model as 18.7 GB and states it requires approximately 22 GB of available VRAM. That gap between file size and VRAM requirement is where KV cache and runtime buffers live, and it is the number to check against your card before pulling anything. A 24 GB card is the obvious fit; a 16 GB card is not, at least not for this SKU.
Install and the actual command surface
The README does not include a full installation section in the material available here, so the build and dependency steps cannot be described from what is on hand. What is documented is the command surface once the binary exists. The quick-start block is three lines: hipfire pull qwen3.5:4b, then hipfire serve qwen3.5:4b -d, then hipfire chat qwen3.5:4b. The -d flag on serve backgrounds the daemon, which then listens on 0.0.0.0:11435 with an OpenAI-compatible API.
One-shot inference reuses the same registry and serving stack rather than a separate code path: hipfire run qwen3.5:4b "What is the capital of France?". Registry inspection is hipfire list -r, which the README calls the authoritative live list. That phrasing matters because the README's own table says 77 pullable entries while a badge near the top says 61 curated model entries. The two numbers disagree, and the README resolves it in favour of the command rather than the table.
For the Redline path there is a reproduction command: python3 -m tools.redline golden. The README points to a Golden Redline guide at docs/GOLDEN-REDLINE.md that also covers making the validated fixture the serve default and connecting Hermes, Pi or another OpenAI-compatible client. Note that the reproduction tool is Python even though the engine is not, which is a reasonable split but worth knowing if you are trying to keep a Python-free deployment.
Where hipfire is the wrong tool
Three cases stand out. The first is hardware outside the RDNA1 to RDNA4 range. The README says Redline supports that family, and describes CDNA as supported at the project level, but none of the published performance data covers a CDNA part. Buying into hipfire for the Redline behaviour on a CDNA card means buying into the part of the documentation that does not exist yet.
The second is model choice. hipfire pulls from its own registry, and the README lists families including Qwen 3.5, Qwen 3.6, Qwen 3.8, Muse Glimmer, DeepSeek V4 Flash, MiniMax-M2.7, North-Mini-Code-1.0, Carnice, Qwopus, LFM2.5 and NEX N2 Mini. That is broad for a project this size, but it is still a curated list. There is no documented path in this material for loading an arbitrary GGUF or safetensors file from disk. If your model is not in the registry, hipfire is not the tool for you, regardless of how well it performs on the models that are.
The third is the fail-closed fallback itself. It is the right default, but it is not free. Every time shadow validation fails or the model changes, you drop to ordinary HIP dispatch, and the README does not quantify what that costs. The 110 to 203.9 tok/s campaign on gfx1201 implies the gap can be large, but that figure describes one card and one model. On a different configuration the fallback could be a modest slowdown or a severe one, and the material here does not say which.
How it differs from llama.cpp and Ollama
The obvious comparison is llama.cpp, and the difference is not the model formats or the chat interface. It is where the optimisation lives. llama.cpp carries a broad set of hand-written kernels intended to run acceptably across many vendors, with HIP as one backend among several. hipfire inverts that: it targets AMD specifically, and Redline adds a runtime layer that records the graph a model actually produces and replays it. That is a per-machine, per-model artefact rather than a shipped kernel library.
The consequence is that hipfire's ceiling can be higher on a supported card while its floor is less predictable. A recorded graph that validates gives you the fast path. One that does not gives you ordinary HIP dispatch, which is roughly where a conventional backend would sit anyway. Ollama is the closer match on interface, and hipfire borrows that shape deliberately: pull, serve, chat, OpenAI-compatible endpoint on a local port. The difference is the runtime underneath, and the fact that hipfire's model registry is its own rather than a mirror of an external hub.
Maintenance, release cadence and the licence question
The release history is dense. v0.1.20 landed on 2026-05-05 with the note engine modularization, v0.2.0 followed on 2026-05-27 with DeepSeek V4 Flash and tokenizer diagnostics, and v0.2.1 arrived on 2026-06-10 headed dispatch unification. That is three releases in roughly five weeks, and the last push to the repository is dated 2026-09-10, so the project is active. The README also advertises a v0.3.0 beta headlined by MQ4R and Redline across RDNA, adding Qwen 3.8 27B and Muse Glimmer 30B.
A cadence like that cuts both ways. You get fixes quickly, and you also get a moving target. The Redline fallback conditions include ABI mismatches and model changes, which means a version bump can invalidate a validated route you were relying on. Upgrading is not a neutral operation here; it may cost you the fast path until the fixture is revalidated with python3 -m tools.redline golden.
On licensing, the repository metadata reports NOASSERTION and the material available here contains no licence text. That is not a statement that the project is unlicensed, only that the terms are not determinable from what is on hand. Anyone planning to ship hipfire inside a product, or to redistribute the curated weights published at huggingface.co/hipfire-models, needs to read the actual licence files in the repository rather than infer anything from the absence of an SPDX identifier. This is not legal advice and should not be treated as such.
Editorial conclusion
Adopt hipfire if you have an RDNA1 through RDNA4 Radeon card and want an Ollama-shaped workflow that never enters Python, and if you are willing to verify two things before trusting it: that your exact gfx target appears in the GPU support documentation, and that the model you need exists in the live registry rather than only in the README table. Do not adopt it if you are on CDNA hardware expecting the Redline path, since the README scopes Redline to RDNA1 through RDNA4 and describes CDNA only as supported, or if you need a model family outside the curated list, because hipfire pulls from its own registry and the README gives no path for loading arbitrary GGUF or safetensors weights. Verify the licence terms first: the repository reports NOASSERTION, so the actual terms of use are not stated in the material available here.
Community notes