llama.rn: Running GGUF Models Inside a React Native App

The gap llama.rn fills between llama.cpp and a phone app

llama.cpp gives you inference in C/C++, and its server example gives you HTTP routes for completion, tokenization, embeddings and reranking. Neither of those is directly callable from JavaScript running in a React Native app. llama.rn exists to close that gap: it is the React Native binding of llama.cpp, and the README states that its design is inspired by the server.cpp example, with the same operations exposed as methods on a context object rather than as HTTP endpoints. The audience is therefore narrow and specific. You are building an iOS or Android app in React Native, you want the model to run on the device rather than behind an API, and you are willing to manage GGUF model files yourself. If any of those three is false, this library is not aimed at you. The README's own model guidance is a HuggingFace search for the GGUF keyword, plus a pointer to llama.cpp's quantize tool if you want to produce a quantized file yourself. There is no bundled model, no download helper, and no hosted fallback.

How the binding is structured: context, params, token callback

The API is deliberately thin. initLlama takes a configuration object and returns a context; the README's example passes model, use_mlock, n_ctx and n_gpu_layers, with embedding as a commented-out option. Everything after that hangs off the context. context.completion accepts either a messages array for chat or a prompt string for raw text completion, plus sampling parameters such as n_predict and stop, and an optional partial completion callback that fires per token with a data object containing token. The resolved value carries text and timings. The same context exposes tokenize, detokenize, embedding and rerank, which map onto the corresponding server.cpp routes. There is also a standalone loadLlamaModelInfo that reads a model file without initializing a context, which is useful for inspecting a GGUF before paying the load cost. Two details in that structure matter more than they look. First, the stop sequences are not inferred: the README's example hardcodes a list of end-of-turn markers covering Llama, Llama 3, Qwen and other chat templates, and you are expected to extend it for your model. Get that list wrong and the model keeps generating past the end of its turn. Second, the token callback is where streaming UI comes from; the library does not hand you a stream object.

Installation, the postinstall download, and the Bun exception

The install is one command, npm install llama.rn, but the interesting part happens afterwards. During postinstall the package downloads a prebuilt ios/rnllama.xcframework and android/src/main/jniLibs from the matching GitHub release, reuses anything already downloaded, and verifies each archive with SHA-256 before extraction. That is a supply-chain decision worth noticing: the JavaScript package is on npm, but the native binaries come from GitHub release assets, and the integrity check is the only thing tying the two together. Bun changes the picture, because it does not run dependency lifecycle scripts unless a package is trusted. The README gives two options: add llama.rn to trustedDependencies in package.json and re-run bun install, or skip the automatic path entirely and run node ./node_modules/llama.rn/install/download-native-artifacts.js yourself before npx pod-install or the Android build. On iOS you re-run npx pod-install, and building from source instead of using the prebuilt framework means setting RNLLAMA_BUILD_FROM_SOURCE to 1 in the Podfile. On Android the equivalent switch is rnllamaBuildFromSource set to true in android/gradle.properties. If ProGuard is enabled, the README asks for a keep rule covering com.rnllama. Expo users configure the library through expo-build-properties and a plugin entry whose documented defaults are enableEntitlements true, entitlementsProfile production, forceCxx20 true and enableOpenCL true.

GPU and NPU offload: manifest entries and a runtime check

Acceleration is opt-in and platform-specific. For OpenCL on Android, the README lists three steps: confirm the device exposes an OpenCL-capable GPU (Qualcomm Adreno 700+ is described as currently supported and tested), add a uses-native-library entry for libOpenCL.so with required set to false so the loader can be resolved at runtime, and set n_gpu_layers above zero when calling initLlama. For Hexagon NPU offload, marked experimental, the requirements are an HTP-capable device (Qualcomm SM8450 or newer, described as supported and tested), a uses-native-library entry for libcdsprpc.so, and a devices parameter set to HTP0 or HTP* alongside a positive n_gpu_layers. The native result from initialization exposes gpu, reasonNoGPU and devices, which is the only documented way to confirm the offload actually happened rather than silently falling back to CPU. That check is not optional in practice. The supported device lists are narrow and written as what has been tested, not as a guarantee, so a positive n_gpu_layers value on an untested phone tells you nothing until you read the result fields.

The New Architecture requirement is the real adoption constraint

The README carries an explicit notice: starting with v0.10, llama.rn requires React Native's New Architecture. Old Architecture support lives on the v0.9 branch, along with the documentation for that line. This is the single largest constraint in the project, and it is a hard fork in the road rather than a configuration flag. If your app has not migrated, you either stay on v0.9 and forgo everything added since, or you migrate React Native before you can take a current version. The release cadence reinforces the point: the three most recent releases listed are v0.13.0-rc.1, v0.13.0-rc.2 and v0.13.0-rc.3, all release candidates. A project whose visible tip is a chain of RCs is telling you the API is still moving, and the README itself flags several features as experimental, including Hexagon NPU acceleration and the text-to-speech path built on codec.cpp. There is also the model-side cost that no binding can remove: context initialization is described in the README's own comment as something that may take a while, and n_ctx, n_gpu_layers and use_mlock all trade memory against latency in ways you have to tune per device.

When to choose something else: llama.cpp's own server, or an on-device runtime

The clearest alternative is llama.cpp's server example, which the binding is modelled on. The difference is where the process runs. With server.cpp you run a binary on a machine you control, expose the /completion, /chat/completions, /tokenize, /detokenize, /embedding and /rerank routes, and your React Native app is an HTTP client. That approach removes the native build, the per-platform binary download, the New Architecture requirement and the app store review questions that come with shipping model weights. It also removes the reason to use llama.rn at all, because inference is no longer on the device. The choice is therefore about offline operation, data locality and per-request cost, not about API shape: llama.rn's method names are close enough to the server routes that porting between them is mostly mechanical. If your requirement is on-device inference but your stack is not React Native, this binding is irrelevant and you want llama.cpp directly or another language binding. If your app needs a managed model catalog, hosted scaling or a stable versioned API, a hosted inference provider is the better fit and llama.rn is the wrong tool.

Maintenance cost, licensing and what the MIT label does not cover

The package is MIT licensed, which is permissive and imposes no copyleft obligation on your app. Two caveats follow from the material. First, llama.rn is a binding, not the inference engine: llama.cpp sits underneath it, and the model weights you load carry their own licences, which the README does not address and which you must check per model. Second, the MIT grant covers this repository's code, not the prebuilt binaries pulled from GitHub releases at postinstall time; those are built from this project and from llama.cpp, and if you need certainty about what is inside the framework you ship, the documented route is to build from source via RNLLAMA_BUILD_FROM_SOURCE on iOS or rnllamaBuildFromSource on Android. On the maintenance side, the upgrade cost is concentrated in two places: the React Native architecture requirement, which is a one-time migration you cannot avoid if you want v0.10 or later, and the release candidate cadence, which means pinning a version and reading release notes before moving. Verifying a SHA-256 hash on every install is a small ongoing cost that the postinstall script already handles for you.

Editorial conclusion

Adopt llama.rn if you are already on React Native's New Architecture and you need GGUF inference on the device with Metal, OpenCL or Hexagon offload, and you can absorb a release candidate as your dependency. Do not adopt it if you are still on the Old Architecture, since v0.10 and later require the New Architecture and the v0.9 branch is the documented escape hatch. Before committing, verify that the prebuilt rnllama.xcframework and jniLibs archives for your target release download and pass their SHA-256 check, and confirm on a real device that initLlama reports gpu: true rather than a reasonNoGPU string.

Official sources

1. Issues
2. License: MIT
3. mybigday/llama.rn on GitHub
4. README
5. Releases