BrowserAI: running MLC, GGUF and Whisper models from a TypeScript package
Run local LLMs like llama, deepseek-distill, kokoro and more inside your browser
At a glance
- What is it?
- BrowserAI is an MIT-licensed TypeScript SDK that loads quantized language, speech and audio-separation models into the browser through WebGPU and WASM. The interesting part is the engine abstraction; the constraint is that model names are hardcoded to a curated list.
- Who is it for?
- Adopt BrowserAI if you are shipping a web app where the model has to live on the user's machine and you are content to pick from the pre-configured model list, because that list is what the API resolves against. Do not adopt it if you need a model outside that list, need server-side inference, or need an OpenAI-compatible HTTP endpoint in front of the weights.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 57 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The server bill BrowserAI is trying to delete
Most browser chat products are thin clients. The weights sit on someone's GPU instance, every token crosses the network, and the operator pays per request. BrowserAI inverts that: the README states that all processing happens locally in the browser, that there are zero server costs, and that the app keeps working offline after the initial model download. The target user is named fairly explicitly in the README's Perfect For list: web developers building AI features, teams with privacy constraints, researchers, and no-code platform builders. The privacy claim is the one that carries weight. If the prompt never leaves the tab, there is no data processing agreement to negotiate and no retention policy to write. That is a different product from a hosted API, not a cheaper version of one.
Four engines behind one generateText call
The architecture is an engine abstraction. The README lists four: MLC, Transformers, Flare and Demucs, and says the SDK supports switching between them. Each covers a different kind of work. MLC handles the quantized chat models, and the loadModel example passes a quantization string such as q4f16_1 alongside the model name. Transformers covers Whisper for speech recognition and Kokoro for text-to-speech. Flare runs GGUF files through WASM, which is the escape hatch when WebGPU is unavailable or when you specifically want a GGUF build. Demucs does audio source separation and is imported from a separate entry point, @browserai/browserai/demucs, rather than the root package. That split matters: the DemucsEngine class is not part of the default import, so a text-only app does not pull the separation code into its bundle. The README also mentions Web Worker support for non-blocking UI and built-in database support for conversations and embeddings, though it does not document the database API in the material available here.
loadModel, generateText and the shape of the response
Installation is a single package: npm install @browserai/browserai, or the yarn equivalent. The basic flow is to construct a BrowserAI instance, await loadModel with a model name and an options object, then call generateText. The loadModel options shown in the README include quantization and an onProgress callback that receives an object with a progress field, which the example logs as a percentage. That callback is the only progress surface documented, and it matters because a multi-gigabyte download with no visible feedback is a bad first run. generateText accepts either a plain string or an array of role and content objects for chat-style prompts, and returns an object whose text is read as response.choices[0].message.content, the OpenAI-shaped path. Generation options include temperature, max_tokens and system_prompt. For structured output the README shows a json_schema option combined with response_format set to json_object, which is the mechanism for getting typed data out rather than prose. Speech recognition uses startRecording, stopRecording and transcribeAudio with return_timestamps and language options. Text-to-speech calls textToSpeech with a voice such as af_bella and a speed multiplier, and returns an audio buffer you decode yourself through the Web Audio API. Note that the README's TTS snippet constructs a new BrowserAI instance after already creating one, which reads as a copy-paste slip rather than an intended pattern.
The model list is the API surface
This is the sharpest constraint in the project. loadModel takes a name like llama-3.2-1b-instruct or kokoro-tts, not a URL or a local file handle. The README's supported-models section enumerates what those names resolve to, organized by engine. MLC carries the bulk of it: the Llama 3.2 1B and 3B instruct models, Hermes-Llama-3.2-3b, three SmolLM2 sizes, Qwen3 from 0.6B to 8B, Gemma-2B-IT, Phi-3.5-mini-instruct, Qwen2.5-1.5B-Instruct, both DeepSeek-R1 distills, and a set of Snowflake Arctic embedding models. Transformers covers Llama-3.2-1b plus the Whisper and Kokoro speech models. Flare lists four GGUF models with specific quantizations, for example SmolLM2-135M-Instruct in Q8_0 and Q4_K_M. Demucs lists exactly one: HTDemucs with four stems. The README says more models will be added and invites issue requests. Until that changes, the practical question for an adopter is not whether BrowserAI can run your model, but whether your model is on this list. The Flare engine's loadAdapter call, which takes a URL pointing at a .safetensors file, is the one documented way to attach custom weights, and it is a LoRA adapter on top of a listed base model, not a replacement for one.
What breaks, and when this is the wrong tool
The first failure mode is hardware. WebGPU is the acceleration path the README leans on, and it is not uniformly available across browsers and devices. The Flare engine exists precisely because GGUF via WASM offers a fallback, but the README does not state the performance relationship between the two paths, and a WASM fallback for anything beyond the small SmolLM2 and Qwen2.5-0.5B entries is not something the documentation promises. The second failure mode is the download itself. Offline capability is real, but it is preceded by fetching weights to the user's machine, and the README does not describe caching semantics, storage quotas, or what happens when a browser evicts the cache. A user returning after eviction pays the download again. The third is the model ceiling. The largest listed chat models are 8B parameters, and the README offers no throughput figures for them, so anyone assuming a 70B-class model will run acceptably in a tab is guessing. The fourth is the wrong-tool case: if your workload is batch, if you need a stable HTTP endpoint that many clients share, or if you need to guarantee a specific GPU, browser inference is the wrong layer. You would be pushing model weights and compute onto hardware you do not control and cannot benchmark centrally.
How this differs from calling a hosted inference API
The obvious alternative is a hosted inference provider behind an OpenAI-compatible endpoint. The difference is not speed or quality, it is where the boundary sits. With a hosted API you send tokens and receive tokens, you pay per call, and you inherit the provider's uptime, rate limits and model catalogue. With BrowserAI the network call happens once, at model download, and after that the compute is the user's. That changes the cost curve from linear in usage to fixed per user, and it changes your failure modes from HTTP 429 and 503 to GPU adapter availability and cache eviction. A second alternative is shipping a desktop application that bundles llama.cpp or a similar runtime. That gives you a predictable execution environment and no browser API surface to worry about, at the cost of an install step and platform-specific builds. BrowserAI's trade is the opposite: no install, but the runtime is whatever browser the visitor happens to be using. If your users are on managed corporate machines with locked-down browsers or no WebGPU, the desktop route is the one that will actually work.
Licence, versioning and what upgrading costs
The repository is MIT-licensed, which permits commercial use and modification provided the copyright notice and permission notice are retained. That covers the SDK code. It does not automatically cover the model weights the SDK downloads, and the README does not state the licence of each listed model. Llama derivatives, Gemma and Qwen carry their own terms, and some have use restrictions that an MIT SDK licence does nothing to resolve. Anyone shipping a product on top of BrowserAI should check the licence of the specific weights they load, separately from the SDK licence. This is not legal advice, and the model-licence question is the one worth putting to a lawyer. On maintenance, the release history shows v2.0.2, then v2.0.4 about five weeks later, then a much longer gap to v2.2.0 roughly eleven months after that. That cadence suggests the project moves in bursts rather than continuously. The practical upgrade cost is small if you stay inside the documented API, since generateText and loadModel appear stable across those releases, but a major version bump in a project with this release rhythm is worth reading the notes for before you pull it into a production branch.
Editorial conclusion
Adopt BrowserAI if you are shipping a web app where the model has to live on the user's machine and you are content to pick from the pre-configured model list, because that list is what the API resolves against. Do not adopt it if you need a model outside that list, need server-side inference, or need an OpenAI-compatible HTTP endpoint in front of the weights. Before committing, verify two things in a real browser: that navigator.gpu reports an adapter on your target hardware, and that the model you intend to ship is named in the README's engine tables, since loadModel takes a name rather than a URL and there is no documented path for registering your own.
Community notes