Chatty: WebLLM chat that keeps prompts on your GPU
ChattyUI - your private AI chat for running LLMs in the browser
At a glance
- What is it?
- Chatty is a Next.js front end that runs quantized LLMs through WebLLM in the browser tab, with local file embeddings and chat history. The trade-off is hardware: the README puts 7B models at roughly 6GB of GPU memory, and the Docker path is explicitly unoptimized for production.
- Who is it for?
- Adopt Chatty if you want a self-hosted, single-user chat UI where prompts never leave the machine and you can accept a 6GB GPU class requirement for 7B models. Do not adopt it if you need multi-user accounts, server-side inference, or a hardened production container, since the README states the Dockerfile is not yet optimized for production.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 38 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem Chatty solves: inference without an inference server
Most chat interfaces that talk to a language model are thin clients over someone else's GPU. The prompt leaves your machine, the completion comes back, and the privacy story depends entirely on the operator of that endpoint. Chatty inverts this. The README describes it as a private AI that uses WebGPU to run large language models natively in the browser, and states plainly that all models run locally on the client, with no server-side processing. The intended audience is therefore narrow and identifiable: developers, researchers and privacy-conscious users who already have a machine with a capable GPU and who want a ChatGPT-style interface without sending text to a remote API. It is also aimed at people who want that interface to feel familiar. The project describes itself as an attempt to bring the familiarity and functionality of interfaces such as ChatGPT and Gemini into an in-browser experience, and the feature list reads accordingly: markdown rendering, code highlighting, chat history, regenerate, light and dark mode, voice input, and export to json or markdown. The distinguishing feature is not the chat UI, it is where the tokens are generated.
How Chatty works: WebLLM in the tab, embeddings in memory
The architecture is a Next.js application that delegates generation to WebLLM, the MLC project that compiles models for WebGPU execution. The README credits WebLLM, HuggingFace, open source LLMs and LangChain as the components Chatty is built on. Model weights are downloaded once and then cached, which is why the README can claim offline use after the initial download. File chat is the second pipeline. Documents are embedded locally using XenovaTransformerEmbeddings, specifically the all-MiniLM-L6-v2 model hosted on HuggingFace, and the resulting vectors go into LangChain's MemoryVectorStore. That last detail matters more than it looks. A memory vector store lives in the page, so embeddings are not persisted across sessions and are not shared between tabs. The README's roadmap acknowledges a related constraint: multiple file embeddings, meaning the ability to embed more than one file at a time per session, is still an unchecked item. So the retrieval flow is one document, embedded on the client, queried in the same session, with the vectors discarded when the page goes away. Everything else in the UI, including chat history and custom memory instructions, sits on top of that loop.
Getting it running: npm, Docker, and the GPU floor
The README gives a straightforward source install. Clone the repository, change into the directory, install dependencies, start the development server:
git clone https://github.com/addyosmani/chatty cd chatty npm install npm run dev
The app is then available on localhost:3000. Node.js 18 or newer is the stated requirement, and npm is assumed. For container users the README provides a Dockerfile and a compose file:
docker build -t chattyui . docker run -d -p 3000:3000 chattyui
docker compose up
It also notes that after changes you can rebuild with docker-compose up --build. The important caveat is attached to the Docker section in the form of a note: the Dockerfile has not yet been optimized for a production environment, and the README points readers to the Next.js Docker example if they want to do that work themselves. Hardware is the other gate. The README states that 7B models require a GPU with about 6GB of memory while 3B models require around 3GB, and adds that smaller models may not process file embeddings as efficiently as larger ones. On the browser side, WebGPU is enabled by default in Chrome and Edge, and the README says it can be enabled in Firefox and Firefox Nightly, linking to MDN's compatibility table rather than promising uniform support.
Where Chatty stops being the right tool
The limitations are mostly consequences of the design rather than defects. Because inference happens on the user's GPU, there is no shared capacity: ten users mean ten GPUs, and a user on integrated graphics or with WebGPU disabled gets nothing. The README's own hardware note sets the floor at roughly 3GB for 3B models, which excludes a large share of laptops. The vector store choice imposes a second boundary. MemoryVectorStore is in-memory, so a long document session cannot be resumed with the same embeddings after a reload, and the roadmap's unchecked item for multiple file embeddings means one file at a time per session for now. The Docker note is a third: the container recipe is a development convenience, not a deployment artifact, and the README says so directly. There is also no mention anywhere in the supplied material of authentication, multi-user accounts, or a server-side fallback for clients without WebGPU. If your requirement is a shared internal assistant with access control, Chatty's model of one browser, one GPU, one user is the wrong shape, and no configuration in the README changes that.
What you would use instead, and how the approach differs
The natural alternative is Ollama with an existing chat front end such as Open WebUI. The difference is architectural, not cosmetic. Ollama runs the model as a local server process on a machine you control, exposes an HTTP API, and lets multiple clients connect to the same model instance. Chatty runs the model inside the browser tab through WebLLM, so there is no server process, no API surface, and no shared endpoint, but also no way for a phone or a second laptop to use the GPU on your desktop. Ollama also moves the GPU requirement off the client, which means a user on a weak laptop can still chat if the server has the hardware. Chatty's advantage is deployment surface: a static-ish Next.js app plus a model download, with the README able to claim that data is processed only on the user's own machine. If your constraint is that no prompt may transit a server you operate, Ollama on a shared host does not satisfy it. If your constraint is many users and one GPU, Chatty does not satisfy it. A second alternative is the hosted chattyui.com instance the README links for people who just want to try the app; it removes the install step but obviously changes the trust boundary, since you are trusting someone else's page to keep the local-execution property intact.
Maintenance, upgrades and the MIT licence
Chatty is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is a permissive starting point for forking, and the README actively invites contributions, pointing to a CONTRIBUTING.md. It is not legal advice, and anyone shipping a modified build should read the licence text and the licences of the bundled dependencies, since WebLLM, LangChain and the HuggingFace-hosted models carry their own terms. On maintenance, the release history in the supplied material shows three releases: 0.1.0 in June 2024, 0.2.0 in July 2024 adding Llama3.1 and Qwen2 support, and 0.3.0 in January 2025 adding DeepSeek R1 reasoning models. That cadence suggests the project tracks new open-weight models rather than freezing an interface, which is good for relevance and less good for stability: a model swap can change output quality and memory needs without any change on your side. The two unchecked roadmap items, multiple file embeddings and a prompt management system, are the features most likely to force you to patch the code yourself in the meantime. The upgrade cost that actually matters is not npm install. It is the re-download of model weights and the re-validation of GPU memory headroom on whatever hardware your users have.
Who should adopt Chatty, and what to check first
Adopt it if you are a single user or a small team with capable GPUs who want a familiar chat interface where prompts and documents never leave the machine, and if you are comfortable running a Next.js dev server or building your own production image from the provided Dockerfile. The MIT licence makes forking realistic, and the file-chat pipeline using all-MiniLM-L6-v2 embeddings plus MemoryVectorStore is small enough to reason about in an afternoon. Do not adopt it as a multi-tenant service, as a replacement for a server-hosted inference stack, or as a drop-in production container, because the README itself flags the Dockerfile as unoptimized and says nothing about authentication. Before you commit, check three things on your own hardware: that WebGPU is actually available in the browser you will target, that your GPU memory matches the README's figures for the model size you plan to run, and that a first-run model download is tolerable for your users. If any of those three fails, the honest answer is that Chatty is a demo-grade local client rather than the foundation for a product, and Ollama on a machine you own is the shorter path.
Editorial conclusion
Adopt Chatty if you want a self-hosted, single-user chat UI where prompts never leave the machine and you can accept a 6GB GPU class requirement for 7B models. Do not adopt it if you need multi-user accounts, server-side inference, or a hardened production container, since the README states the Dockerfile is not yet optimized for production. Before committing, verify that WebGPU is available in your target browser, that your GPU has the memory the README specifies for the model size you intend to run, and that a first-load model download is acceptable for your users.
Community notes