serverless-chat-langchainjs: a RAG chat template wired to Azure Static Web Apps, Functions and Cosmos DB

The problem it solves is assembly, not retrieval quality

Nothing in this repository is a novel retrieval technique. The value on offer is that someone at Microsoft already decided how a RAG chat app should be split across services and wrote the glue: a Lit web component for the chat surface, an Azure Functions API that runs LangChain.js, Cosmos DB for NoSQL as the vector store, and Blob Storage for the source documents. The README frames the goal plainly, saying the sample can be used "as a starting point for building more complex AI applications." That is the honest description. If you have built a RAG pipeline before, you will recognise every stage; what you save is the wiring between them and the deployment scripts. The intended reader is a TypeScript developer who wants a deployable chat experience over a document set and does not want to choose a vector database, a hosting model and a front-end stack from scratch. The sample data is a fictitious company called Contoso Real Estate, with documents covering terms of service, a privacy policy and a support guide. That scenario is narrow on purpose: a support chatbot answering questions about policy text. Applications with different shapes, such as agents that call tools or workflows that fan out across many documents, are outside what the sample demonstrates.

Two Functions, one vector container, and the HTTP chat protocol in between

The architecture splits into four pieces. The web app lives in `packages/webapp` and is a single chat web component built with Lit, hosted on Azure Static Web Apps. The API lives in `packages/api`, runs on Azure Functions, and uses LangChain.js for two distinct jobs: ingesting documents and generating responses to user queries. Cosmos DB for NoSQL holds both the chat sessions and the vectors that LangChain.js produces, plus the text extracted from the documents. Blob Storage holds the original source files. The web app and the API talk over the HTTP protocol for AI chat apps, which the README links as `aka.ms/chatprotocol`. That protocol choice matters more than it looks. It means the chat transport is a documented contract rather than an ad hoc fetch call, so a different front end could in principle speak to the same Functions backend. The ingestion path and the query path are separate concerns sharing one database: documents land in Blob Storage, the API extracts and embeds them, and the resulting vectors sit alongside the extracted text in Cosmos DB. Chat history is stored per user, which the README lists as a feature ("Maintains a personal chat history for each user"). One thing the README does not spell out is how re-ingestion behaves when a source document changes. There is no described upsert or delete strategy for stale vectors, so treat that as an open question to answer in the code before you put real documents behind it.

Getting it running: azd, Codespaces, or Ollama with no cloud spend

The README offers three routes. The fastest is GitHub Codespaces, opened through the Codespaces badge, which gives a preconfigured environment. The third is fully local. The middle route, a local machine against Azure, needs Node.js LTS, the Azure Developer CLI (`azd`), Git, and on Windows PowerShell 7+ with the note that `pwsh.exe` must be runnable from a PowerShell command; Git Bash or WSL also work for the `azd` commands. Azure Functions Core Tools is listed as normally installed through NPM, with a manual install only if the API fails to start. The README states the Node.js requirement as >=20 in the badge, so a Node 18 environment is out. The local-only path is the one worth calling out, because it is the cheapest way to evaluate the sample: the README says you can "test this application locally without any cost using Ollama," with Llama 3.1 as the model named in the badge. That path still requires following the local environment instructions rather than the Codespaces quickstart, and the README marks this with an important note. The project code is retrieved by forking the repository; the README's getting-started section begins with a fork step before the environment setup. Because the README text supplied here is truncated mid-sentence at that fork instruction, the exact `azd` subcommands and environment variable names beyond the tool list cannot be confirmed from this material. Do not assume `azd up` is the provisioning command without checking the repository's own documentation.

Ollama locally, Azure OpenAI in the cloud: the swap is the interesting part

The README presents local Ollama development as a cost-avoidance measure, and that is its real function. It lets you exercise the ingestion pipeline, the Cosmos DB writes and the chat component without paying for model inference. The trade-off is that the two environments are not identical. A local Llama 3.1 model and a hosted Azure model differ in context window, latency and output style, so prompt behaviour you tune locally may not transfer. The sample does not claim parity, and you should not assume it. The more consequential design point is that the model provider is abstracted behind LangChain.js rather than hardcoded to a single Azure service. That is what makes the Ollama path possible at all. If you are evaluating this repository, the useful exercise is to trace where the chat model is instantiated in `packages/api` and confirm how the provider is selected. That single location tells you how much work a provider swap costs, which is the question most teams actually have when they consider adopting a template like this.

Where the template pushes you toward Azure and holds you there

The constraints are structural, not incidental. Cosmos DB for NoSQL is the vector database, and the README links specifically to the NoSQL vector documentation, which means the vector search capability is tied to that one Azure service. If your organisation standardises on a different store, or if you need hybrid search features Cosmos DB does not offer, you are replacing the retrieval layer, not configuring it. Blob Storage is the document source, so ingestion assumes files land in an Azure storage account. Static Web Apps hosts the front end, which fixes the deployment model and its routing and authentication conventions. None of these are wrong choices; they are choices, and each one is a place where leaving the template costs more than starting inside it. The second limitation is maturity signalling. The repository has no published releases, and the README's own framing is that of a sample. Samples are maintained on a different schedule than products. The last push date is recent, which suggests activity, but activity is not the same as a support commitment, and there is no versioned artifact to pin to. The third limitation is scope. The feature list is a chat with RAG, session history and serverless deployment. There is no mention of evaluation tooling, no guardrails layer, no citation rendering, and no described strategy for document updates. If your requirements include any of those, you are building them, and the template's contribution is the scaffolding around them.

What a hand-rolled LangChain.js service would look like instead

The obvious alternative is not another Azure template. It is writing the same pipeline yourself: a Node service that loads documents, splits them with a LangChain.js text splitter, embeds them, writes to whichever vector store you prefer, and exposes a chat endpoint. The difference in approach is where the decisions live. In this template, the decisions are made for you and encoded in the deployment scripts and the folder split between `packages/webapp` and `packages/api`. In a hand-rolled service, you pick the vector store, you pick the host (a container, a long-running Node process, a different serverless platform), and you pick the transport. That freedom costs you the deployment automation and the pre-wired front end. A second alternative is a managed RAG service, where the ingestion, chunking, embedding and retrieval are handled by a vendor and you supply documents and a query endpoint. That removes the LangChain.js code entirely but also removes your ability to inspect or modify the retrieval step, which is exactly the step most teams end up tuning. The honest comparison is this: the template wins when Azure is already your platform and a chat-over-policies demo is close to your use case; a hand-rolled service wins when the vector store or the hosting model is a decision you need to own.

Maintenance cost and the MIT licence boundary

The licence is MIT, which is permissive and places few obligations on how you reuse the code. This is not legal advice, and the practical questions are about dependencies rather than the repository's own licence. You inherit LangChain.js as a dependency, and LangChain.js moves quickly, so the cost of adopting this template includes tracking that library's changes. You inherit the Azure SDKs for Functions, Cosmos DB and Blob Storage. You inherit a Node.js >=20 floor. The repository has no releases, so there is no upgrade path in the conventional sense: you fork, you pull changes from upstream if you choose, and you resolve conflicts yourself. That is a real cost, and it is the cost of every sample-derived codebase. The mitigation is to treat the fork as your code from day one rather than as a dependency you track. The README also points to official Microsoft Learn documentation for the template and to a dev.to walkthrough, which are the places to look when the in-repo README is thin. Given how little of the deployment mechanics is visible in the material supplied here, those linked resources are not optional reading; they are where the actual instructions live.

Editorial conclusion

Adopt this template if you are already committed to Azure Static Web Apps, Azure Functions and Cosmos DB for NoSQL, and you want a working RAG chat skeleton with a Lit web component and the HTTP protocol for AI chat apps rather than a blank repository. Do not adopt it if your retrieval needs span multiple vector stores, if you cannot host your documents in Azure Blob Storage, or if you want a framework you will extend for years: this is a sample with no published releases, and the repository layout is organised around a demo scenario. Before committing, verify the Node.js version constraint (>=20), confirm that `azd up` provisions the Cosmos DB vector container and the Blob Storage account in your target region, and check whether the HTTP chat protocol version used by `packages/webapp` matches what your own front end expects.

Official sources

1. Azure-Samples/serverless-chat-langchainjs on GitHub
2. Issues
3. License: MIT
4. README