Serverless AI Chat with RAG Using LangChain.js: An Azure Sample Application

What the Application Does and Who It Is For

The sample is a document-grounded chatbot. You upload a set of documents, the application indexes them into a vector database, and users can ask questions about those documents through a chat interface. The foundation model retrieves relevant document chunks before generating a response, which is the Retrieval-Augmented Generation (RAG) pattern.

The README describes it as a starting point for building more complex AI applications. The target audience is TypeScript or JavaScript developers who want a working example of the full RAG stack on Azure, with all the infrastructure code included, rather than assembling the pieces from scratch. The sample ships with data from a fictitious company called Contoso Real Estate, covering terms of service, privacy policy, and a support guide, so the application can be tested immediately after deployment without uploading custom documents.

Architecture: Four Azure Services Plus LangChain.js

The application is composed of four Azure services working together. Azure Static Web Apps hosts the web front end, a single-page application built with the Lit web component library. Azure Functions hosts the serverless API, written in TypeScript with LangChain.js. Azure Cosmos DB for NoSQL serves as both the chat session store and the vector database, holding extracted text and the vectors generated by LangChain.js. Azure Blob Storage holds the original source documents.

The web app and the API communicate through the HTTP protocol for AI chat apps, referenced in the README as `https://aka.ms/chatprotocol`. The two packages live in separate directories: `packages/webapp` for the front end and `packages/api` for the Azure Functions backend.

The README lists the main features: serverless architecture through Azure Functions and Static Web Apps, RAG through Cosmos DB and LangChain.js, per-user chat session history, and local development using Ollama at no cost.

Local Development with Ollama

The README notes that the entire sample can be tested locally without any Azure account or cost using Ollama. The tip at the top of the README states to follow the local environment section for this path.

Local development requires Node.js LTS, the Azure Developer CLI, Git, and Azure Functions Core Tools (which the npm install process should handle automatically). On Windows, PowerShell 7 or later is required, with Git Bash or WSL as alternatives.

After cloning, local development starts both the webapp and API together:

npm start

This runs `npm:start:*` scripts concurrently, starting the webapp dev server and the Azure Functions host in parallel. Documents can be uploaded to the local instance using:

node scripts/upload-documents.js http://localhost:7071

The `http://localhost:7071` address is the Azure Functions local host. Replacing the URL with a deployed Azure Functions endpoint uploads documents to the cloud instance instead.

Deploying to Azure with the Azure Developer CLI

Azure deployment uses the Azure Developer CLI (`azd`), which reads the `azure.yaml` file at the repository root to understand the application structure. Deployment provisions all four services, configures them, and deploys the application code in a single command.

The quickest path to running the sample in a pre-configured environment is GitHub Codespaces. The README provides a button link to open the project directly in Codespaces with the required tools already installed. VS Code Dev Containers provide a similar environment locally using Docker.

For teams that need to replace the sample Contoso Real Estate data with their own documents, the upload script runs against any deployed Azure Functions endpoint. The `infra/` directory contains the Bicep infrastructure-as-code files that define the Azure resources, which can be modified before deployment to adjust resource sizes, regions, or storage settings.

Is Azure Functions a serverless app? Yes. Azure Functions is Microsoft's serverless compute service, and this sample uses it to host the RAG API. The API is stateless between invocations; Cosmos DB stores all persistent state including chat history and vector embeddings.

LangChain.js Integration: Ingestion and Retrieval

LangChain.js handles two phases in the application. During ingestion, it extracts text from uploaded documents, splits the text into chunks, generates vector embeddings, and stores the chunks and vectors in Cosmos DB. During retrieval, it takes a user query, generates a query embedding, performs a vector similarity search in Cosmos DB, retrieves the matching document chunks, and passes them as context to the language model.

The `packages/api` directory contains this logic inside the Azure Functions handlers. LangChain.js abstracts the embedding and retrieval steps, making it possible to swap the vector store or the embedding provider by changing the LangChain configuration rather than rewriting the retrieval logic.

The README notes that Cosmos DB for NoSQL serves as the vector database. This is a specific capability of Cosmos DB that must be enabled; it is not a default feature of all Cosmos DB configurations.

Limitations: Sample Scope and Azure Cost

The README explicitly describes this as a sample, not a production system. The Contoso Real Estate data is fictitious and designed for demonstration. Authentication, access control, document management, and administrative features are outside the sample's scope.

Deploying this application to Azure incurs costs. Azure Static Web Apps has a free tier, but Azure Functions, Cosmos DB, Azure Blob Storage, and Azure OpenAI or third-party model inference all carry charges based on usage. The README does not include a cost estimate.

The last push was on 2026-06-03. The repository has no GitHub releases. As both LangChain.js and the Azure SDKs evolve, specific API calls and configuration patterns in the sample may become outdated. The `AGENTS.md` file in the repository root suggests the project has explored or documents multi-agent configuration, but the README's feature list focuses on the single-chatbot RAG pattern.

The sample targets the Azure ecosystem specifically. Teams that want the same RAG pattern on AWS, GCP, or a self-hosted stack cannot directly reuse the infrastructure code, though the LangChain.js integration logic in `packages/api` is portable to other runtimes.

Comparing This Approach to the azure-search-openai-demo Repository

The azure-search-openai-demo repository is another Azure-Samples project that implements a similar RAG chatbot pattern, but using Azure AI Search as the vector store instead of Cosmos DB, and primarily targeting Python with an optional TypeScript frontend. It is one of the most widely referenced Azure RAG samples and appears in the related search data for this project.

The key architectural difference is the choice of vector store. Azure AI Search provides full-text search in addition to vector search, query expansion, and semantic ranking, which can improve retrieval quality on heterogeneous documents. Cosmos DB for NoSQL with vector search is a newer capability that consolidates the chat session store and vector index into a single service, reducing the number of Azure resources to manage.

For teams working primarily in TypeScript who want a single-service vector store and a serverless deployment model, this LangChain.js sample is more aligned. For teams that want mature search tooling with hybrid retrieval and a broader community of deployment guides, the azure-search-openai-demo Python sample is better documented.

Editorial conclusion

This sample suits TypeScript developers who want to understand how to wire LangChain.js into a serverless Azure deployment with a vector store and chat history, and who want working infrastructure-as-code to adapt for their own documents. It is not a production-ready application: the README describes it as a starting point and the sample data is a fictitious company. Before deploying to Azure, review the costs for Azure Functions, Cosmos DB, Azure Blob Storage, and the chosen Azure OpenAI or third-party model, since none of these are free at scale.

Frequently asked questions

Official sources

1. Azure-Samples/serverless-chat-langchainjs on GitHub
2. Issues
3. License: MIT
4. README