aws-genai-llm-chatbot: A CDK Blueprint for Multi-Model RAG on AWS

The gap this blueprint fills is assembly, not model access

Calling a foundation model through Amazon Bedrock is a few lines of SDK code. What takes weeks is everything around it: an identity layer, a document store, a vector index, a retrieval path that respects per-user permissions, a conversation store, and a web front end that non-engineers can actually use. The repository positions itself as a solution for exactly that surrounding work. The README describes it as an enterprise-ready chatbot with RAG capabilities, and the topic list on the repository points at the specific services involved: bedrock, kendra, opensearch, opensearch-serverless, pgvector, aurora, sagemaker, langchain.

The intended audience is an engineering team inside an AWS account, not an end user shopping for a hosted chat product. The prerequisites confirm this. You need an AWS account with appropriate permissions, the AWS CLI configured with credentials, Node.js 18 or later, Python 3.8 or later, and a CDK CLI compatible with aws-cdk-lib 2.206.0. That is a deployment operator's checklist. Someone who only wants to chat with documents will find nothing here they can use directly.

What CDK actually provisions, and where the retrieval path lives

The README lists the architecture components plainly: Amazon Bedrock for LLM access, Amazon OpenSearch for vector storage, Amazon S3 for document storage, Amazon Cognito for authentication, AWS Lambda for serverless processing, Amazon API Gateway for API access, and a React-based web interface. Each of those is a separate CDK construct, which is why the project calls itself modular. You can in principle swap the vector store or the model provider without rewriting the application layer.

The model layer is deliberately plural. The description names Amazon Bedrock, Anthropic, HuggingFace, OpenAI, Meta, AI21, Cohere and Mistral, and the README narrows the shipped paths to Bedrock (Claude, Llama 2), SageMaker, and custom model endpoints, plus something the README calls GenAIEH Gateway integration. That last item is the least explained in the material available. If you need a provider that is not Bedrock or SageMaker, treat the custom endpoint path as the extension point and expect to read the CDK source rather than the README.

Retrieval is the part worth scrutinising before you commit. The topics list includes both OpenSearch and OpenSearch Serverless, plus pgvector on Aurora. That means the repository supports more than one vector backend, and the choice is a deployment-time decision with real cost and operational consequences. Serverless removes capacity planning but constrains how you can tune the index. Provisioned OpenSearch gives you control and a standing bill. The README does not walk through that trade-off, so it is on you.

Getting it running: the commands the README actually gives

The documented path is short. Install or update the CDK CLI globally:

npm install -g aws-cdk@latest

Then verify it:

cdk --version

The README is explicit about why this matters. The CDK CLI version must be compatible with the aws-cdk-lib version used in the project, currently 2.206.0, and a mismatch surfaces as a Cloud assembly schema version mismatch error during deployment. That error is not a hint about your application code. It is a toolchain version conflict, and the documented fix is to update the CDK CLI.

Beyond that, the README says deployment is fully automated using AWS CDK and SeedFarmer, and points at the GitHub Pages site for complete documentation. It does not reproduce the full deploy command sequence in the excerpt available. If you are evaluating this for adoption, the honest position is that the bootstrap and deploy steps live in the linked documentation, and you should read them there before promising a timeline to anyone.

The configuration surface is where the real work sits. Model selection, vector store selection, authentication settings and data source connections are all CDK context or config values rather than a single YAML file, which is normal for CDK projects but means a change to the model provider is a code change, not a settings toggle.

The version coupling is a maintenance tax, not a one-time setup step

Pinning aws-cdk-lib at 2.206.0 and requiring a matching CLI is a reasonable engineering choice. It is also a recurring obligation. Every developer who clones the repository needs the right CLI version, and CI needs it too. The README anticipates the failure mode by telling you to run npm install -g aws-cdk@latest when the schema mismatch appears, which works until a newer CDK CLI drops support for the schema your pinned library emits. At that point you upgrade aws-cdk-lib, which means touching constructs, which means re-testing the deployment.

The release cadence visible in the material makes this concrete. v5.0.0 landed in January 2025, after v4.0.14 in August 2024 and v4.0.13 in June 2024. That is a major version roughly every eighteen months with patch releases between. A major bump in a CDK blueprint usually means breaking changes to construct interfaces or removed deployment options, so plan for a migration window rather than a drop-in upgrade.

There is also the SeedFarmer dependency mentioned in the deployment section. Any additional orchestration layer is another thing your platform team has to understand and keep current, and the README does not describe what SeedFarmer handles versus what plain CDK handles.

Where this is the wrong tool

If you want a chatbot and not a platform, this is the wrong choice. The stack requires you to operate Cognito user pools, an OpenSearch domain or collection, S3 buckets, Lambda functions and an API Gateway stage. Each has its own quotas, its own IAM surface and its own failure modes. A team of two building an internal FAQ bot will spend more time on AWS plumbing than on retrieval quality.

The cost profile is also not obviously favourable at small scale, and the README does not give figures. OpenSearch Serverless has a minimum capacity charge that applies whether or not anyone sends a query. Provisioned OpenSearch bills for instances around the clock. Token usage tracking is listed as a feature, which tells you the project expects you to watch model spend, but it does not tell you what the baseline infrastructure spend is. Run your own numbers against the AWS pricing pages for your region before assuming this is cheaper than a managed alternative.

Finally, the RAG quality ceiling is set by your chunking and embedding choices, not by the fact that the stack supports multiple vector stores. Swapping from OpenSearch to pgvector will not fix retrieval that returns the wrong passages. If your problem is answer quality rather than deployment, this repository does not solve it for you.

How it differs from LangChain-only or Bedrock-only starting points

The topic list includes langchain, so the retrieval orchestration is likely built on that library rather than on hand-rolled prompt assembly. The meaningful comparison is with two other starting points.

A plain LangChain application gives you chains, retrievers and memory abstractions, and nothing else. You still choose and provision a vector store, still build authentication, still write a front end, still deploy. It is a library, not a deployment. The aws-genai-llm-chatbot repository is the opposite trade: it makes the infrastructure decisions for you through CDK constructs and asks you to accept its service choices, or fork them.

A Bedrock-only integration, such as calling the Converse API from a Lambda behind API Gateway, gives you the smallest possible surface. One function, one endpoint, one IAM role. You get no document ingestion pipeline, no vector index, no conversation persistence beyond whatever you write yourself, and no web UI. For a narrow internal tool that answers questions from a fixed prompt, that is often the better engineering decision. This repository earns its complexity only when you genuinely need multi-source retrieval with per-user access control.

The distinguishing mechanism here is the combination of Cognito for identity and OpenSearch for vectors inside one CDK app. That pairing is what lets retrieval be filtered by who is asking, which is the hard part of enterprise RAG and the reason the blueprint exists.

Licence and the cost of forking

The repository is MIT-0. That is the MIT licence with the attribution requirement removed: you can use, modify and redistribute the code, including in commercial and closed-source products, without carrying a copyright notice. For an AWS sample blueprint this is the permissive end of the spectrum, and it is a deliberate choice to lower the barrier for enterprises that do not want licence review in the critical path. This is a description of the licence text, not legal advice; have counsel confirm it against your own policy.

The practical cost of forking is not the licence, it is the divergence. Once you modify the CDK constructs to fit your account structure, you own those changes. Upstream v5.0.0 will not merge cleanly into a fork that has restructured the networking or replaced the vector store. Budget for either tracking upstream closely and keeping changes shallow, or accepting that you have forked permanently and will cherry-pick security fixes by hand.

There is no published support commitment in the material. It is an aws-samples repository, which in practice means it is maintained by AWS engineers as a reference rather than as a product with an SLA. Treat the issue tracker as a place to learn, not a place to escalate.

Who should deploy this, and what to check first

This fits a platform or applied-AI team that already runs CDK in production, has an AWS account with Bedrock model access granted, and needs a multi-source RAG chatbot with real authentication. It fits less well if you are prototyping on a laptop, if you have no OpenSearch operational experience, or if a single-model integration would answer your actual question.

Before you commit, verify three things. First, run cdk --version and confirm it against the aws-cdk-lib 2.206.0 pin, because the schema mismatch error is the first thing that will stop you. Second, confirm which Bedrock models your account and region actually permit, since the README's model list is broader than the shipped Bedrock path of Claude and Llama 2. Third, price the OpenSearch option you intend to use, because that line item dominates the bill long before token spend does.

The repository is a blueprint, and blueprints are judged by how much of your own architecture you can keep. Here you keep the RAG pipeline shape and the CDK patterns, and you replace the service choices that do not match your account. That is a fair trade for a team that wants a head start. It is a poor trade for a team that wanted a product.

Editorial conclusion

Adopt it if you want a working multi-model RAG reference you can fork and modify, and if your team already writes CDK. Do not adopt it if you need a drop-in product, if you cannot accept the CDK CLI and aws-cdk-lib version coupling, or if your organisation has no appetite for owning OpenSearch, Cognito and API Gateway. Before deploying, verify the CDK CLI version against the aws-cdk-lib 2.206.0 pin in the project, confirm the Bedrock model access your account has been granted in the target region, and read the MIT-0 licence text rather than assuming it matches Apache-2.0 or MIT.

Official sources

1. aws-samples/aws-genai-llm-chatbot on GitHub
2. License: MIT-0
3. Project website
4. README
5. Releases