WandBot: how Weights & Biases answers support questions in Discord and Slack

What WandBot answers and where it runs

WandBot is a support assistant for two Weights & Biases products, Experiment Tracking and Weave. It answers questions from a corpus built out of the wandb documentation and code, the Weave documentation and code, example Colab notebooks, and educational content, and the README credits the v1.3.0 index with pulling all of those together at chroma_index:v50.

The repository description lists four surfaces it can run in: Discord, Slack, ChatGPT and Zendesk. The README's feature list names Discord and Slack integration specifically, and the packaging backs that up, since the optional bots extra in pyproject.toml carries discord>=2.3.2, slack-bolt>=1.18.0 and slack-sdk>=3.21.3.

The retrieval pipeline after v1.2.0

Version 1.2.0 is where the architecture took its current shape. The release notes for that version describe replacing llama-index with LCEL so LLM calls run in parallel, moving the vector store from FAISS to ChromaDB to get metadata filtering and speed, cutting the query enhancer down to a single LLM call, and splitting the RAG pipeline into three modules: query enhancement, retrieval, and response synthesis.

Two retrieval refinements came with that split. Parent document retrieval pulls a wider chunk around a matched passage so the answer has more context, and sub-query answering breaks a complex question into parts that get answered separately before synthesis. The API was restructured at the same time into separate routers for retrieval, database and chat operations. v1.3.0 later removed the retriever endpoint for now and added a new one, according to its changelog.

The model stack behind one answer

The feature list names five components: a hosted ChromaDB vector store, OpenAI v3 embeddings, Gemini flash-2.0 for query enhancement, GPT-4o for response synthesis, and a Cohere re-ranking model. The v1.3.0 changelog shows that lineup was itself a change, since query enhancement moved from gpt-4o to Gemini flash-2.0 and re-ranking moved from rerank-v2.0 to rerank-v3.5 in the same release.

A correction sits in the notes under the evaluation table: v1.3.1 switched the response synthesizer to Claude Sonnet-3.7, away from gpt-4o-2024-11-20. So the feature list and the release notes disagree about which model composes the final answer, and the release notes are the more recent statement. The dependency list in pyproject.toml keeps both paths installed, with openai>=1.71.0, anthropic>=0.49.0 and google-genai>=1.9.0.

The README also mentions a fallback mechanism for model selection, and periodic data ingestion with report generation whose output is published as a W&B report.

Installing WandBot from the repository

The README has an Installation section, but the copy of the README available for this review ends at that heading, so the exact commands are not quoted here. What the checked-in packaging does state is precise. pyproject.toml sets the project metadata like this:

[project]
name = "wandbot"
version = "1.3.3"
requires-python = ">=3.12,<3.13"

So the current source is version 1.3.3 and it needs Python 3.12, not 3.13 and not anything older. The v1.3.0 changelog says package management moved to uv, and the repository carries a uv.lock, which is the path the maintainers now use. The dependencies that matter for a first run are pinned or floored in the same file:

dependencies = [
    "chromadb==1.0.3",
    "slack-bolt>=1.18.0",
    "discord>=2.3.2",
    "cohere>=5.13.0",
    "weave>=0.52.10",
]

The bots extra, listed under [project.optional-dependencies], is where the Slack and Discord clients live, so a chat deployment needs that extra and a plain API deployment does not. Before the first query returns anything useful you also need the vector store populated, which is what the ingestion job and its published report are for.

What the evaluation numbers actually show

The README publishes response correctness by version, judged by an LLM. English scores run 53.8 percent at 1.0.0, 72.5 percent at 1.1.0, 81.6 percent at 1.2.0, then 91.2 percent at 1.3.0 and 1.3.1, 90.4 percent at 1.3.2, and 82 percent at 1.3.3.

That last step down is worth attention. The v1.3.3 row carries the comment that it uses a v61 index, 500 token Flash thinking, Flash Lite stable and a gpt-4o judge, plus a knowledge base update. Earlier rows used v50, v52 and v54 indexes. So the drop from 90.4 to 82 percent happened across an index change and a model change at the same time, and the README does not separate the two.

Two other caveats are printed with the table. The 1.3.0rc rows show the same system scored 71.3 percent by a gpt-4-preview judge and 88.8 percent by a gpt-4o judge, which tells you the judge moves the number by a wide margin. And the README notes that evaluations before the v1.3.0 production run had 10 to 12 errors out of 490 total calls, so some noise is present. Trials are single runs for 1.0.0 through 1.2.0 and five runs from 1.3.0rc onward.

Japanese support is measured separately and stops earlier: 56.3 percent at 1.2.0 and 71.9 percent at 1.2.1 after a translation step was added. The v1.3.1 notes add that the updated index excludes Korean and Japanese versions of the docs because they are old, along with Fully Connected blog posts.

Where WandBot is the wrong tool

It answers from an ingested corpus, and v1.3.0 turned web search off. Anything newer than the last ingestion, or outside wandb and Weave documentation, code, example notebooks and educational content, has no path into an answer.

It is also tied to a specific stack of paid APIs. Hosted Chroma, OpenAI embeddings, a Gemini or OpenAI or Anthropic model for synthesis, and Cohere re-ranking all bill separately, and the repository adds Weave for tracing plus google-cloud-bigquery in its dependency list. A team that wanted a support bot over its own documentation would be replacing most of that, not reusing it.

The licence is Apache-2.0, stated both in pyproject.toml and in the LICENSE file in the repository listing, which removes the usual blocker on internal reuse. The cost that remains is operational: the index has to be rebuilt and re-evaluated, and the published history shows that rebuilding it moved correctness by several points in both directions.

The stack WandBot moved away from

An obvious alternative is the stack this project used before v1.2.0: llama-index for orchestration over a FAISS index. The difference in approach is visible in the changelog reasons given for leaving it. LCEL replaced llama-index so that LLM calls could run in parallel, and ChromaDB replaced FAISS to get metadata filtering, which FAISS does not give you.

Choosing llama-index and FAISS today means accepting serial LLM calls and doing your own metadata filtering in application code, in exchange for a single library that bundles retrieval and synthesis instead of the three separate modules WandBot now maintains. For a small corpus with few metadata constraints, that trade is reasonable. The reason WandBot made the opposite call is that its corpus spans documentation, source code, notebooks and blog content, which is exactly where filtering by source type starts to matter.

Repository state and running cost

The repository carries src/, tests/, modal/ deployment files, an e2b Dockerfile and build script, eval_requirements.txt, uv.lock, plus CONTRIBUTING.md and SECURITY.md. The last push was on 2026-05-20. The most recent tagged release in the data available is v1.3.2, published 2025-05-20, while pyproject.toml already reads version 1.3.3, so tags and source version are not in step.

Upkeep here is not mostly about code. The recurring work is ingesting a fresh knowledge base, rebuilding the Chroma index, and re-running the Weave evaluation, which is why the README links a data ingestion report for each index version.

Editorial conclusion

Use WandBot if you run Weights & Biases or Weave support in Discord or Slack and want answers grounded in the current documentation rather than a general model's memory. Do not use it if your questions fall outside that corpus, since web search was turned off in 1.3.0. Before deploying, check which index version you are ingesting and re-run the Weave evaluation yourself, because the published score moved from 91.2 percent to 82 percent across the last two index updates.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. wandb/wandbot on GitHub