Vector Database from Scratch trades an HTTP API for six days of DataFusion internals

Six days, one database capability per day

The guided Rust course is structured as six days, and each day adds exactly one database or indexing capability. Every day ships with starter code, focused tests and a completed reference, so the shape of the exercise is fill in the gap rather than start from an empty file. The book carries the setup, the checkpoint commands, the SQL walkthroughs and the benchmark instructions. Entry requirements are stated in one line and they are narrower than the subject suggests: you need basic Rust, and you do not need prior knowledge of vector search or DataFusion. What the course deliberately does not do is wrap the work in a service. The framing is the boundary where vector-search algorithms become database features, so the algorithms, the evaluation contracts, query planning and the execution path stay visible instead of sitting behind an HTTP API or an ANN library.

Four indexes, and an evaluation contract for each

The index list is the syllabus. By the end of the course the database carries IVFFlat, NSW, HNSW, and residual IVF-PQ, which is four different approximate-search strategies rather than four parameter settings of one. Alongside them sit SQL commands for creating indexes and comparing query plans, so the index choice is something you can inspect through the planner instead of only through timing. Approximate results are compared against exact search throughout, and the shared benchmark reports three quantities on SIFT1M: recall, build time, and search latency. Splitting those three is the point of the exercise, because an index that raises recall while doubling build time is a different decision from one that only changes latency, and a single throughput number hides the trade.

DataFusion arrives with its default features switched off

The workspace manifest shows how little of the query engine the course depends on. DataFusion is requested at version 54 with default-features set to false and only two features named, nested_expressions and sql. What is built on top is an Arrow-backed in-memory table and, in the same list, a DataFusion optimizer rule described as safe. Asking for the SQL feature and nothing else keeps the engine small enough that a learner can still hold the execution path in their head, which is the trade the course description makes when it says it covers the execution path rather than hiding it. Tokio is pulled with macros and rt-multi-thread, async-trait is there for the async interfaces, and sqllogictest at 0.29 rounds out the six workspace dependencies.

The starter and the solution share a version and not a name

Six crates make up the workspace: xtask, vector-db-benchmark-support, vector-db/core, vector-db/datafusion, vector-db-starter/core and vector-db-starter/datafusion. All three published path dependencies carry the version 0.2.0-alpha.1, which is also the workspace version, so a day of work and a finished day are the same release number. Their names are not the same, though. The solution crates are vector-db-from-scratch-core and vector-db-from-scratch-benchmark-support, while the starter side is imported as vector-core, which is a rename of a package actually called vector-db-from-scratch-core-starter. That renaming is done with the package key so the two can sit side by side in one dependency list, and it is worth reading before you wire anything up. vendor/ holds a vendored datafusion-cli at 54.1.0, excluded from the workspace members list, and unsafe_code is forbidden across every crate.

The code is Apache 2.0 and the book is not

The licence section splits the repository in two and the second half is the one to read twice. Code in this repository is under the Apache License 2.0. The book, and that includes its Markdown and its figures, is under CC BY-NC-SA 4.0, which carries a non-commercial clause as well as the share-alike requirement. For a course that is defensible, but it means the prose cannot be folded into commercial internal documentation while the code next to it can. Two git submodules, bustub-vectordb-starter and bustub-vectordb-solution, retain their own upstream licence terms, so a third regime applies to those directories. There are no GitHub releases for the repository at all, which means the alpha version in the manifest is the only version number in play and nothing has been tagged for others to install.

The C++ track is still reachable and explicitly unmaintained

The course originally shipped as a C++ and BusTub edition, and that version remains online at its own path for existing readers. The page is explicit about its status: deprecated and unmaintained. The same wording shows up in the repository description, which points at Rust and DataFusion as the current track and marks the C++ one deprecated. What this means in practice is that a search for the course can land you on pages that no longer receive work, and that the two editions are not interchangeable, since the Rust one is built on Arrow and DataFusion while the older material is not. For someone arriving fresh, the Rust path is the only one being written; for someone with the old material in progress, the submodules at the root are still the entry point.

One dataset, three measurements, and a community server

The benchmark section names its dataset and its metrics rather than leaving them open: SIFT1M is the corpus, and recall, build time and search latency are what gets recorded. A shared benchmark support crate at the root of the workspace is what makes those numbers comparable across the four indexes, and it is published as its own package rather than living inside the database crate, which is what stops a learner from quietly changing the measurement while changing the index. The course itself also expects SQL to be part of the answer, since creating indexes and comparing query plans are both SQL commands in the deliverable list. Around the material there is a Discord server for people studying the course together, and the book is the only place the checkpoint commands and the walkthroughs are written down.

Editorial conclusion

This course fits a systems or database engineer who wants to see where a vector index stops being an algorithm and becomes a database feature, because the query planner, the optimizer rule and the SQL surface are all in scope rather than hidden. It is a poor fit for an application team that needs a running vector store this week, since six days of exercises are the deliverable and there is no service to deploy. Before starting, read the licence split: the Rust code is Apache 2.0 and permissively usable, while the book text and figures are CC BY-NC-SA, which bars commercial use of the material. Treat the C++ and BusTub pages as a read-only archive, and check that your Rust version meets the workspace minimum.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. skyzh/vector-db-from-scratch on GitHub