# OpenData: four databases on one object-native foundation

> OpenData is a Rust project building open source databases on a shared object-native foundation, with Buffer, Log, Timeseries and Vector each keeping its own query layer while inheriting the same SlateDB storage engine and the same operational tooling. The interesting part is the second infrastructure layer, the parts that are still mostly unchecked boxes on the roadmap.

**opendata-oss/opendata** — OpenData is a collection of open source databases built on a common, object-native storage and infrastructure foundation. 

- Repository: https://github.com/opendata-oss/opendata
- Website: https://www.opendata.dev
- Stars: 541 · Forks: 48
- Language: Rust
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/opendata-oss-opendata

## One storage engine underneath four databases

OpenData is a collection of open source databases built on a common, object-native storage and infrastructure foundation, and the case for it is stated as a single sentence: object storage is the most durable, scalable and cost-effective storage system in the cloud, and OpenData builds first class open source databases on top of it.

The four databases today are listed with a one line description each. Buffer is high-throughput write-path buffering and ingestion pipelines. Log is replayable event streams and durable data movement. Timeseries is Prometheus-compatible metrics storage. And Vector is large-scale semantic search and retrieval.

The interesting part is the shared foundation, and it has two components. The first is SlateDB as the common storage layer, an object-store-native LSM tree that handles write batching, tiered caching and compaction, and provides snapshot isolation via atomic manifest updates using S3's compare-and-set. Each individual database implements its own domain-specific data structure and query layer on top of that.

The second is OpenData itself as the common infrastructure layer: service infrastructure, service catalog, admin tooling, distributed state infrastructure, configuration systems and testing frameworks, all shared by the databases above the storage line.

The stated benefit of inheriting those pieces is that there is only one storage engine and one set of operational tooling to learn across all systems, so individual databases can focus on their unique query semantics, data layout and optimisations.

The project is written in Rust, MIT licensed, and the homepage is www.opendata.dev. The recent releases are separate per database rather than a single version line: opendata-vector v0.2.2, opendata-timeseries v0.4.0 and opendata-log v1.0.0, all in June and July 2026. The last push was on 2026-08-07.

## The workspace lists members that the top level does not

The Cargo workspace manifest is the most concrete description of the project structure, and it is worth reading against the directory listing.

```toml
[workspace]
resolver = "2"
members = [
    "bencher",
    "buffer",
    "common",
    "keyvalue",
    "log",
    "log/bench",
    "log/c",
    "log/inspect",
    "timeseries",
    "timeseries/bench",
    "vector",
    "vector/bench",
    "macros",
]
```

The workspace resolver is version 2 and the workspace package version is 0.3.0 on edition 2024, which puts the code on the most recent Rust edition.

The members list shows what is actually built. There is the bencher crate, the buffer crate, the common crate, and the log crate with three sub-crates of its own: a bench crate, a c crate and an inspect crate. Timeseries has a bench crate alongside it, and vector has one too. The macros crate is the last member.

Now compare that with the top level directory listing, which holds .dockerignore, .github/, .gitignore, AGENTS.md, CONTRIBUTING.md, Cargo.lock, Cargo.toml, GOVERNANCE.md, LICENSE, MANIFESTO_SIGNERS.md, PROMPT.md, README.md, and then the bencher/, buffer/, common/, keyvalue/, log/, macros/, public/, queue/, rfcs/, timeseries/ and vector/ directories.

Three directories exist at the top level but are not workspace members. The keyvalue/ directory is one of them, and the queue/ directory is another, so those are either work in progress or not yet part of the build. The rfcs/ directory is the third, and that one is explained elsewhere in the documentation as the place where the open RFCs live, the active design efforts for people who want to build.

The common crate is versioned separately from the databases that depend on it. The common dependency is declared with a path, a package name of opendata-common and a version of 0.1.18, while buffer is declared with the package name opendata-buffer and version 0.3.0.

So the workspace version of 0.3.0 is the umbrella, and each database is published under its own name with its own version. That is consistent with the release tags, which are per database rather than per project.

## SlateDB and the dependency set that follows from it

The shared workspace dependencies are where the design decisions become visible as a list of crates, and two entries dominate the reasoning.

```toml
common = { path = "./common", package = "opendata-common", version = "0.1.18" }
buffer = { path = "./buffer", package = "opendata-buffer", version = "0.3.0" }
slatedb = { version = "0.14.0", features = ["compaction_filters"] }
slatedb-common = "0.14.0"
```

The SlateDB entries are pinned at 0.14.0, with the compaction_filters feature enabled on the main crate, and there is a separate slatedb-common at the same version. Pinning a shared storage engine to one exact version across the whole workspace is the point of the exercise, since every database inherits from it and a version drift between two of them would be a divergence in exactly the layer that was supposed to be uniform.

The rest of the list is the standard Rust asynchronous stack plus a set of choices that say something about the domain. Tokio with the full feature set, axum for the service layer with axum-extra for query and form handling, tower and tower-http with CORS, reqwest pinned with rustls-tls and default features off, and tracing with tracing-subscriber for logs and an env-filter.

There are also some entries that reflect the kinds of data the fleet handles. roaring for compressed bitmap sets, moka for concurrent caches with the future feature, foyer for storage, bytemuck with extern_crate_alloc for zero-copy serialisation, bitpacking with the bitpacker8x feature, and icu_segmenter for text segmentation.

The data-oriented choices are the ones worth noting. bitpacking exists to compress numeric columns densely, roaring to hold large sets of identifiers compactly, and bytemuck to reinterpret bytes as typed structs without copying. A metrics store and a vector store have very different payloads, and the shared dependency list is where you can see that the foundation is expected to handle both.

The configuration and serialisation stack is yaml and toml through config and figment, with serde and serde_json, serde_yaml, serde_with with a base64 feature for embedded binary values, and serde_derive style feature flags throughout. The command line side is clap with derive and env features, which is how the binaries pick up configuration from the environment. Testing uses rstest, and the project also depends on semver-adjacent tooling through the workspace.

And there is a metadata and identity layer: uuid, ulid for sortable identifiers, chrono for timestamps, and opentelemetry-proto for tracing export.

## A roadmap that is mostly infrastructure, and mostly unchecked

The roadmap section is revealing because it shows what the project considers unfinished, and the pattern is that the databases are further along than the shared services around them.

Each database has its own roadmap in its own README, and SlateDB has one as well, so the section here covers only the rest of the shared foundation. The items listed are a common service infrastructure with a server runtime supporting pluggable protocols, shared metrics and health checks. A service registry for discovering and browsing deployed databases. Admin tooling for deploying, teardown, upgrading, inspecting and migrating databases. Benchmark and regression testing frameworks. And distributed mode, which covers state sharding and request routing.

Every one of those is an unchecked box, which is the honest state of a project that describes itself as early and building in the open.

The bigger ideas for later are listed separately and are more speculative. A shared ingest layer, so that all writes go through a shared service for better batching, with a compactor framework that creates queryable state. Flexible deployment modes, described as embedded writers plus hosted readers, fully embedded, fully distributed and similar combinations. And deterministic simulation testing.

That last one is a research-grade ambition rather than a feature request. Deterministic simulation testing means a test can reproduce an exact sequence of inputs and get the same output every time, which for a distributed database is the difference between a reproducible bug report and a story.

The deployment modes point at the same thing the architecture is trying to solve. An embedded database that writes to object storage is a very different operational shape from a service with a query layer, and the shared foundation has to support both without the databases above having to care which one they are running in.

The project is explicit about its own stage. It says it is early and building in the open, with most discussions happening on Discord, and points contributors at the open RFCs, at issues labelled good-first-issue, or simply at filing a bug or a feature request. It also asks a question that is worth taking seriously for anyone deciding whether to depend on it: what databases should exist under OpenData, and what operational problems matter most.

## The governance files tell you how decisions get made

The top level of the repository carries more governance than most project of this size, and it says something about where this is heading.

There is a GOVERNANCE.md, a CONTRIBUTING.md, an AGENTS.md, a PROMPT.md, a MANIFESTO_SIGNERS.md and a LICENSE, alongside the code directories.

The MANIFESTO_SIGNERS.md file connects to the last line of the project documentation, which invites people to sign the manifesto at www.opendata.dev/manifesto. The file being in the repository, rather than only on the website, means the list of signatories is versioned with the project.

PROMPT.md is a file whose purpose the documentation does not explain, and it sits next to a rfcs/ directory that does get explained. The natural reading is that one of them is the instruction set for automated tooling and the other is the human design document store, but the documentation available here does not settle it, so it is worth reading both before contributing.

The AGENTS.md file suggests the same audience as the PROMPT.md, since both are conventional places for instructions aimed at automated coding assistants, and both are unusual to find in a repository that otherwise follows a fairly conventional layout.

The other half of the top level is ordinary infrastructure: a .github/ directory for workflows, a .dockerignore and a .gitignore, a Cargo.lock so the dependency versions are pinned for the build, a public/ directory holding the banner image referenced at the top of the documentation, and the LICENSE file.

The structure of the repository therefore mirrors the structure of the project itself. The databases are directories, the shared foundation is a crate called common, the design discussion has its own directory, and the decision-making has its own files at the root. For a project that describes itself as a fleet of databases rather than a single product, that organisation is the message.

## Conclusion

OpenData suits someone who wants to run a metrics store, an event log or a vector index without learning a separate operational stack for each, because the shared SlateDB layer and the planned service registry and admin tooling are the point of the project. It does not suit you if you need a distributed database today, since state sharding and request routing are still unchecked on the roadmap, and it does not suit you if you want a stable release, because the project describes itself as early and the individual databases are still at 0.x apart from the log store. Before you build on it, read the rfcs directory for the design efforts in flight and check which database versions you actually need, since each is released separately and the workspace version does not track them.

## FAQ

### What databases are included in OpenData?

Four. Buffer for high-throughput write-path buffering and ingestion pipelines, Log for replayable event streams and durable data movement, Timeseries for Prometheus-compatible metrics storage, and Vector for large-scale semantic search and retrieval.

### What storage engine do the OpenData databases share?

SlateDB, an object-store-native LSM tree that handles write batching, tiered caching and compaction, and provides snapshot isolation via atomic manifest updates using S3's compare-and-set. Each database implements its own data structure and query layer on top of it.

### How does OpenData version its releases?

Each database is published under its own name with its own version, while the workspace carries an umbrella version. Recent tags are opendata-vector v0.2.2, opendata-timeseries v0.4.0 and opendata-log v1.0.0, and the workspace package version in the manifest is 0.3.0 on edition 2024.

### What is still on the OpenData roadmap?

The shared foundation items are common service infrastructure with a pluggable protocol server runtime, a service registry for discovering deployed databases, admin tooling for deploy, teardown, upgrade, inspect and migrate, benchmark and regression testing frameworks, and distributed mode with state sharding and request routing. Every one of them is listed as unchecked.

### How can I contribute to OpenData?

The project says it is early and building in the open, with most discussion on Discord. You can work on the open RFCs in the rfcs directory, pick up an issue labelled good-first-issue, or file a bug or a feature request. There is also a manifesto at www.opendata.dev/manifesto with its signers recorded in MANIFESTO_SIGNERS.md.

## Sources

- [License: MIT](https://github.com/opendata-oss/opendata/blob/main/LICENSE)
- [opendata-oss/opendata on GitHub](https://github.com/opendata-oss/opendata)
- [Project website](https://www.opendata.dev)
- [README](https://github.com/opendata-oss/opendata/blob/main/README.md)
- [Releases](https://github.com/opendata-oss/opendata/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/opendata-oss-opendata