Feldera: SQL views that update from changes only, with a heavy source build and fault tolerance still in preview

A pipeline is a set of SQL tables and views that react to changes

The README defines the unit of work simply. A Feldera pipeline is a set of SQL tables and views, and views can be deeply nested. You start, stop or pause a pipeline to manage and advance a computation.

Input is a stream of changes: any number of inserts, updates or deletes to the tables. When changes arrive, Feldera updates all the views by looking only at the changes, and it avoids recomputing over older data. While a pipeline runs, you can inspect the results of the views at any time.

The README says this approach also gives unified offline and online compute over both live and historical data. Users have built batch and real time feature engineering pipelines, ETL pipelines, and incremental or periodic analytical jobs over batch data.

The core engine is published as the dbsp crate on crates.io, and the repository's crates/ directory holds dbsp next to the pipeline manager, connectors, sqllib, iceberg, storage and rest-api crates. sql-to-dbsp-compiler/ turns SQL into a program for that engine.

The SQL claim: joins, window functions, recursive queries and UDFs, evaluated incrementally

The first of seven listed features is full SQL support. The README says the engine is the only one in existence that can evaluate full SQL syntax and semantics completely incrementally. It lists joins and aggregates, group by, correlated subqueries, window functions, complex data types, time series operators, UDFs and recursive queries, and says pipelines can process deeply nested hierarchies of views.

That is a vendor claim, and a sweeping one. The README does not say which constructs are unsupported or which dialect it follows. The ad hoc query feature mentions minor dialect and rounding differences, which is the only caveat on the front page.

The consistency claim is more specific. Feldera is described as strongly consistent, and the README says it guarantees that the state of the views always corresponds to what you would get if you ran the queries in a batch system for the same input. It links a blog post titled synchronous streaming for the reasoning.

The README claims millions of events per second on a laptop, with no benchmark on the front page

On performance the README says Feldera is incredibly fast, with millions of events per second on a laptop. In the feature list, it says users have reported implementing complex use cases in 30 minutes or less and hitting millions of events per second without any tuning.

The README attributes these to users, and provides no benchmark table, hardware spec or query description beside them. The repository does contain a benchmark/ directory and a nexmark crate, so there is material to rerun, but the README does not point at it.

Datasets larger than RAM are also claimed. Feldera is designed to spill to disk, taking advantage of recent advances in NVMe storage. No size limit is given.

The opening paragraph compares Feldera to batch engines, warehouses, stream processors and streaming databases, and calls it more expressive and performant. No named competitor is measured. Run your own queries on your own data before accepting the ranking.

One docker run command starts the pipeline manager and a console on port 8080

The quick start needs only Docker:

docker run -p 8080:8080 --tty --rm -it images.feldera.com/feldera/pipeline-manager:latest

After the image downloads and the Feldera logo appears in the terminal, you open the WebConsole at http://localhost:8080. The README then points to a tutorial at docs.feldera.com. The container uses the latest tag, so two runs on different days can start different versions. Pin a tag if you need repeatable behaviour.

The --rm flag removes the container on exit. That is fine for trying the product, but the README gives no volume mount, so assume state is lost when the container stops unless the docs say otherwise.

For experiments with Kafka and other auxiliary services, the README sends you to Docker Compose instructions at docs.feldera.com/get-started.

Building from source needs 13 GB of disk, a JDK 19 or newer, and a locally built librdkafka

To run from sources, the README says to ensure at least 6 GB of free space in the sources directory and an additional 7 GB in your home directory. The dependency list is long: a Rust toolchain, a C and C++ compiler, cmake, libssl-dev, libsasl2-dev, zlib1g-dev, libzstd-dev, pkg-config, clang, graphviz, a JDK version 19 or newer (21 recommended), maven, Python 3.10, Bun and Node v20. golang-go is needed only to build with --features fips, which compiles aws-lc-fips-sys from source.

The Kafka connectors link librdkafka dynamically, so it must be installed first with ./scripts/install-librdkafka.sh. The script builds it against AWS-LC, and the README warns that distribution packages are built against OpenSSL and are usually older than the version the rdkafka-sys crate requires. Run it again after an rdkafka version bump, and set PREFIX to install elsewhere than /usr/local.

Then build the SQL compiler and start the manager:

cd sql-to-dbsp-compiler
./build.sh

cargo run --bin=pipeline-manager

The workspace sets rust-version 1.96.1 as its minimum supported Rust version.

Fault tolerance is a preview feature that depends on connector support

The README says Feldera can restart from the exact point of an abrupt shutdown or crash, without dropping or duplicating input or output. It then qualifies the claim: fault tolerance is a preview feature that requires support from input and output connectors.

So exactly once recovery is not a property of the engine alone. It holds only for connector pairs that support it, and the README does not list which ones do. If your pipeline reads from Kafka and writes to a warehouse, check both sides before you rely on it.

The connector list is broad, covering Kafka, HTTP, CDC streams, S3, data lakes and warehouses. If a connector is missing, the README asks you to open an issue.

Ad hoc queries work on a running or paused pipeline. They are evaluated in batch mode with Apache DataFusion, and their results match the incremental engine's output apart from minor dialect and rounding differences. That is useful for debugging materialized views, but it is a second engine answering your query.

Daily releases and a NOASSERTION licence field that needs a manual check

Releases are very frequent. v0.370.0 was tagged on 2026-10-28, v0.371.0 on 2026-10-29 and v0.372.0 on 2026-10-30, and the workspace version in Cargo.toml is already 0.373.0. The last push was on 2026-10-25 and the repository is not archived. Version numbers in the hundreds with daily tags mean you should pin an exact version, and read changes between versions before upgrading.

The licence is unclear from the metadata, which reports NOASSERTION. Cargo.toml says MIT OR Apache-2.0, and the README badge links to the MIT licence. A LICENSE file sits in the root. Read it before redistributing.

The repository also contains AI_POLICY.md, CONTRIBUTING.md and CODE_OF_CONDUCT.md, and an openapi.json describing the REST API, with a Python SDK under python/. The workspace authors are listed as the Feldera Team.

Editorial conclusion

Use Feldera if your workload is SQL views over changing tables and recomputing them in a batch warehouse is the cost you want to remove. Treat the README's performance and SQL coverage claims as the vendor's own statements and test them on your queries. Before depending on it, check which connectors your sources and sinks need, because fault tolerance is a preview feature that requires connector support, and confirm the licence terms, since the repository metadata shows NOASSERTION.

Frequently asked questions

Official sources

1. feldera/feldera on GitHub
2. Issues
3. Project website
4. README
5. Releases