Apache Fluss: A Streaming Storage Layer for Lakehouse Analytics

What Problem It Solves and Who It Is For

Apache Fluss addresses the gap between data streaming and data lakehouse architectures. Traditional pipelines often buffer streaming data in Kafka or similar systems, then batch-load it into a data lake, which introduces latency. Fluss instead provides a streaming storage that can serve as the real-time data layer for a lakehouse, enabling continuous ingestion and immediate data availability. This is aimed at teams that need sub-second data freshness for real-time analytics and decision-making, while still wanting to unify real-time and historical data under a single table abstraction. The README explicitly mentions integration with Apache Flink and Apache Spark, with StarRocks coming soon, so the target user is someone already invested in those engines and looking to reduce the complexity of managing separate streaming and batch systems.

How It Works: Columnar Streaming and Merge Engines

The core mechanism is built around Apache Arrow, which allows columnar streaming. This means data streams can support database primitives like column pruning and predicate pushdown, so compute engines only read the necessary columns, reducing I/O and network costs. Fluss also implements compute-storage separation: stream processors focus on computation, while Fluss manages state and storage. A distinctive feature is the set of merge engines: deduplication, partial updates, delta joins, and aggregation. These are not just storage options; they change how streaming data is processed and stored. For example, partial updates allow you to update specific fields in a record without rewriting the entire row, which is useful for slowly changing dimensions. The system also generates changelogs, providing an append-only history of state changes, which supports auditing and reproducibility. This is a design that treats storage as an active participant in stream processing, not a passive sink.

Getting It Running: Build and Setup

The README shows that Fluss is a Java project built with Maven. Prerequisites are a Unix-like environment (Linux, macOS, Cygwin, WSL), Git, Maven version 3.8.6 or higher, and Java 11. The build command is straightforward: clone the repository, then run `./mvnw clean package -DskipTests`. The Maven Wrapper ensures the correct Maven version is used, so you do not need to manually install Maven. After the build, the distribution is installed in a directory called `build-target`. There is no mention of a pre-built binary release or a Docker image in the README, so the only documented path is building from source. This is a barrier for quick evaluation, but the process is simple and reproducible. Once built, you would need to consult the documentation for deployment details, which are not covered in the README.

Limitations and When It Is the Wrong Tool

The most obvious limitation is the lack of production readiness evidence. The project is in incubating status, with version numbers like 0.9.1-incubating, which signals that the API and behavior may change between releases. The README does not provide operational guidance, such as how to configure replication, handle failover, or manage cluster scaling. That means you cannot use it for a critical production workload without significant additional research. Also, the integration list is short: only Flink and Spark are explicitly mentioned, with StarRocks coming soon. If your compute engine is not supported, Fluss is not useful. Another limitation is that the merge engines, while powerful, may not cover all use cases. For example, if you need complex event processing or stateful windowing, Fluss is not designed for that; it is a storage layer, not a stream processor. Finally, the requirement for Java 11 and a Unix-like environment may exclude some Windows-only shops, though WSL is an option.

Alternative Approaches and How They Differ

The most common alternative is to use a message queue like Apache Kafka for streaming ingestion, then a separate batch layer to load data into a data lake like Apache Iceberg or Delta Lake. This approach separates concerns but introduces latency and requires maintaining two systems. Fluss tries to unify these by providing streaming storage with lakehouse integration, so you do not need a separate queue. Another alternative is to use a database like Apache Druid or ClickHouse for real-time analytics, which can ingest streaming data and serve queries directly. However, those systems are not designed to be a data layer for a lakehouse; they are standalone analytics databases. Fluss's approach is different because it uses a table abstraction that spans real-time and historical data, and it integrates with external compute engines rather than providing its own query engine. The trade-off is that Fluss is less mature and has a smaller ecosystem compared to Kafka or Druid.

Maintenance and Upgrade Cost

The release history shows a rapid cadence: 0.8.0 in November 2025, 0.9.0 in March 2026, and 0.9.1 in May 2026. This suggests active development, but it also means you should expect frequent upgrades with potential breaking changes, especially given the incubating status. The README does not provide a migration guide or a compatibility matrix, so you will need to read release notes and possibly adapt your code and configuration with each upgrade. The project is licensed under Apache-2.0, which is permissive and allows commercial use, but you should be aware that the 'incubating' label means the project has not yet fully established its governance and long-term stability. The build process is simple, but upgrading may require rebuilding from source, as there are no binary artifacts mentioned. This is a real cost for teams that prefer stable, long-term support.

What Is Not Covered and What to Verify First

The README is sparse on operational details. It does not describe how to deploy Fluss in a cluster, how to configure storage or replication, or how to monitor it. It also does not provide examples of using the merge engines or the changelog feature. If you are evaluating Fluss, you should first verify that the documentation at fluss.apache.org covers these topics. Second, check whether the current version supports your specific compute engine and whether the merge engine semantics match your data processing needs. Third, test the build in your environment, as the README only guarantees Unix-like systems with Java 11. Finally, look at the changelog for the 0.9.x releases to understand what changed since 0.8.0, and whether any of those changes affect your use case. Without this verification, adopting Fluss would be premature.

Editorial conclusion

Adopt Apache Fluss if you need low-latency data ingestion that lands directly into a lakehouse, especially if you already use Apache Flink or Spark and want a single table abstraction for real-time and historical data. Do not adopt it if you require a mature, production-hardened system with extensive documentation, or if your workloads are purely batch and do not need sub-second freshness. Before committing, verify that the current version supports your target compute engine (Flink and Spark are mentioned, StarRocks is coming) and that the merge engines (deduplication, partial updates, delta joins, aggregation) meet your exact requirements. Also confirm that your Java 11 environment and Maven setup can handle the build, and review the changelog for any breaking changes in the 0.9.x series.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes