Apache StreamPark: a control plane sitting on top of Flink and Spark

What the project considers its job

The README describes StreamPark twice, and both descriptions matter. It is a streaming application development framework, and it is a cloud-native real-time computing platform. The first half deals with code: prebuilt APIs, connectors and templates that simplify Flink and Spark streaming development. The second half deals with everything after the code exists, which is where most streaming projects actually lose their time.

The project was originally called StreamX and renamed to StreamPark in August 2022, then became an Apache Top-Level Project in January 2025. That history explains the Java language tag and the Maven-heavy tree, and it explains why the topic list runs to eight entries spanning cloud-native, control-plane, data-computing, flink, kubernetes, lakehouse, spark and streaming.

The repository description is one line: The Open Control Plane for Real-Time Data. That is a useful filter. If your problem is writing a Flink job, StreamPark adds a layer rather than removing one. If your problem is that six Flink jobs across two clusters need submitting, monitoring, upgrading and version-tracking, that layer is the product.

One service, five minutes to a console

The quick start is one command, and it is the clearest statement of the design:

docker run -d -p 10000:10000 apache/streampark:latest

Port 10000 is the console. There is also a local installation path that downloads and runs a script:

curl -L https://streampark.apache.org/quickstart.sh | sh

The README claims single-service deployment and going from zero to running jobs in minutes, and both of those are plausible given the design. The batching of concerns is what makes it possible: application definition, submission to the engine, monitoring of running state and version tracking all live behind one interface instead of being spread across a submission CLI and a separate dashboard.

Building from source is `./build.sh`, with a `mvnw` wrapper and `mvnw.cmd` checked in alongside `pom.xml`. Development documentation is linked separately at the docs site rather than in the README, which tells you where the real content lives.

The module layout and where the code actually is

The top level of the tree is a Maven reactor with clearly named modules, and the names are a good map of the project's responsibilities. `streampark-flink` is the Flink integration, `streampark-spark` the Spark one, `streampark-console` the web interface and server, `streampark-common` shared code, `streampark-scala` the Scala side, and `streampark-shaded` relocated dependencies for a clean artifact.

Supporting infrastructure sits alongside them: `dist-material/` for packaging, `docker/` for images, and `tools/` for build and release tooling. The repository also carries the markers of a properly run Apache project: `.asf.yaml`, `NOTICE`, `.licenserc.yaml` for license header checking, `.sonarcloud.properties`, `qodana.yaml` for JetBrains static analysis, and `.dlc.json` for the distribution license check.

`AGENTS.md` and `.git-blame-ignore-revs` are worth noting for anyone planning a contribution. The first suggests agent-assisted tooling is expected in the repo, and the second is the mechanism for ignoring reformatting commits in blame, which tells you the project has run automated formatters across its history.

Multi-engine and multi-environment, which is the hard part

The feature list calls out two capabilities that are related but distinct. Multi-engine and multi-version support means running and managing applications written for different Flink or Spark versions, which in practice means the control plane has to speak several engine APIs and several deployment descriptors. Multi-environment compatibility means the same control plane works on standalone clusters, YARN on Hadoop 2.x and 3.x, and Kubernetes.

Those two together are the reason to use something like this at all. Flink's own cluster and job APIs are usable, but they are per-engine and per-version, and Kubernetes adds a third vocabulary on top. A control plane that hides that variation is worth building, and it is also where the bugs live.

The ecosystem compatibility claim names Flink, Spark, Paimon and Doris, along with machine learning and AI ecosystems. Naming Paimon specifically is a signal: that project is a lakehouse format over Flink, so compatibility with it is close to a requirement rather than a bonus for anyone building streaming ingestion into a lakehouse table.

Release cadence and project status

Three releases appear on the repository. v2.1.7 was published on 2025-11-05, its release candidate v2.1.7-rc1 on 2025-10-28, and v2.1.6 on 2025-07-09. Both final releases point to release notes on the project site rather than listing changes in the tag body, so the substantive changelog lives outside GitHub.

The release candidate body is more informative than the finals, and it reads like a maintenance pass: improved license headers in the Maven utilities, stricter validation of Maven arguments during builds, a refined login authentication process, and a fix for retrieving Flink configuration files. That last item is a good clue about where the fiddly problems are, since configuration retrieval is the kind of thing that differs between Flink versions.

The repository is not archived, and the last push was on 2026-09-15, so the 2.1.x line is still receiving work. With 4,337 stars, 1,087 forks and 53 open issues, it is an active Apache project with a real user base. The default branch is `dev`, which is worth knowing before you clone: `main` is not where development happens.

Community channels and where documentation lives

An Apache project has a predictable contribution surface, and StreamPark fills it in. The pull requests and issues pages are the entry points, and the README asks new reporters to read two well-known guides on asking good questions and reporting bugs before posting. Contribution documentation lives on the project site, split into a general submit guide and a code submission guide with its own style and quality rules.

Communication runs through the `[email protected]` mailing list, with subscribe and unsubscribe addresses and an archive. Social channels include an X account under the Apache StreamPark name, plus Zhihu, bilibili and a WeChat official account, all three of which are Chinese-language. The presence of a Zhihu account and a related search for Dinky, another Apache Flink tool, suggests a substantial Chinese-speaking user community around the Flink ecosystem in general.

For documentation, the README links the official docs at the get-started page and the download page for releases. The README itself stops at the project description, the two quick start commands, the build command and the contact details, which means anything beyond an introduction is a docs-site read.

Editorial conclusion

StreamPark earns its place when you are running several Flink or Spark streaming applications and the operational work has outgrown scripts. The README's own pitch, single-service deployment and running jobs in minutes, is the accurate summary: one process that owns development, deployment, monitoring and operations, rather than four tools assembled from templates and cron. What it does not do is replace Flink or Spark, which is the point most readers get wrong on first contact. The honest limit is that the repository README is thin and the substance lives on streampark.apache.org, where the quick start, the development guide and the per-version release notes all sit. The module layout in the repository, with separate `streampark-flink`, `streampark-spark`, `streampark-console` and `streampark-scala` trees, tells you where to look in the code. Start with the Docker image on port 10000 to see the console, then read the docs page for the deployment mode that matches your cluster, since standalone, YARN and Kubernetes differ in more than a flag.

Frequently asked questions

Official sources

1. apache/streampark on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases