Apache StreamPipes and the industrial data pipeline you can assemble by hand

Two audiences in one platform: domain experts and developers

Apache StreamPipes is an end-to-end industrial IoT data platform, and the README splits its story into two audiences. For domain experts there is a graphical user interface: a guided setup flow for connecting data sources, a visual pipeline editor, charts and dashboards built for time-series data, and an asset-centric way to organise pipelines, streams and dashboards around the machines they belong to.

For developers there is an extensible framework. Custom adapters, processors and sinks can be written against the Java SDK, pipeline elements can run as standalone microservices in central or edge deployments, existing processing logic and machine learning models can be folded in as reusable pipeline elements, and client libraries for Python, Go and Java let an application talk to the platform directly.

That split is the design. Most users never see the developer side, and most developers never use the GUI. The platform is meant to absorb industrial data once, at the edge if necessary, and then hand it to analytics, monitoring and application layers without every team writing its own ingestion code.

NATS messaging and a time-series database in the standard install

A standard installation ships the whole stack. The README describes a messaging system for real-time data exchange, with NATS as the default and support for additional protocols, plus a time-series database for storing historical data. The compose file in the repository root confirms the shape of that stack: the backend service depends on couchdb, and the environment sets SP_PRIORITIZED_PROTOCOL to nats.

The docker-compose.yml at the repository root carries its own warning. A notice at the top states that this file is for validating releases only, and that anyone wanting to actually use StreamPipes should follow the installation instructions on the download page. That is worth reading before cloning anything, because the compose file in the tree is not the supported install path.

The same file also shows the platform treats its own configuration seriously. Secrets are injected through environment variables that fail loudly when unset, the backend persists to a named volume, and Prometheus and Grafana have their own directory in the tree. The reset endpoint is enabled through a configuration flag rather than hardcoded.

OPC UA support and the connectors behind it

Connectors are where an industrial platform wins or loses, and the README leans on OPC UA as the flagship case, describing extensive support including a web-based node browser and support for OPC UA events. A node browser matters more than it sounds: browsing the address space of a machine from inside a web UI is the step that usually requires a separate engineering tool.

The broader list of sources named in the README covers OPC UA, PLCs, MQTT, REST, Pulsar and Kafka. That is a spread from device protocols to message brokers to plain HTTP, and it is the reason the project positions itself as a connector platform rather than a single-protocol tool.

The repository layout shows how much surface that implies. There are separate modules for the data explorer, the extensions registry, connect management, the audit API, the audit events and the audit influx pieces, plus dedicated modules for the Python and Go clients. StreamPipes itself is the larger part of the tree, with separate service cores and a data format module that the runtime and the client libraries share.

Getting a running install: compose first, CLI deprecated

The Quick Start section lists three Docker-based installers: StreamPipes Compose for container-based deployment, StreamPipes CLI for developers who want to extend StreamPipes, and StreamPipes k8s for cluster-based operation. The CLI entry is explicitly marked deprecated, which tells you the project has settled on containers as the normal route and moved away from the local development installer.

For first-time users the README points at the compose directory, and the first command is simply to move into it:

cd installer/compose

The compose bundle is described as the fastest way to get a full installation with extensions. The k8s installer is the option for anyone who needs to run the platform across a cluster, and the repository includes a prometheus-grafana directory, which suggests monitoring is part of the intended deployment shape rather than an afterthought.

The build itself is Maven based, with a pom.xml at the root and a .mvn directory, and the Java version badge points at Java 25. The README does not give a step-by-step local build command for the backend, so anyone wanting to run from source beyond the installer directories has to work that out from the build files.

Where the platform is opinionated, and where that costs you

The README uses the phrase opinionated architecture for how StreamPipes manages industrial IoT data, and that word is accurate in both directions. It is good news for a team that wants a working stack without choosing a broker, a database and an ingestion protocol: NATS, the time-series store and the connector set arrive as a package.

It is bad news for a team with existing infrastructure, or one that wants to embed a single connector rather than adopt a platform. Replacing the messaging layer or the time-series store is not something the README describes as supported, and there is no documented way to run StreamPipes without a full deployment around it. For a developer who wanted only the OPC UA connector and nothing else, this is the wrong size of tool.

The governance and packaging also come with an opinion. The platform is hosted by the Apache Software Foundation, and the tree contains AGENTS.md, CLAUDE.md, DESIGN.md, RELEASE_NOTES.md, RELEASE_VALIDATION.md and a Jenkinsfile, which points to a large release-validation apparatus. That is a good sign for a project you intend to run in production for years, and a slow one to adopt if you need a patch today.

Security features and the operational files that come with them

A platform that ingests live plant data is a security project whether or not it says so. The README lists enterprise-grade capabilities that most research prototypes skip: user and role management, OAuth 2.0 integration, and support for geographically distributed deployments that simplify communication between OT and IT networks. That last item is the one plant operators ask about, because the OT and IT networks in a factory are usually separated on purpose, and a platform that has to straddle that line needs a story about how it does it.

The repository carries the files that back this up. There is a SECURITY.md, a THREAT_MODEL.md and a NOTICE file, and the audit side of the tree is split into streampipes-audit-api, streampipes-audit-events and streampipes-audit-influx, with health monitoring in its own module. An audit API with its own event and storage modules suggests who-did-what tracking is built in rather than bolted on.

The Apache-2.0 licence and the ASF governance remove most commercial questions for teams that worry about vendor lock-in. They do not remove the operational ones: a platform with role management and audit trails needs someone on your side to administer it.

Release cadence: 0.98.0 in December 2025 and work since

The repository is not archived and the last push was on 2026-10-22, so work on the default dev branch is current. The most recent tagged release is 0.98.0 from 2025-12-15, preceded by 0.97.0 in February 2025 and 0.95.1 in July 2024. The version numbering has not reached 1.0, and the gap between the last release and the last commit means the dev branch moves ahead of what you would download.

For an adopter this shapes the upgrade path. Docker images are published for the backend, so a deployment can pin a released version, but the compose file in the tree points at a locally built backend image, streampipes_backend:release-validation, which is a validation setup rather than a production pin.

The practical takeaway is to decide early whether you track releases or the dev branch. Release tracking gives you reproducible images and a slower move. Tracking dev gives you current fixes and a moving target for your own integration tests.

Editorial conclusion

Apache StreamPipes fits teams that already own industrial equipment and want to analyse it without building a bespoke streaming stack, and it suits developers who want to extend a real platform through REST APIs and the Java SDK. It does not suit a team that wants a lightweight library, or one that cannot run containers or cannot accept an opinionated architecture with NATS and a time-series database baked in. Before adopting, check that a connector exists for your equipment protocol, confirm the installer you plan to use is still supported, and read the security notes in SECURITY.md and THREAT_MODEL.md, since the platform holds live plant data and carries user, role and OAuth 2.0 machinery by default.

Frequently asked questions

Official sources

1. apache/streampipes on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases