Data on EKS: opinionated blueprints for data platforms on Kubernetes

What the blueprints are for

DoEKS, pronounced Do.eks, is a tool to build, deploy and scale data platforms on Amazon EKS, and it is presented as optimized blueprints for running scalable data workloads on Kubernetes with Amazon EKS. The problem it names is specific. Kubernetes is a widely adopted system for orchestrating containerized software at scale, and as more users migrate their data platforms to Kubernetes they face the complexity of managing the ecosystem and selecting the right tools and configurations for their needs. At AWS, the authors say they understand those challenges, and they developed DoEKS to simplify the process and enable users to conduct proof-of-concepts and build production-ready clusters. So the deliverable is opinionated open-source blueprints that provide end-to-end logging and observability, making it easier to deploy and manage Spark on EKS, Airflow, Presto, Kafka and other data workloads.

Four focus areas and the components in each

The solution is categorised into four focus areas, and the categories are more useful than the individual tools because they tell you what problem each blueprint solves. Data Processing covers Spark, EMR and Ray. Streaming Platforms covers Kafka and Flink. Orchestration covers Airflow and Argo Workflows. Databases and Query Engines covers Trino, Pinot and ClickHouse. Each focus area has its own documentation page under the website, and each links to the specific blueprints. The architecture diagram in the README shows the whole picture, with the open-source data tools, the Kubernetes operators and the frameworks used by DoEKS alongside the AWS Data Analytics managed services, which is what the integration claim means in practice: the blueprints are not purely open source, they mix in the managed service where that is the better answer.

The blueprints included in the repository

The repository holds a selection, and the README is explicit that these examples are just a small selection of the available blueprints, with the website carrying the complete list. What is in the repository covers the main categories. EMR on EKS runs EMR Spark workloads on EKS with cost-effective autoscaling. Spark Operator with YuniKorn is self-managed Spark with multi-tenant scheduling. Apache Flink on EKS gives self-managed Flink clusters on EKS. Apache Kafka with Strimzi provides high-throughput Kafka messaging on EKS. Airflow on EKS is DAG-based data pipeline orchestration using Apache Airflow, and Argo Workflows is a Kubernetes-native workflow engine for CI/CD or data pipelines. The two Spark-related entries are the interesting contrast: one is the managed service and one is self-managed, which is the decision most people reaching for this repository are actually making.

Where the code actually lives

The repository layout matches the shape of the project. There is a data-stacks directory, which holds the blueprints themselves, an infra directory for the infrastructure side, and a website directory containing the documentation site with its static assets. The repository also carries the housekeeping that a project with many contributors needs: an ADOPTERS.md, a CODEOWNERS file, a CODE_OF_CONDUCT.md, a CONTRIBUTING.md, an AGENTS.md, a .pre-commit-config.yaml, a .codespellrc and a NOTICE file next to the LICENSE. That last pair is the one to know about, because a project distributing AWS blueprints will include third-party attributions and the NOTICE file records them. The tfsec.yaml is a separate signal, since it configures a static analysis scanner for the Terraform that the infrastructure directory contains.

Managed services are part of the blueprint, and that is the point

The README's architecture section is where the design decision becomes visible: the diagram shows the open-source data tools and frameworks used by DoEKS and also highlights the integration of AWS Data Analytics managed services with the capabilities of the open-source tools. EMR on EKS is the clearest case, because it is a managed service that runs on the cluster, and the blueprint positions it as running EMR Spark workloads with cost-effective autoscaling rather than as a way to avoid EMR. The same pattern appears in the other entries, where Strimzi is the operator for Kafka, YuniKorn the scheduler for Spark, Argo Workflows the workflow engine for pipelines. The choice in each blueprint is between running the component yourself and running the managed version, and the blueprint takes a position rather than leaving it open.

This is a community project, not an AWS service

The support section is unusually direct about what you are getting. DoEKS is maintained by AWS Solution Architects and is not an AWS service. Support is provided on a best effort basis by the Data on EKS Blueprints community. If you have feedback, feature ideas, or wish to report bugs, you are asked to use the Issues section of the GitHub repository. That boundary matters for anyone planning a production data platform on it, because an unsupported configuration is your problem to solve, and the roadmap is driven by the issue tracker rather than by a service team. The repository also notes that DoEKS is in active development and points to the issues section for upcoming features and enhancements, so the shape of the blueprints can change under you.

Where DoEKS is the wrong choice

Four limits apply. The blueprints are opinionated, which is their value and their constraint: the autoscaling, the operator choices and the observability wiring are pre-decided, and a team with strong existing opinions about, say, their Kafka topic strategy or their Spark scheduler configuration will spend time fighting the blueprint or forking it. The cost model is the second limit, because a blueprint that wires EMR on EKS, Kafka, Airflow and a query engine onto one cluster is affordable for a proof of concept and a different proposition at production scale, and the Terraform in the repository makes those choices explicit rather than hiding them. Third, the tools move: the blueprint for a component is coupled to a version of that component and its operator, so upstream releases are the upgrade path and the repository's release cadence is not. And fourth, this is an AWS-specific project, so the Kubernetes knowledge in the blueprints is EKS knowledge, and portability to another cloud means rewriting rather than reconfiguring.

Editorial conclusion

Use DoEKS when you are standing up a data platform on EKS and want a working configuration faster than you want to assemble one yourself, since the blueprints pair the operators, the autoscaling and the observability rather than shipping the pieces separately. Do not expect it to be an AWS service or a supported product, because the project says it is maintained by AWS solution architects on a best effort basis, and you will be running the open-source components yourself. Verify first which blueprint matches your workload and your tenancy constraints, since the list on the website is longer than the set in the repository, that the AWS account and region assumptions in the Terraform hold for you, and that the cost of the managed services in the blueprint is understood before a proof of concept turns into a bill.

Frequently asked questions

Official sources

1. awslabs/data-on-eks on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases