Netflix Maestro: a workflow orchestrator you run yourself

What Maestro is, and the workload it was built for

Maestro is a general-purpose workflow orchestrator that the README describes as a fully managed workflow-as-a-service for data platform users at Netflix. The distinction matters. This is not a library you import into an application; it is a service with its own HTTP API, its own persistence layer and its own scheduling components. The repository is split into modules that reflect that: maestro-server, maestro-engine, maestro-dsl, maestro-database, maestro-queue, maestro-timetrigger, maestro-signal, maestro-flow, maestro-http, maestro-kubernetes and maestro-aws.

The stated audience is broad: data scientists, data engineers, machine learning engineers, software engineers, content producers and business analysts. Those are different jobs with different tolerances for YAML, and the README does not explain how the same workflow model serves all of them. What it does say is that the system schedules hundreds of thousands of workflows and millions of jobs per day at Netflix and operates with a strict SLO during traffic spikes. That is a description of the deployment Netflix runs, not a guarantee about a local single-node install.

If you are looking for a scheduler to run three nightly jobs on one box, the module list alone should tell you this is heavier than the problem. The interesting case is a team that needs a workflow engine it can host, extend and expose over HTTP, and is willing to run a Java service to get it.

How the pieces fit: server, engine, queue and extensions

The architecture visible in the repository layout is a set of cooperating modules rather than a monolith. maestro-server exposes the REST API on port 8080 and is the entry point for creating workflows and triggering runs. maestro-engine holds the execution logic. maestro-dsl handles workflow definition. maestro-database and maestro-queue are the persistence and messaging layers. maestro-timetrigger and maestro-signal cover scheduled and event-driven starts.

The extensions service is the most concrete example of the design. According to the README, maestro-extensions runs as a separate Spring Boot service that listens to Maestro events via SQS, subscribed to the SNS topic that maestro-server publishes to, and provides additional functionality such as foreach step flattening views. It runs on port 8081. That is a real architectural commitment: if you want the extension behaviour, you need an event path between the two services, and the README's local recipe for that is LocalStack standing in for SQS and SNS.

This split is a trade-off worth naming. It keeps the core server smaller and lets extension logic evolve separately, but it also means a local development environment has more moving parts than a single process. The README does not describe what happens to extension views when the event stream is interrupted, so durability of that path is something you would have to establish yourself.

Installing Maestro and running your first workflow

The README lists four prerequisites: Git, Java 21, Gradle and Docker. Note the version. Java 21 is specified explicitly, so a machine on an older JDK will not match the documented setup. The build is a single Gradle command from the repository root:

./gradlew build

Running the server locally is the next command. The README gives it without arguments for the default profile:

./gradlew bootRun

With that process up, the server listens on port 8080 and you can create a workflow. The README uses curl against the v3 API and a sample DAG that ships in the test resources, with a user header identifying the caller:

curl --header "user: tester" -X POST 'http://127.0.0.1:8080/api/v3/workflows' -H "Content-Type: application/json" -d @maestro-server/src/test/resources/samples/sample-dag-test-1.json

Creating a workflow does not run it. The README separates definition from execution: you fetch the latest version, then post to an actions/start endpoint with an initiator object marking the run as manual.

curl --header "user: tester" -X POST 'http://127.0.0.1:8080/api/v3/workflows/sample-dag-test-1/versions/latest/actions/start' -H "Content-Type: application/json" -d '{"initiator": {"type": "manual"}}'

After that, the instance and run can be read back with a GET against the workflow's instances path. The README also documents the reverse operation, a DELETE on the workflow that removes the workflow and its data, which is useful when you are repeating this loop locally.

If you prefer to author in Python, the README points at a separate client package, installed with pip:

pip install maestro-sdk

The README's example builds a workflow object, sets an owner and tags, adds a NoOp job, and serialises it to YAML. A MaestroClient constructed with a base URL and a user name then pushes that YAML and can start the workflow with run parameters. The README links the maestro-python project for further detail, which is where you would look for anything the short example omits.

The AWS and Kubernetes paths are not decorative

Two optional integrations change what you have to run. The AWS path starts a compose file for dependencies and then boots the server with the aws Spring profile active:

docker compose -f maestro-aws/docker-compose.yml up
./gradlew bootRun --args='--spring.profiles.active=aws'

The README's combined recipe for server plus extensions is more specific: bring up LocalStack in detached mode, start maestro-server on 8080 with the aws profile, then start maestro-extensions on 8081. The extensions service consumes step instance status change events from the maestro-event SQS queue. That queue name and the profile flag are the two things to get right; the README does not document an alternative event transport for local work.

The Kubernetes path is thinner. The README says to set up Kubernetes configs so the kubectl command works, run bootRun, then create and start a separate sample workflow file, sample-kubernetes-wf.json. There is no manifest, no operator and no Helm chart described in the README, so the integration appears to be about jobs that talk to a cluster rather than about deploying Maestro itself to one. Treat that as a gap if cluster-native deployment is what you need.

Where Maestro is the wrong choice

The README is a getting-started document, and it is silent on several things an operator would ask about. There is no section on high availability, no description of how the database is backed up or migrated, and no documented rollback procedure for a workflow definition that has already been versioned. The versioning endpoints imply definitions are immutable once published, but the README does not state what happens to in-flight runs when a new version is pushed.

The operational surface is also real. Running the full local topology means a Java service, a second Spring Boot service for extensions, and LocalStack or actual AWS services for the event path. That is a lot of machinery for a team whose actual problem is "run these five steps in order every night." A shell script with a cron entry has fewer failure modes, and you already know how to debug it.

There is a subtler mismatch. The README positions Maestro as serving thousands of users across many roles at Netflix. A self-hosted install inherits the API and the module structure but none of the operational support that makes that scale work. The README does not claim otherwise, but the gap between the blog-post framing and the repository contents is wide enough to be worth saying plainly. If you need someone to call when the scheduler is down, this is not that product.

How it compares with Airflow and Argo Workflows

The obvious comparison is Apache Airflow, and the difference is in the deployment model rather than the feature list. Airflow is a Python-first system where DAGs are Python files and the scheduler, webserver and workers are separate processes you assemble. Maestro's workflow definitions are data pushed to an HTTP API, authored either as JSON or YAML or through the Python SDK, and the Java service owns execution. If your team writes Python and wants DAG logic expressed as code with the full Python ecosystem available inside the DAG file, Airflow's model is a closer fit. If you want workflow definitions to be artifacts you POST and version through an API, Maestro's model is the more natural one.

Argo Workflows takes a third position: workflows are Kubernetes custom resources and every step runs as a pod. That gives you container isolation and cluster scheduling for free, at the cost of requiring a cluster. Maestro's Kubernetes support, as documented, is about workflows that interact with a cluster rather than about running on one. The README's local instructions assume you run the server yourself on a host with Java and Gradle.

The honest summary is that the three systems make different bets about where workflow definitions live and what executes a step. Maestro's bet is a managed HTTP service with a pluggable execution layer, which is why the module list includes separate queue, signal, timetrigger and Kubernetes modules rather than one execution path.

Licence, maintenance and what upgrading costs you

Maestro is licensed under Apache-2.0, with the copyright notice attributing it to Netflix, Inc. and the standard disclaimer that the software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND. For most teams that is a permissive licence with few obligations beyond preserving notices, but the disclaimer is doing real work here: the README documents a local development setup, not a supported production deployment. Whether the Apache-2.0 patent grant and notice requirements fit your distribution model is a question for your own legal review, not something the README answers.

The repository is not archived, and the last push was on 2026-09-03. That is recent enough that the codebase is moving, but the README does not publish a release cadence, a versioning policy or a compatibility guarantee for the v3 API. There is also no retrieved release list to check against. Practically, that means upgrading is a source-level exercise: you rebuild from the branch you track and read the diff. The gradle.lockfile and dependencies.gradle at the repository root suggest dependency versions are pinned deliberately, which helps reproducibility but also means a dependency bump is a real change rather than an incidental one. Budget for that. If your team cannot absorb a rebuild-and-retest cycle on its own schedule, the lack of a documented release process is the constraint that matters most.

Editorial conclusion

Maestro fits teams that already run JVM services and want a workflow engine they can host themselves, with a REST API, a YAML workflow model and a Python client for authoring. Teams without Java or Docker on the build machine, or anyone expecting a hosted product with a support contract, should look elsewhere first. Verify three things before committing: that Java 21 and Gradle are available in your build environment, that the AWS path is acceptable if you want the extensions service, and that the sample workflow at maestro-server/src/test/resources/samples/sample-dag-test-1.json covers the step types your own DAGs need.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Netflix/maestro on GitHub
4. Project website
5. README