Netflix Maestro: a Java workflow orchestrator you can boot locally in four commands

The problem Maestro solves, and the users it was built for

The README describes Maestro as a general-purpose workflow orchestrator that provides a fully managed workflow-as-a-service to data platform users at Netflix. The audience named there is broad: data scientists, data engineers, machine learning engineers, software engineers, content producers, and business analysts. That list is the tell. Maestro is not a scheduler for one team's nightly ETL; it is a shared service that has to accept workflow definitions from people who do not write Java.

The problem it addresses is the gap between a cron job and a platform. A cron job runs a command. A workflow orchestrator has to track a definition, version it, accept a trigger with parameters, persist an instance, and expose the state of each run so a human or another system can act on it. Maestro's REST surface is built around exactly those nouns: workflows, versions, actions, instances, runs. If your current setup is a crontab plus a Slack channel where people ask whether last night's job finished, that is the gap.

The README also states that Maestro schedules hundreds of thousands of workflows and millions of jobs every day with a strict SLO. Treat that as context for why the project is shaped the way it is, not as a performance claim you can transfer to your own deployment. Nothing in the supplied material gives throughput numbers for a self-hosted install.

What the repository layout tells you about the architecture

The build is a multi-module Gradle project. The README references three modules by name: maestro-server, maestro-aws, and maestro-extensions. The server is the Spring Boot application that serves the API on port 8080. The AWS module is a Docker Compose file that stands up local SQS and SNS through LocalStack. The extensions module is a second Spring Boot service on port 8081.

That split is the most informative design decision in the material. Maestro does not do everything in one process. The server publishes events to an SNS topic, and maestro-extensions subscribes to a queue called maestro-event to consume step instance status change events. The README gives foreach step flattening as the example of what the extension service provides, and says you query the flattened views through the extensions REST API. So the core engine and the additional features are decoupled through a message bus rather than through shared tables or in-process calls.

Two consequences follow. First, the extension service is optional: you can run the server alone and lose the flattening views. Second, if you run the server without the AWS profile, there is no LocalStack SNS or SQS, so the event path that the extensions module depends on is not present locally. The README does not describe what replaces it in that mode, which is a gap worth noting before you plan a non-AWS deployment.

Getting a workflow from curl to a running instance

The prerequisites are Git, Java 21, Gradle, and Docker. Build with `./gradlew build` and run with `./gradlew bootRun`. That is the whole local path for the core server.

The README's sample flow uses a file that ships in the repository, maestro-server/src/test/resources/samples/sample-dag-test-1.json. You create the workflow with a POST to `http://127.0.0.1:8080/api/v3/workflows`, passing a `user: tester` header and the JSON body. You read it back with a GET on `/api/v3/workflows/sample-dag-test-1/versions/latest`. You start it with a POST to that same path plus `/actions/start`, with a body containing `{"initiator": {"type": "manual"}}`. You check the result at `/api/v3/workflows/sample-dag-test-1/instances/1/runs/1`. You clean up with a DELETE on the workflow path.

Notice the identity model. There is no token or session in these examples. The user is a plain `user:` header. For a local sample that is fine. For anything shared, the README does not describe authentication or authorization, and you should not assume the v3 API ships with either.

The AWS path adds two commands: `docker compose -f maestro-aws/docker-compose.yml up` and then `./gradlew bootRun --args='--spring.profiles.active=aws'`. The Kubernetes path is different again: set up kubectl so the command works, run the server normally, then POST the sample-kubernetes-wf.json file and start it the same way.

The Python SDK is a separate project with its own release cycle

The README documents a Python client installed with `pip install maestro-sdk`. The usage examples construct a Workflow object, chain `.owner("tester").tags("test")`, attach a `Job(id="job1", type='NoOp')`, and call `to_yaml()`. Pushing uses `MaestroClient(base_url="http://127.0.0.1:8080", user="tester")` and `client.push_yaml(wf_yaml)`. Starting uses `client.start(workflow_id="test-wf", run_params={"foo": {"value": "bar", "type": "STRING"}})`.

The detail that matters is at the end of that section: the README points to a GitHub project called maestro-python for more details. That means the SDK lives outside this repository. Its versioning, its issue tracker, and its compatibility with a given server build are not governed by the Apache-2.0 licence covering the Java code. If your plan is to have data scientists author workflows in Python while your platform team runs the server, you are depending on two repositories with two maintenance schedules, and the README does not state a compatibility matrix between them. Verify that pairing yourself before you build tooling on top of it.

The YAML round trip is also worth understanding. The Python objects serialize to YAML, and the server accepts that YAML through the push call. The curl examples, by contrast, post JSON directly. Both paths reach the same v3 API, but they are not the same authoring experience, and the material does not say whether every field expressible in the JSON sample is expressible through the SDK builders.

Where Maestro is the wrong tool

The README lists no releases. The repository metadata shows no recent releases retrieved, and the README has no changelog, no version compatibility table, and no upgrade notes. For a project you intend to run for years, that absence is the single largest operational risk in the material. You cannot plan an upgrade path from what is here.

Second, the deployment story is thin outside the happy path. The AWS profile depends on LocalStack for local development, which is a testing convenience rather than a production topology. The README does not describe how the SNS topic and SQS queue are provisioned in a real account, what IAM permissions the server needs, or how the extensions service is scaled. If you are not already fluent in SNS and SQS, the aws profile will be a wall rather than a shortcut.

Third, the Java 21 requirement is not negotiable in the material. If your platform standardizes on Java 17, you are changing a runtime baseline to adopt this. That is a real cost, not a footnote.

Fourth, and most importantly, this is a workflow engine, not a data movement tool. The README's incremental processing blog post pairs Maestro with Apache Iceberg, which suggests the intended pattern is that Maestro schedules and something else does the reading and writing. If what you actually need is a connector library that moves rows between systems on a schedule, Maestro gives you the schedule and nothing else.

How this differs from Airflow's approach

Apache Airflow is the obvious comparison, and the difference is architectural rather than cosmetic. Airflow's core abstraction is the DAG defined in Python, with the scheduler, the webserver, and the workers as separate long-running processes and a metadata database underneath. Maestro's core abstraction is a workflow definition submitted over HTTP to a Spring Boot service, with optional extension services consuming events off a message queue.

That means the extension point differs. In Airflow you extend by writing Python that runs inside the scheduler's process model. In Maestro you extend by subscribing to SNS events and running a separate service, which is what maestro-extensions does for foreach flattening. The Maestro approach keeps the core engine smaller and lets you add features without touching it, at the cost of a message bus you now have to operate and a second service to deploy.

The authoring story differs too. Airflow users write Python DAGs. Maestro users submit YAML or JSON to a REST API, and the Python SDK is a client-side builder that produces YAML rather than a runtime that executes your code. If your team's mental model is "my workflow is a Python file," Maestro will feel indirect. If your mental model is "my workflow is a resource I create through an API," it will feel natural.

Licence, maintenance, and what you are actually taking on

Maestro is Apache-2.0, copyright 2024 Netflix, Inc. The licence text in the README is the standard Apache 2.0 grant: you may use the software, and it is provided on an as-is basis without warranties or conditions of any kind. Apache-2.0 includes a patent grant and requires attribution and notice retention. It does not require you to open your own modifications. This is a permissive licence, and nothing in the supplied material suggests any additional restriction.

The practical question is not the licence but the maintenance surface. You are taking on a Java service that needs Java 21, a Gradle build, and Docker for the AWS module. If you enable the extensions service, you are taking on a second Spring Boot process and a message queue between them. If you use the Python SDK, you are taking on a dependency in a different repository. Each of those is a thing that can break independently.

The material does not include a release cadence, a support policy, or a statement about which versions receive fixes. The repository is not archived and was pushed to recently, which tells you the code is live, but it does not tell you what an upgrade from your build to the next one will involve. Budget for reading the source and the blog posts rather than for reading release notes, because the release notes are not in evidence here.

Editorial conclusion

Adopt Maestro if your team already operates JVM services, wants a REST-driven DAG engine it can run on its own hardware, and is comfortable reading the source when the documentation stops. Do not adopt it if you need a managed control plane, a published release history, or a UI you can hand to non-engineers on day one. Before committing, verify three things: that a clean `./gradlew build` succeeds on Java 21, that `./gradlew bootRun --args='--spring.profiles.active=aws'` against the bundled LocalStack compose file brings up SQS and SNS without errors, and that the sample DAG at maestro-server/src/test/resources/samples/sample-dag-test-1.json produces a run you can retrieve from the instances endpoint.

Official sources

1. Issues
2. License: Apache-2.0
3. Netflix/maestro on GitHub
4. Project website
5. README