Rill Flow: A Java Workflow Engine That Puts HTTP Calls and LLM Steps on the Same DAG
Rill Flow is a high-performance, scalable workflow orchestration engine for distributed workloads and LLMs
At a glance
- What is it?
- Rill Flow is a Java, Apache-2.0 workflow orchestration service from Weibo that runs DAGs of function tasks over HTTP and ships a Docker Compose stack for local evaluation. Its YAML task model is simple enough to read in one sitting, but the README stops well short of explaining scheduling, persistence and failure semantics.
- Who is it for?
- Adopt Rill Flow if your pipeline is already a set of HTTP endpoints and you want a Java service that turns them into a DAG with a UI and execution records, rather than writing that scheduler yourself. Do not adopt it if you need a documented exactly-once guarantee, a plugin SPI you can extend from the README alone, or a deployment that avoids MySQL and Redis.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 156 days ago.
- What is it written in?
- Mainly Java, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The Gap Rill Flow Fills: DAG Semantics for Services That Are Just HTTP Endpoints
Most teams that need a workflow engine already have the work. It lives behind HTTP endpoints: an internal greeting service, a model inference gateway, a data cleanup job exposed as a REST route. What they lack is a scheduler that knows Bob must run before Alice, that retries the second call if it fails, and that shows a human which step broke. Rill Flow targets exactly that shape. The README describes it as a distributed workflow orchestration service, and the example DAG makes the intent concrete: two tasks, both with category function, both pointing at resourceName values that are plain HTTP URLs on a sample executor. The engine's job is ordering, input mapping and execution records, not the work itself.
The audience follows from the deployment story. The quickstart is Docker Compose with MySQL, Redis, Jaeger, a Java service, a Python sample executor and a UI. That is a stack for a backend team that has somewhere to run containers and wants a self-hosted control plane rather than a managed cloud workflow product. The project is written in Java and licensed Apache-2.0, which matters if your platform team already operates JVM services and has opinions about what enters the cluster.
The Task Model: YAML, JSONPath Mappings and a next Pointer
The unit of work is a task in a YAML document. In the README example the document opens with version, workspace, dagName, alias and type, then declares an inputSchema as a JSON array of required typed fields. Each task then carries a category, a name, a resourceName, a pattern, a tolerance flag, a next field and a list of inputMappings.
Two of those keys carry most of the design. The next field is how the DAG is expressed: Bob's task sets next: Alice, so the edge is a property of the node rather than a separate edge list. The inputMappings array is the data flow. Each entry has a source and a target, and both are JSONPath expressions such as $.context.Bob into $.input.Bob. That means the engine is not passing opaque payloads between tasks. It is resolving a path against a context object and writing the result into the next task's input. The context namespace appears to be where the flow-level inputs land, since the inputSchema declares Bob and Alice as required String fields and the mappings read them from $.context.
pattern: task_sync says the call is synchronous. tolerance: false says a failure in that task is not tolerated. Neither is elaborated in the README, and that is the honest limit of what can be said here: the YAML shows the vocabulary, not the semantics behind every keyword.
Getting It Running: Four Commands and a Compose File
The documented path is short. Clone the repository, enter the docker directory, and bring the stack up:
git clone https://github.com/weibocom/rill-flow.git cd rill-flow/docker docker-compose up -d
The README notes that systems with Compose V2 should use docker compose instead of docker-compose, and suggests checking with docker compose version. Verification is docker-compose ps, and the expected output lists six containers: rill-flow-mysql, rillflow_cache_1, rillflow_jaeger_1, rillflow_rill-flow_1, rillflow_sample-executor_1 and rillflow_ui_1. Ports in that output are worth reading before you start: MySQL on 3306, Jaeger's UI on 16686, the Java service on 8080, and the UI on 80. The admin console is at http://localhost with the credentials admin/admin.
Submitting work is a UI operation, not a CLI one. Create a flow definition, open the graph editor, enable one-click import, paste the YAML, submit, then use Test to supply the required parameters and Submit to run it. The result appears on an execution details page reachable through Execution Records. There is no documented curl against the 8080 port for submitting a flow, so if you want to drive Rill Flow from CI you will be reading the source or the docs site, not the README.
What the README Does Not Tell You About Running This in Production
The overview lists four claims: tens of millions of tasks per day, task execution latency under 100ms, orchestration of heterogeneous distributed systems, and cloud native deployment. Treat all four as vendor statements. There is no benchmark in the README, no description of the test topology, and no indication of what task shape the latency figure refers to. A 100ms figure is plausible for dispatching an HTTP call and is meaningless for a task that waits on a slow downstream service. The number describes the engine's overhead, if it describes anything, and the README does not say so.
The bigger gap is operational semantics. The Compose stack includes MySQL and Redis, so state is external, but the README never states what happens to an in-flight DAG when the rill-flow container restarts, whether a task can execute twice after a failover, or how the tolerance flag interacts with retries. Those are the questions that decide whether an engine is safe for payment-adjacent work, and they are not answered in the material available here. The absence is not proof of a flaw. It is proof that you cannot evaluate this dimension from the README, and you should not assume it away.
The LLM Angle Is a Topic Tag Until the Docs Say Otherwise
The repository description calls Rill Flow an engine for distributed workloads and LLMs, and the topics list includes llm and agent. The README's own feature list says it supports rapid integration of LLM model services. Nothing in the quickstart demonstrates that. The sample executor is a Python FastAPI service serving a greet.json endpoint, and the example DAG calls it twice with different user parameters. That is a generic HTTP task, not an LLM call.
This does not mean the LLM support is absent. It means the README gives you a claim and no mechanism. There is no example of a prompt template, a model provider config key, a streaming response, or a token accounting field. If LLM orchestration is why you are here, the honest next step is the documentation site, not the repository front page. A workflow engine that can call an HTTP endpoint can call a model gateway, so the capability is likely real at some level. Whether it is a first-class integration or simply HTTP with a marketing label is a question the README does not settle.
How Rill Flow Differs From Airflow and Temporal
The natural comparison is Apache Airflow. Airflow's centre of gravity is scheduled batch pipelines: cron-style intervals, backfills, a Python DAG definition file, and an operator ecosystem built around data systems. Rill Flow inverts several of those choices. Its DAGs are YAML submitted through a UI rather than Python files in a repository, its default trigger in the documented example is manual, and its task abstraction is an HTTP call rather than a Python operator. If your pipelines are data-warehouse loads on a schedule, Airflow's model fits better and its ecosystem is larger. If your pipelines are service calls triggered on demand, Rill Flow's model is closer to the problem.
Temporal is the other useful contrast. Temporal asks you to write workflow logic as code in a client SDK and handles durable execution state as its core value proposition. Rill Flow keeps the workflow as declarative configuration and the tasks as external services. That is a lighter conceptual load and a weaker guarantee: with declarative HTTP tasks, the engine cannot replay your business logic, only re-invoke the endpoint. The two tools are answering different questions, and the answer you need depends on whether your steps are idempotent.
Maintenance Surface and the Apache-2.0 Licence in Practice
The Compose stack tells you what you are signing up to operate. MySQL holds state, Redis is a cache, Jaeger collects traces, and the Java service and UI are the application. That is four stateful or semi-stateful dependencies for a workflow engine, and each one has its own upgrade path. The Jaeger container in the expected output runs the all-in-one binary, which is a development configuration rather than a production tracing deployment. Expect to replace it if you keep the stack.
The repository lists three maintainers and eight additional contributors, which is a small team for a system that would sit on the critical path of your pipelines. That is a statement about bus factor, not about code quality, and it is the kind of thing you weigh when the engine is between your services and their callers. Apache-2.0 is permissive: it allows commercial use and modification, and it includes a patent grant. It also means no vendor is obligated to support you. Nothing here is legal advice, and if you plan to redistribute a modified Rill Flow you should read the licence text and your own counsel's view of it.
Who Should Take the Compose File for a Spin
The lowest-risk way to evaluate Rill Flow is the path the README already lays out: run the Compose stack on a workstation, import the two-task greet DAG, and watch an execution record appear. That takes minutes and tells you whether the YAML model matches how your team thinks about pipelines. From there, the questions worth answering are specific. Replace the sample executor URL with one of your own endpoints and see what an inputMapping failure looks like in the execution record. Kill the rill-flow container mid-run and inspect MySQL to learn what the engine persisted. Try a task that fails with tolerance set to true and observe whether downstream steps still run.
Those three experiments will tell you more than the feature list does. If the answers are acceptable, Rill Flow is a reasonable fit for internal service orchestration where a Java control plane and a visual editor are worth more than a large operator ecosystem. If they are not, the README has not given you a reason to keep going.
Editorial conclusion
Adopt Rill Flow if your pipeline is already a set of HTTP endpoints and you want a Java service that turns them into a DAG with a UI and execution records, rather than writing that scheduler yourself. Do not adopt it if you need a documented exactly-once guarantee, a plugin SPI you can extend from the README alone, or a deployment that avoids MySQL and Redis. Verify three things before committing: what the persistence layer does to a running DAG when the rill-flow container restarts, whether the task categories beyond function are described anywhere in the docs, and whether the LLM integration is a real feature or a topics tag.
Community notes