delivery-harness: a Java reference implementation for AI-assisted on-demand delivery workflows

The problem: separating model advice from the decision that pays out

Most teams wiring a language model into an operations flow hit the same wall. The model can summarize an abnormal order well enough, but nobody wants it to decide how much money a customer receives. delivery-harness takes a position on that split. The README states that compensation amounts are decided by the rule engine, never by the model, and it points to ADR-0002 in docs/ for the reasoning. The model's job is to produce an analysis that sits next to a deterministic baseline, not to replace it.

The audience is narrow and specific. This is a reference implementation for engineers evaluating how an AI harness should be structured: which parts stay deterministic, where the model call is isolated, how a trace is threaded through a request, and what guardrails look like in code. It is written in Java 17 with Spring Boot 3.5.16, so it speaks to teams already on that stack. If you are looking for a delivery product, this is not one. The repository description calls it an AI harness reference implementation, and the README's own callout labels it an educational MVP rather than a production-ready delivery or compensation system.

How the harness is wired: modules, synchronous steps and a trace per request

The architecture graph in the README shows a single request path. A REST client hits harness-api, which handles HTTP, validation and tracing. From there the core agent runs a fixed workflow, calling out to four locally registered tools for orders, ETA, station capacity and compensation rules, to the knowledge module for seeded rules and lexical retrieval, to the llm module for the model call, and to the observe module for traces and metrics. The eval module also drives the agent, which is how evaluation runs reuse the same workflow code instead of duplicating it.

Two design choices stand out. First, everything is synchronous and in-process. The README states that all workflow steps run synchronously in the gateway process, and that apart from the configured model endpoint the default application has no required external services. Second, the deterministic timeline is treated as a baseline the model has to beat. The README describes it splitting an order into dispatch wait, to-shop, merchant prep and on-road legs, reported alongside the model's answer. That framing matters: you can compare a model's output against arithmetic instead of against nothing.

The Maven build was consolidated from nine modules into three. The README notes that Java packages are unchanged, with com.delivery.harness.{agent,tool,llm,knowledge,eval,observe} still present and still meaning the same thing, and that only the build boundary moved. ADR-0004 in docs/ covers that decision. The llm module sits behind an interface, which the README gives as the reason workflows are testable without a model. That is the single most reusable idea in the repository.

Installing delivery-harness and running your first analysis

Prerequisites are listed in the README: JDK 17 or newer, Maven 3.6.3 or newer or the included Maven Wrapper, Docker with Compose support only if you run Ollama in a container, and enough memory for the model you choose. The default example is qwen2.5:7b.

Start the model first. If Ollama already runs on your machine, the README says to skip the first command. The Compose port binds to 127.0.0.1 only, and the README notes that model weights are downloaded separately and are not part of the repository or its MIT license.

docker compose up -d ollama
docker compose exec ollama ollama pull qwen2.5:7b

Then build and test. The README gives this as the verification command:

./mvnw clean verify

Package and run the API. The README states the API listens on http://localhost:8080 by default.

./mvnw -pl harness-api -am package
java -jar harness-api/target/harness-api.jar

Now call the abnormal-order workflow. The README gives this exact request, with order_id DEMO-001.

curl --fail-with-body \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{"order_id":"DEMO-001"}' \
  http://localhost:8080/api/v1/analyze/abnormal-order

Expect a JSON envelope with code, message, data and traceId fields. Inside data you should see executionId, traceId, status and a steps array. The README states the same trace ID is returned in the X-Trace-Id response header, so you can correlate the body with the header. For the compensation workflow, the README's example uses order_id DEMO-002 with complaint_type OVERTIME. For DAMAGED complaints it says to pass damage_level as FULL or PARTIAL; without evidence the policy returns zero and requires manual review. Configuration lives in environment variables documented in .env.example, including HARNESS_LLM_ENDPOINT, HARNESS_LLM_MODEL, HARNESS_LLM_API_KEY, HARNESS_LLM_TIMEOUT_SECONDS and HARNESS_MAX_COMPENSATION_AMOUNT.

DEGRADED status: what happens when the model endpoint is down

The failure mode the README documents most clearly is partial degradation. Completed workflows with an unavailable optional dependency are reported as DEGRADED, the deterministic result remains available, and the affected step is recorded as failed. That is a deliberate design: the timeline and the rule-engine output survive a model outage, and the response tells you which step did not run.

What the README does not document is rollback. There is no described mechanism for undoing a workflow, and there is no durable state to roll back to, because most state is held in memory. The trace store is described as bounded, so traces are evicted rather than archived. Restart the process and the seeded rules, ingested documents and evaluation cases are gone. For an educational harness that is acceptable; for anything resembling an audit trail it is not.

The evaluation scaffolding has a subtler honesty feature worth noting. The README states that scorers report not measured rather than a perfect score when a case declares no expectation. A missing expectation therefore cannot masquerade as a passing test. That is a small detail with real consequences for anyone reading an eval report: a green run with many not-measured entries means less than it appears to.

Where delivery-harness is the wrong tool

The README's own callout is unusually direct, and it should be taken literally. Business tools use synthetic data, most state is in memory, and model output is advisory. It instructs readers not to expose the application to the public internet, not to submit real personal or order data, and not to execute compensation decisions without authentication, policy enforcement and human approval. None of those three are implemented. The README lists authentication and rate limiting among the things intentionally not claimed as implemented, alongside autonomous agent planning, production tool integrations, durable persistence, vector embeddings or Milvus RAG, distributed tracing and automatic compensation execution.

So the boundary is sharp. If you need durable order state, real carrier or merchant integrations, or an agent that plans its own steps, this repository gives you none of them and says so. The four tools are registered locally and every tool derives its result from its arguments, which means they are deterministic stubs, not connectors. Synthetic orders vary by order ID across six named scenarios, one of which is delivered on time. That is a test fixture, not a data source.

There is also a versioning caveat that matters more than it looks. The README describes this as a 0.x release where the HTTP surface and response fields may change without a major-version bump, and advises pinning a tag if you depend on the response shape. If your integration parses the envelope, pin the tag.

How this differs from a general CI/CD or DevOps platform

Search traffic around the word harness overwhelmingly means something else: a commercial software delivery platform used for pipelines, deployments and feature flags. delivery-harness shares a word and nothing else. It does not build, test or deploy your code. The Maven Wrapper and GitHub Actions CI in this repository are how the project tests itself, not a product feature.

The real alternative for the problem this project addresses is assembling the same pieces yourself: a Spring Boot controller, an HTTP client for an OpenAI-compatible chat-completions endpoint, a rules table, and a logging filter for trace IDs. That is maybe a few hundred lines, and you would own the schema. The difference in approach is that delivery-harness pre-commits to a structure: the model client sits behind an interface so workflows run without a model, the rule engine owns the compensation amount, and the deterministic timeline is emitted as a baseline rather than as an internal detail. You get a worked example of those boundaries plus 116 tests and a local Ollama profile, at the cost of adopting someone else's module split and a 0.x response contract. If your team has already settled those boundaries, copying the ADRs may be more useful than depending on the JAR.

Licence, maintenance and the cost of upgrading

The project is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. Two caveats are worth stating without straying into legal advice. The README notes that model weights are downloaded separately and are not part of the repository or its MIT license, so the licence you care about for qwen2.5:7b is the model's, not this project's. The repository also carries a THIRD_PARTY_NOTICES.md file, which is where dependency attributions live.

On maintenance, the last push to the default branch was on 2026-09-16, and the repository is not archived. Two releases are listed: v0.1.0 on 2026-08-27 and v0.2.0 on 2026-09-01. The CHANGELOG.md file at the repository root is the place to look for what moved between them.

Upgrade cost is dominated by the 0.x contract. Because the README states the HTTP surface and response fields may change without a major-version bump, an upgrade can break a client that parses the envelope even when the version number only moves in the minor position. The README's advice to pin a tag is the practical mitigation. Tagged releases publish the runnable harness-api fat JAR and a SHA-256 checksum as GitHub Release assets, so a pinned deployment can be verified against the published checksum. Beyond that, the README does not document a migration process between releases.

Editorial conclusion

Adopt delivery-harness if you need a small, readable Java codebase that shows how a deterministic timeline, mock tools, lexical retrieval and an OpenAI-compatible model call fit together behind a REST envelope, and you are comfortable treating the result as a teaching artifact. Do not adopt it for production dispatch or payouts: the README states plainly that business tools use synthetic data, most state is in memory, and there is no authentication, rate limiting or durable persistence. Before you build anything on top, verify two things in the repository: the rule engine's ownership of the compensation amount in docs/adr/0002-rule-engine-owns-the-compensation-amount.md, and the three-module Maven layout described in docs/adr/0004-three-maven-modules-instead-of-nine.md, because the README warns that the HTTP surface and response fields may change without a major-version bump.

Frequently asked questions

Official sources

1. diudiu-tech/delivery-harness on GitHub
2. Issues
3. License: MIT
4. README
5. Releases