WilmerAI: A Node-Based Workflow Engine for Routing LLM Prompts

The Problem: Routing on Context, Not Keywords

Most LLM routers classify a prompt by looking at the latest message. WilmerAI's README frames the limitation directly: a query like "What do you think it means?" is meaningless in isolation. If the preceding turns discussed the Rosetta Stone, the same sentence is a historical query; without that context it is small talk. WilmerAI's stated origin is the need for a router that understands the full conversation, not just the most recent message. The project is aimed at engineers building chatbots or any application that talks to an LLM over REST, and who want the routing decision to depend on conversation history. It is not aimed at someone who wants a thin proxy in front of one API key. The README describes the software as still under development and provided as-is, without warranty, which is worth reading literally before you put it in a request path.

How the Node-Based Workflow Engine Actually Runs

The core is a node-based workflow engine. Workflows are JSON files defining a sequence of steps, or nodes. Each node performs a specific task and its output can be passed as input to the next, which is how chained-thought processes are built. Routing itself is a workflow: the prompt router categorizes a request through a sequence of nodes, and the route chosen kicks off another specialized workflow, which can call further workflows from there. Two routing mechanisms are documented. Prompt routing runs at the start of a conversation and selects a specialized workflow such as Coding, Factual or Creative. In-workflow routing provides conditional if/then logic so a process can choose its next step based on a previous node's output. Each node can connect to a different LLM endpoint or execute a tool, so a single prompt can fan out to several machines and proprietary APIs depending on how the workflow is built. To the client application this multi-step process appears as a standard API call, which is the design decision that matters most: the complexity lives server-side, and existing front ends need no changes.

The Four-Part Memory System and What It Implies

Context-aware routing needs context to be stored somewhere. WilmerAI uses four memory components: a chronological summary file, a continuously updated rolling summary of the entire chat, a searchable vector database, and a continuously maintained state document describing the current state of the conversation. The vector database is keyword search by default, with optional embedding-based semantic or hybrid search; embedding endpoint support is called out in the v0.7.2 and v0.7.2.1 release notes. This design has a cost that the feature list does not advertise. Every turn is potentially writing to several stores, and the rolling summary is maintained continuously, which means an extra LLM call or calls on the critical path unless the workflow is built to avoid it. The README also notes that routing decisions can use the entire conversation history subject to the workflow's configured context limits, so the quality of routing is bounded by how much history you can afford to feed the router. Treat memory configuration as a tuning surface, not a checkbox.

Getting It Running: Python Version, Endpoints and Workflow JSON

The maintainer's notes state a minimum Python version of 3.13.14 or 3.14.5, with a caveat: if you do not intend to use the web fetcher, 3.11.10 or 3.12.13 should be fine, since web fetch is the only item that requires the higher versions. That caveat is the single most useful setup fact in the README, because it decides whether an existing deployment can be upgraded in place. WilmerAI exposes OpenAI- and Ollama-compatible API endpoints, so connection from an existing client is a base URL change rather than a code change. The work you actually do is authoring workflow JSON: a prompt router workflow, the specialized workflows it selects, and any reusable sub-workflows you want to call as nodes. The README gives a concrete example of tool use in a workflow, showing Open WebUI connected to two Wilmer instances where one hits Mistral Small 3 24b directly and the other calls the Offline Wikipedia API before the same model. Note that this recording predates multi-user support; a single instance can now serve multiple users.

Where WilmerAI Is the Wrong Tool

The obvious failure mode is treating WilmerAI as a drop-in proxy. It is not one. If your requirement is to forward a request to one model and return the response, the workflow layer adds JSON to maintain and nodes to debug for no routing benefit. The second limitation is authoring cost. Routing logic lives in JSON files, and the README's own illustrations are described as not-so-pretty pictures to help people visualize what it can do, which is a fair signal that the mental model takes effort to acquire. Chained workflows that call other workflows are powerful and also hard to trace when a route goes wrong; the README does not document a debugger or trace viewer. Third, the project is explicitly under development and provided as-is. Fourth, the embedding-based semantic search is optional rather than default, so out of the box you get keyword search over the vector store, which will miss paraphrases that an embedding search would catch. Finally, the README's own performance language is hedged: results depend on the model, prompts and task. Iterative LLM calls can improve a zero-shot answer, but nothing in the material promises a specific gain.

Alternatives and the Real Difference in Approach

The nearest comparison is LiteLLM, a widely used proxy that presents many providers behind one OpenAI-compatible interface. The difference is where the logic lives. LiteLLM routes by model name and provider configuration: the caller picks the target. WilmerAI routes by inspecting the conversation through a workflow, and the caller does not pick anything. If your problem is key management, fallbacks and cost tracking across providers, a proxy is the right shape. If your problem is that different kinds of requests should reach different models or different tool chains without the client knowing, WilmerAI's node engine is the closer fit. A second comparison is a hand-rolled FastAPI service that inspects the last message and dispatches. That is less code to maintain and easier to debug, and for single-keyword routing it is sufficient. WilmerAI's claim is specifically that routing on the whole conversation is different from routing on the last message, and that claim is the only reason to accept the workflow JSON.

Licence and Maintenance Costs to Weigh

WilmerAI is GPL-3.0. That is a copyleft licence, and if you plan to embed it inside a proprietary product or distribute a modified version, the licence terms govern how you may do so. This is not legal advice; read the licence text or consult counsel before shipping. On maintenance, the release history supplied shows v0.7.2 on 2026-07-20, v0.7.2.1 on 2026-08-09 and v0.7.3 on 2026-09-08, so the project is moving on a roughly monthly cadence with patch releases between minor ones. The v0.7.2.1 notes mention embedding endpoint support, new memories and bug fixes. Frequent releases are good for fixes and bad for workflow stability: JSON you author against one version may need revisiting as node behaviour changes. The README also states the maintainer is not doing contract, freelance or collaboration work, which means support comes from the repository and community rather than paid arrangements. Budget for someone on your team owning the workflow definitions, because that is the artifact that will drift.

Editorial conclusion

Adopt WilmerAI if you already run multiple LLM endpoints (local or proprietary) and want to encode routing logic as reusable JSON workflows behind an OpenAI-compatible endpoint. Do not adopt it if you need a single-call proxy, cannot work under GPL-3.0, or are unwilling to author and debug workflow JSON. Before committing, verify that your Python version satisfies the web fetcher requirement (3.13.14 or 3.14.5), that your target client speaks the OpenAI or Ollama API shape, and that the four-part memory system's context limits fit your conversation lengths. Run one trivial workflow end to end first; the routing layer is the part you are actually buying.

Official sources

1. Issues
2. License: GPL-3.0
3. README
4. Releases
5. SomeOddCodeGuy/WilmerAI on GitHub