ralph-loop-agent: an outer loop around the AI SDK tool call cycle

What the loop is and where the idea comes from

The package description is Continuous Autonomy for the AI SDK, and the README explains the idea as the Ralph Wiggum technique, a development methodology built around continuous AI agent loops.

The technique is described as elegantly simple: keep feeding an AI agent a task until the job is done. It is attributed to Geoffrey Huntley, whose formulation the README quotes directly: Ralph is a Bash loop. The name comes from the persistently enthusiastic Ralph Wiggum from The Simpsons, and the method embraces iterative improvement over single-shot perfection.

The contrast drawn is with conventional agentic workflows, which stop when the model finishes calling tools. Ralph keeps going, verifying completion, providing feedback and running another iteration until the task actually succeeds. The README frames it as while (true) for AI autonomy: the agent works, an evaluator checks the result, and if it is not done the agent tries again with context from previous attempts.

The architecture diagram makes the nesting explicit. There is an outer Ralph Loop and, inside it, an inner AI SDK tool loop where the model talks to tools until it is done. When the inner loop finishes, verifyCompletion asks whether the task is actually complete. If not, feedback is injected and another iteration runs. If yes, the final result is returned.

That two-level structure is the point of the package. The AI SDK's tool loop is a good inner loop, but it terminates on the model's judgement that it has nothing left to do, which is a statement about tool calls rather than about the task. The outer loop substitutes a different termination criterion.

The reasons the README gives for wanting it are verification, whether the agent accomplished what was asked; persistence, retrying on failure instead of giving up; feedback loops that guide the agent from real-world checks; and long-running tasks such as migrations, refactors and multi-file changes.

The package is marked experimental, with a note that APIs may change between versions, and it carries the Vercel Labs experiment badge.

The API: an agent, a stop condition and a verifier

The basic example is short enough to show the whole shape:

import { RalphLoopAgent, iterationCountIs } from 'ralph-loop-agent';

const agent = new RalphLoopAgent({
  model: 'anthropic/claude-opus-4.5',
  instructions: 'You are a helpful coding assistant.',
  stopWhen: iterationCountIs(10),
  verifyCompletion: async ({ result }) => ({
    complete: result.text.includes('DONE'),
    reason: 'Task completed successfully',
  }),
});

const { text, iterations, completionReason } = await agent.loop({
  prompt: 'Create a function that calculates fibonacci numbers',
});

Four configuration pieces matter. The model and instructions are the ordinary AI SDK pair. stopWhen is the safety limit, and iterationCountIs(10) is one of several available limits. verifyCompletion is the function that decides whether the work is actually finished, and it returns both a complete flag and a reason.

The result object carries the final text, the number of iterations it took, and the completionReason, which is the value from the verifier. That last field is what makes the loop debuggable: you can log why it stopped without instrumenting the loop itself.

The stop conditions are described in the feature list as flexible, limiting by iterations, tokens or cost. Three different budgets for three different failure modes: an agent that keeps making progress but needs more turns, an agent that is looping without producing much, and an agent that is making expensive calls. Only the first is a good reason to raise a limit, and the other two are reasons to lower one.

The feature list also names full AI SDK compatibility, using the AI Gateway string format and supporting all the AI SDK tools, streaming support to stream the final iteration for responsive user interfaces, built-in summarisation for context management across long loops, and feedback injection so that a failed verification can guide the next attempt.

The migration example, where the design shows its value

The second example is the one that justifies the package, because it is a task where the completion criterion is genuinely checkable.

The instructions carry their own completion criteria: all test files use vitest imports, vitest.config.ts exists, and all tests pass when running pnpm test. The agent is given three tools, readFile, writeFile and execute, and a much larger iteration budget, iterationCountIs(50), because a codebase migration is many small edits.

The verifier is the interesting part:

  verifyCompletion: async () => {
    const checks = await Promise.all([
      fileExists('vitest.config.ts'),
      !await fileExists('jest.config.js'),
      noFilesMatch('**/*.test.ts', /from ['"]@jest/),
      fileContains('package.json', '"vitest"'),
    ]);
    
    return { 
      complete: checks.every(Boolean),
      reason: checks.every(Boolean) ? 'Migration complete' : 'Structural checks failed'
    };
  },

Four structural checks, run concurrently, and the loop continues until every one passes. This is a task where the naive approach fails constantly: the model migrates most files, declares success, and the remaining stragglers are exactly what a script should catch. The verifier here is a test suite of one, and the agent cannot talk its way past it because the check does not read the model's output.

That is the design point worth taking from the example. The strongest verifiers are ones that examine the world rather than the answer, and this one reads the filesystem.

The two lifecycle hooks are also here, onIterationStart and onIterationEnd, the latter receiving the iteration and a duration. Those are what you would use to log, to display progress, or to abort from outside the loop.

And the result is destructured into text, iterations and completionReason, the same shape as the basic example.

Tools, context and the package layout

The tools example shows how the AI SDK's tool definition is passed straight through, with the parameters declared as a zod schema and an execute function:

import { RalphLoopAgent, iterationCountIs } from 'ralph-loop-agent';
import { tool } from 'ai';
import { z } from 'zod';

const agent = new RalphLoopAgent({
  model: 'anthropic/claude-opus-4.5',
  instructions: 'You help users with file operations.',
  tools: {
    readFile: tool({
      description: 'Read a file from disk',
      parameters: z.object({ path: z.string() }),
      execute: async ({ path }) => ({ content: '...' }),
    }),
  },
  stopWhen: iterationCountIs(10),
  verifyCompletion: ({ result }) => ({
    complete: result.text.includes('All files updated'),
  }),
});

Nothing special is required to add a tool. The definition is the AI SDK's own, the zod schema is the parameter contract, and the execute function does the work. That is the compatibility claim in the feature list taken literally.

Context management is the other feature that matters for long loops. Built-in summarisation keeps the conversation within the model's window while the agent keeps working, so a fifty-iteration run does not blow the context by iteration ten. The trade is that a summarised history is a lossy history, which is why the feedback injection exists: the thing the next iteration most needs to know is why the last one was judged incomplete, and that arrives as injected feedback rather than as remembered context.

The repository is a pnpm workspace with turbo driving the builds. The root package.json is named ralph-monorepo and is private, with turbo as its only devDependency and pnpm 9.15.4 as the package manager. The scripts are build, dev, test and type-check, all delegating to turbo, plus an examples:cli script that filters to the CLI example package.

There are two published things described in the README's tables. The core package is ralph-loop-agent, described as the core agent framework with loop control, stop conditions and context management. And there is one example, a full-featured CLI agent with Vercel Sandbox, Playwright, PostgreSQL and GitHub PR integration.

Limits: verification is the hard part, and the loop can burn tokens

The limits are inherent to the approach rather than gaps in the implementation.

The first is that the verifier is the hard part, and the package cannot write it for you. The basic example verifies on the string DONE appearing in the output, which works for a toy task and is exactly the pattern that fails on real work. The migration example is the one to copy, because it checks the filesystem. Any team adopting this has to write a verifier that actually reflects their task, and that is usually the bulk of the work.

The second is that the loop terminates on your limit, not on the task. An unbounded outer loop around a model that keeps failing the same check is an expensive way to make no progress, and the token and cost limits exist for that reason. Setting them is not optional in practice, and choosing the right one is a judgement about how many attempts a failure mode deserves.

The third is the retry pattern itself. When the verifier fails for a structural reason, such as the agent is not calling the right tool, the loop will keep retrying with the same mistake and the feedback will not help, because the failure is not something the model can reason its way out of. Loops amplify whatever the agent is already doing, good or bad.

The fourth is the experimental status. The README says plainly that the package is experimental and that APIs may change between versions, and the repository carries a single topic of experimental. There are no GitHub releases, so the version you get from npm is the version you depend on and there is no tagged history to consult.

The fifth is the dependency on the surrounding stack. The package targets the AI SDK and expects zod for tool schemas, and the CLI example pulls in Vercel Sandbox, Playwright and PostgreSQL. A team that is not already on the AI SDK has to adopt that first, and the AI SDK's own API is where most of the churn in this area has been.

This package against writing your own retry loop and against an agent framework with built-in planning

Two comparisons make the choice clearer.

The first is against writing the outer loop yourself around the AI SDK's generateText. It is not much code: call the model, run the verifier, and call again. What the package adds is the parts that are easy to get wrong and tedious to get right. The stop conditions are uniform, whether by iteration, token or cost. The context summarisation across long loops is handled, which is the thing that otherwise breaks silently once the window fills. The feedback injection has a defined shape. And the iteration lifecycle hooks exist so you can observe and abort from outside.

The cost is a dependency on an experimental package, and a layer between you and the AI SDK where the underlying API is still moving. For a single task where two retries will do, writing the loop inline is less trouble than taking the dependency.

The second comparison is against an agent framework with planning built in. Those frameworks add a planner that decomposes a task into steps, tracks progress across them, and decides what to do next. The difference in approach is where the correctness criterion comes from. A planner framework decides it is finished because its own plan says every step is done. This package decides it is finished because your code says the work is verified. For tasks with a mechanical success condition, the second is far stronger, and the planner's self-assessment is the weakest link.

There is a third path worth naming, which is a different kind of automation entirely: a script that does the migration with no model in it. The migration example's verifier is a script. If the task is mechanical enough to verify with four filesystem checks, you can often perform it with a script, and the model is only worth adding where the work requires judgement.

So the honest framing is that this package is for the middle case: work that needs a model to do, and that can be checked without asking the model. The verifier is the contract, and the loop is the mechanism for satisfying it.

Editorial conclusion

Adopt ralph-loop-agent when you have a task whose success can be checked mechanically, such as a migration with verifiable structural conditions, and the AI SDK tool loop stopping early is the failure you keep hitting. Do not adopt it for tasks where completion is a matter of judgement, since the loop needs a programmatic check to terminate. Verify first that your stop condition is set, because the default behaviour of an unbounded loop is to keep spending tokens, and that the verify callback you write actually reflects the task rather than a marker in the output text.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. vercel-labs/ralph-loop-agent on GitHub