The model is the expensive part of most agent bills, and the reflex is to change the model. Usually that is the third cheapest option. There are three levers that come before it, and they are worth more than a model swap in most cases.
Lever one: cache the prefix
Almost every request in an agent run repeats a large stable prefix: the system prompt, the tool definitions, the project instructions, and often the same file contents. Providers will serve a repeated prefix from a cache at a much lower price than the normal input rate.
Three things decide whether this works:
- The prefix has to be byte-identical. A timestamp, a request id, or a differently ordered list of tools in the prefix and the cache misses.
- Keep the stable part first. Anything that changes per request belongs at the end, after the cached prefix, or every request is a miss.
- The cache is not forever. It expires, and the lifetime differs per provider. A workload with a 30-second gap between requests gets less from this than one with a busy loop.
Measure it before you believe it: the provider's own usage fields usually report cached input separately from fresh input, and that number is the only way to know whether the prefix is actually hitting.
Lever two: make the output structured
The failure mode of a free-text model is not that it is wrong, it is that it is right in a format you have to parse. Every parse failure becomes a retry, and a retry is a full request.
A JSON schema in the request removes the parsing problem:
{
"name": "project_facts",
"description": "Facts about a project page",
"parameters": {
"type": "object",
"properties": {
"license": { "type": "string" },
"maintained": { "type": "boolean" },
"last_push": { "type": "string" }
},
"required": ["license", "maintained"]
}
}
/tools/json-schema-generator writes the schema, /tools/json-repair fixes the ones that come back broken, and /learn/structured-output explains what the model is doing when it fills one in.
The point that matters: one request that returns valid JSON is cheaper than two requests where the second one is "sorry, here it is again".
Lever three: use tool calls instead of asking for JSON in prose
Asking a model for JSON in the prompt gets you JSON most of the time. A tool call gets you JSON with a schema the provider enforces, and it gives the model a way to ask for the thing it does not have.
This changes the shape of the request. Instead of one prompt with a long instruction about the output format, you have a set of tool definitions and a short instruction. The definitions cost tokens in every request, so there is a real trade-off: twenty tools is twenty schemas carried on every turn.
What to do with the numbers
The order that works: count first, price second, change third.
- /tools/token-counter on the system prompt, the tool definitions and one representative request tells you what you are actually sending.
- /tools/llm-cost-calculator turns that into a monthly figure, with a field for the cached share of the input if your provider has one.
- Then change the thing that dominates. Often it is the tool definitions, often it is the number of round trips, and occasionally it is the model.