Questions3 replies
I am calling a model with a tool defined, and the tool call comes back as a JSON string inside a string, so my parser gets a quoted string rather than an object.
Instead of:
{"query": "how many stars"}
I get:
"{\"query\": \"how many stars\"}"
The model is clearly producing the right object, but wrapped. What is the right way to handle this — parse the string, or is something in my request causing it?
Accepted answer
Something in your request is causing it, and it is almost always one of two things.
1. You asked for JSON in the prompt as well as in the tool. If the system prompt says "respond with JSON" and there is a tool with a JSON schema, the model satisfies both by producing JSON as its text and then calling the tool with the same content — so you get a string containing the JSON, and the tool call. Use one or the other. If you have a tool, drop the instruction to produce JSON.
2. The tool is not declared as a function the model can call. Some providers have two modes, a "function" and a "custom tool" with free-form input. If you are using the free-form one, the input is a string by design, and the model is doing exactly what you asked. Switch to the function form, where the input is an object validated against your schema.
How to tell which. Look at the raw response, not the parsed one. If the tool call has a type of custom or the input field is a plain string while your schema is an object, it is the second. If the type is function and the input is a string containing JSON, it is the first, and the fix is the prompt.
What not to do. Do not write a parser that unwraps strings. It will work until the model returns a genuinely malformed tool call, at which point the unwrapper will hide the real error and you will spend an afternoon on the wrong bug. A missing tool call is a bug you want to see.
The fix for a model that is not using the tool at all is /tools/json-schema-generator for the schema and /learn/function-calling for the difference between the two.
The advice not to write an unwrapping parser deserves emphasis, because it is the difference between a loud failure and a silent one.
A string containing JSON is a recoverable case. A tool call that was never made, or one with a missing required field, is not, and an unwrapping parser turns both into the same confusing error.
It was the first one. The system prompt asked for JSON output and the tool was declared with a schema, so the model produced both: the tool call with a string, and the text.
Removing the "respond with JSON" sentence fixed it immediately. Accepted, and the "one or the other" framing is the bit I will remember.