Getting JSON out of the model

So far every answer from the model has been for people to read. But once a program has to process the answer, say, to file users' requests for help in a database automatically or route them to different people by category, what you need is JSON with a fixed format and every field present that can be parsed directly.

Getting the model to "output JSON" is easy. Getting it to output valid JSON with the right fields every time takes a bit of engineering. This lesson covers three layers of protection: JSON mode for the syntax, Pydantic validation for the content, and strict-mode tool calls for the structure.

What goes wrong with prompting alone

The most direct approach is to write "please output JSON" in the prompt. Most of the time it works, but now and then:

• It puts "好的，以下是提取结果：" (OK, here are the extracted results:) in front of the JSON, or wraps it in a ```json code block, and json.loads fails straight away.
• Field names are inconsistent: httpx_version this time, version the next.
• A field that should be a number comes back as a string; one that should be a list comes back as a comma-separated string.
• The answer is too long and gets cut off by max_tokens, and the JSON is missing its final brackets.

For a program called tens of thousands of times a day, even a 1% failure rate means hundreds of errors a day. So you add protection layer by layer.

Layer 1: JSON mode

DeepSeek and many OpenAI-compatible services support JSON mode: add response_format={"type": "json_object"} to the request, and the model's output is guaranteed to be syntactically valid JSON with no preamble.

DeepSeek's documentation (as of September 2026) has three requirements for JSON mode:

1. Set response_format={"type": "json_object"}.
2. The word "json" must appear in the system or user message, along with an example of the expected format.
3. Set max_tokens high enough that the JSON isn't cut off.

The second is a hard requirement. I tried leaving "json" out of the prompt, and the server simply refused:

提示词里没有 json，报错： Error code: 400 - {'error': {'message': "Prompt must contain the word 'json' in some form to use 'response_format' of type 'json_object'.", 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}

The documentation also warns that the API may occasionally return empty content. That means that even with JSON mode on, your code can't assume it will get a result.

Layer 2: validating with Pydantic

JSON mode guarantees syntax, not content: fields may be missing, types may be wrong. So after getting the JSON, check it with code.

The handiest tool in Python is Pydantic. It was already installed as a dependency when you installed openai. First define the structure you want as a class:

from pydantic import BaseModel


class BugReport(BaseModel):
    title: str
    httpx_version: str | None  # 原话里没提就是 null
    python_version: str | None
    os: str | None
    error: str | None
    missing_info: list[str]

str | None means the field can be a string or null. BugReport.model_validate_json(text) parses the JSON and checks every field: a missing field or a wrong type raises a ValidationError that says clearly which field has what problem.

Say clearly in the prompt what you want and give a format example, which also meets DeepSeek's requirement that "the word json appear":

SYSTEM = """从用户的求助原话中提取信息，输出 JSON。原话里没有的字段填 null，不要猜。
JSON 格式示例：
{"title": "一句话概括问题", "httpx_version": "0.27", "python_version": "3.11",
 "os": "macOS", "error": "报错类型或信息", "missing_info": ["排查还需要知道的信息"]}"""

When validation fails: tell the model what went wrong

When validation fails, the simplest effective move is to send the error message back to the model as it is and let it correct itself. Here's a piece of code you can reuse directly:

def extract(report, max_attempts=3):
    messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": report}]
    for attempt in range(1, max_attempts + 1):
        response = client.chat.completions.create(
            model=MODEL,
            messages=messages,
            response_format={"type": "json_object"},
            max_tokens=1000,
            extra_body={"thinking": {"type": "disabled"}},
        )
        content = response.choices[0].message.content
        try:
            return BugReport.model_validate_json(content), attempt
        except ValidationError as e:
            # 把错误原样告诉模型，让它改正。json 语法错误和字段错误都会走到这里
            print(f"第 {attempt} 次校验失败：{e.errors()[0]['msg']}")
            messages += [
                {"role": "assistant", "content": content or ""},
                {"role": "user", "content": f"你的输出没有通过校验：{e}\n请重新输出完整、正确的 JSON。"},
            ]
    raise RuntimeError(f"{max_attempts} 次都没有得到合法的输出")

A few details:

• When retrying, put the model's previous faulty output back as an assistant message, then explain what was wrong in a user message. When the model can see where it went wrong, it corrects itself more reliably than when you just ask again from scratch.
• Empty content (content is None or an empty string) also fails validation and likewise triggers a retry, which handles the documentation's "occasionally returns empty content".
• Set a maximum number of retries. If three retries don't fix it, the prompt or the data itself is probably the problem, and retrying further just wastes money; raise an error and let a person look.

Try it on the httpx request for help from lesson 1 (the complete code is in code/02-prompting/json_output.py):

第 1 次成功：
{
  "title": "httpx stream下载大文件中途ReadTimeout",
  "httpx_version": "0.27",
  "python_version": "3.11",
  "os": "macOS",
  "error": "ReadTimeout",
  "missing_info": [
    "具体的ReadTimeout异常堆栈",
    "当前timeout配置值",
    "重试逻辑或下载代码片段",
    "网络代理或内网限制情况"
  ]
}

This time it passed on the first try. In my tests, once JSON mode and a format example were in place, validation failures were rare. But "rare" isn't "never", and the retry code is insurance for those few cases.

The report you get back is a Python object: you can access fields directly as report.httpx_version, and your editor can autocomplete them. That's much more reliable than pulling values out of a dictionary with strings.

Layer 3: structured output through tool calls

There's another way, which constrains the structure from the start: have the model "call a tool" whose parameters are the structure you want.

Tool calling (function calling) is really for letting the model call external functions, which lesson 3 of module 03 covers in detail. Here we borrow just one of its features: you describe the tool's parameters with a JSON Schema, and the parameters the model generates follow that structure. DeepSeek also offers a strict mode: turned on, the parameters the model outputs conform strictly to the schema, and enum values can only come from the options you give.

As of September 2026, DeepSeek's strict mode is a beta feature. You change base_url to https://api.deepseek.com/beta, write "strict": True in the function definition, and the schema must include "additionalProperties": False:

client = OpenAI(
    api_key=os.environ["LLM_API_KEY"],
    base_url="https://api.deepseek.com/beta",
)

tool = {
    "type": "function",
    "function": {
        "name": "save_bug_report",
        "description": "保存从用户原话中提取出的问题信息",
        "strict": True,
        "parameters": {
            "type": "object",
            "properties": {
                "title": {"type": "string", "description": "一句话概括问题"},
                "httpx_version": {"type": "string", "description": "原话里没有就填空字符串"},
                "os": {"type": "string", "enum": ["macOS", "Windows", "Linux", "未知"]},
                "severity": {"type": "string", "enum": ["阻塞", "严重", "一般"]},
            },
            "required": ["title", "httpx_version", "os", "severity"],
            "additionalProperties": False,
        },
    },
}

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "提取这段求助里的信息并保存：\n" + REPORT}],
    tools=[tool],
    # 强制调用这个工具，而不是让模型自己决定要不要调用
    tool_choice={"type": "function", "function": {"name": "save_bug_report"}},
    extra_body={"thinking": {"type": "disabled"}},
)
call = response.choices[0].message.tool_calls[0]
print("模型调用了：", call.function.name)
print(json.dumps(json.loads(call.function.arguments), ensure_ascii=False, indent=2))

The output:

模型调用了： save_bug_report
{
  "title": "用httpx stream下载2G大文件到一半报ReadTimeout连接中断",
  "httpx_version": "0.27",
  "os": "macOS",
  "severity": "严重"
}

Both os and severity fall within the given enum values. The model didn't actually "save" anything; the save_bug_report function doesn't even exist, and we're only borrowing its parameters to get structured data.

tool_choice specifies which tool must be called. Without it, the model may decide not to call the tool and answer in text instead.

Choosing between the three

Whichever you use, don't skip validation in your program. Strict mode guarantees structure, not content: the model can still extract the wrong version number, or mark an "ordinary" problem as "severe". Correct structure is only the first step; whether the content is right has to be checked by the evaluation covered in module 06.

Two more small reminders:

• Fewer fields are more stable. Extracting twenty fields at once is far more error-prone than five. With many fields, consider splitting the work into several calls.
• Allow "none". Always give the model a way to say "this information isn't in the source", such as null or an empty string, and say so in the prompt. Otherwise, to fill every field, it will start making things up.

Exercises

1. Change missing_info in BugReport to list[int] (deliberately wrong) and run json_output.py to see what failing validation and retrying look like.
2. Add a severity field to BugReport that can only be one of "阻塞" (blocking), "严重" (severe) or "一般" (ordinary) (hint: use typing.Literal). Deliberately give it a request for help whose severity can't be told, and see how the model handles it.
3. Using the approach in json_strict.py, make a tool for lesson 2's message classification with the categories restricted by enum, process the 20 messages, and check whether every format is correct.

Self-check

1. With JSON mode on, why validate with Pydantic as well?

JSON mode only guarantees syntactically valid JSON, not that every field is present with the right name and type. DeepSeek's documentation also notes that the API occasionally returns empty content. Pydantic validation catches these problems, and together with retries deals with them.

2. When retrying after a validation failure, why put the model's previous faulty output back into the messages?

So the model can see what it output last time and, with the error message you give, correct it specifically. If you just ask the original question again, the model doesn't know what went wrong and is likely to make the same mistake.

3. What does tool_choice do when you use a tool call to get structured output?

It specifies that the model must call a particular tool. Without it, the model decides for itself whether to call a tool, and may answer in text instead, in which case you don't get the structured parameters.