0%

Structured Output · practice

Show the Model the Shape

Your program now knows exactly what a valid quiz looks like: QuizSet says so. The model does not. All it gets is QUIZ_SYSTEM, a sentence describing the object in words. This lesson sends the model the same description your program checks against, so the two cannot drift apart.

A model can describe itself

Every pydantic model can write out its shape as JSON Schema, a standard format for describing JSON documents:

import json

from pydantic import BaseModel


class QuizQuestion(BaseModel):
    question: str
    choices: list[str]
    answer_index: int


class QuizSet(BaseModel):
    questions: list[QuizQuestion]


schema = QuizSet.model_json_schema()
print(type(schema).__name__)
print(schema["required"])
print(schema["$defs"]["QuizQuestion"]["properties"]["answer_index"])
dict
['questions']
{'title': 'Answer Index', 'type': 'integer'}

model_json_schema() returns a dictionary. Its properties list the fields with their JSON types (string, integer, array for a list), required names the fields that must be there, and $defs holds the description of QuizQuestion, which questions refers to. You do not need to write JSON Schema yourself; you need to know that it is the model’s own description of itself, in a form other programs, and language models, understand.

Build quiz_system_message

quiz_system_message() returns the system message with the schema added after QUIZ_SYSTEM:

{"role": "system", "content": QUIZ_SYSTEM + "\n\nJSON schema:\n" + json.dumps(QuizSet.model_json_schema(), indent=2)}

json.dumps(..., indent=2) turns the dictionary into JSON text spread over indented lines, as in Python in Practice. A model reads the text either way; the indented form is what a person can check by eye. Build the schema inside the function, from QuizSet. If you pasted the schema text in instead, a later change to QuizSet would leave the message describing the old shape.

Then change quiz_request: its first message becomes quiz_system_message() instead of {"role": "system", "content": QUIZ_SYSTEM}. The note message, model and response_format stay as they are.

Who reads the schema

A live model reads the schema as part of its prompt and uses it to shape its reply. The schema gives it the exact field names and types, so it has less to guess than from a description in words. It is still a request, not a guarantee: the model can get it wrong, which is why your program validates every reply anyway.

The scripted model does not read the schema. It answers from its table of prepared replies, the same as before, and the system message plays no part in which reply it picks. So the output below is the same quiz as in Lesson 6.

There is a stricter way to send a schema, the json_schema response format, which asks the service itself to hold the model to the schema. You will use it in Chapter 5, through the openai client, which builds that request from QuizSet for you.

Run it

Run python structured.py. The demonstration prints the content of your system message. If quiz_request does not send it yet, it says so on the next line. Then it sends a prompt whose first reply is invalid on purpose, which the next lesson handles:

The system message from quiz_system_message:
You write multiple-choice quiz questions for a Python course. Reply with JSON only: an object with a questions list. Each question has question, choices (three or four strings) and answer_index (the position of the correct choice, counting from 0).

JSON schema:
{
  "$defs": {
    "QuizQuestion": {
      "properties": {
        "question": {
          "title": "Question",
          "type": "string"
        },
        "choices": {
          "items": {
            "type": "string"
          },
          "title": "Choices",
          "type": "array"
        },
        "answer_index": {
          "title": "Answer Index",
          "type": "integer"
        }
      },
      "required": [
        "question",
        "choices",
        "answer_index"
      ],
      "title": "QuizQuestion",
      "type": "object"
    }
  },
  "properties": {
    "questions": {
      "items": {
        "$ref": "#/$defs/QuizQuestion"
      },
      "title": "Questions",
      "type": "array"
    }
  },
  "required": [
    "questions"
  ],
  "title": "QuizSet",
  "type": "object"
}
A reply that fails validation, and one retry:
  First reply: {"questions": [{"question": "What does len() return?", "choices": ["The number of items", "The last item", "The first item"], "answer_index": "abc"}]}
Run stopped at the next function to build: retry_request is built in Lesson 8.

That is the whole system message the model receives with every quiz request: the instructions in words, then the exact shape in JSON Schema.

Why does quiz_system_message build the schema from QuizSet each time instead of containing the schema as pasted text?

The quiz request now carries the JSON schema in its system message. What does that guarantee about a live model's reply?

Task

In structured.py, replace the unfinished body of quiz_system_message(), then change quiz_request to use it.

  • quiz_system_message() returns {"role": "system", "content": ...}, where the content is QUIZ_SYSTEM, then "\n\nJSON schema:\n", then json.dumps(QuizSet.model_json_schema(), indent=2). Build the schema inside the function, from QuizSet, rather than pasting its text.

  • In quiz_request, replace the first message, {"role": "system", "content": QUIZ_SYSTEM}, with quiz_system_message(). Leave the rest of the body as it is.

Run python structured.py to see the system message with the schema, then check your work.