0%

Using the OpenAI Client · practice

Structured Output Through the Client

Chapter 4 got a validated quiz in four moves: ask for json_object, put the schema in the system message so the model knows the shape, validate the reply with QuizSet.model_validate_json, and send one corrective message when it failed. The client has a helper that does the first three in one call, and it uses a stricter request format that Chapter 4 could only mention.

The parse helper

response = client.chat.completions.parse(model=MODEL, messages=messages, response_format=QuizSet)
quiz = response.choices[0].message.parsed

client.chat.completions.parse takes the same arguments as create, plus response_format=QuizSet: the pydantic model class itself, not a dictionary. From that class it does three things:

  1. It builds a JSON Schema from QuizSet, as QuizSet.model_json_schema() did in Chapter 4, and sends it in the request’s response_format as json_schema, the stricter format. With "strict": true, a provider that supports it makes the model’s output follow the schema exactly.

  2. It sends the request as create would.

  3. It validates the reply’s text with QuizSet, and puts the resulting QuizSet object in response.choices[0].message.parsed.

Because the schema travels in the request format, the prompt no longer needs to carry it. quiz_messages goes back to the plain QUIZ_SYSTEM, without Chapter 4’s quiz_system_message(). QUIZ_SYSTEM still describes the quiz in words, which helps a live model and costs little.

As in Chapter 4, the scripted model does not read the schema: it answers the quiz prompt from its table, and SCRIPTED_MODEL.md says how it treats json_schema. A live model reads it, and with strict output it follows it.

When there is no quiz

Three things can happen instead of a quiz, and parse_quiz_reply handles each one differently.

The model refuses. A live model can decline a request, for example one it judges unsafe. Then message.parsed is None and message.refusal holds the model’s explanation as text. The scripted model never refuses. parse_quiz_reply turns a refusal into an error the caller cannot miss: raise ValueError(f"The model refused: {message.refusal}"). parsed is also None if a reply has no content at all. Then refusal is None too, and your error reads The model refused: None.

The reply does not validate. Then parse itself raises pydantic.ValidationError, the same error parse_quiz raised in Chapter 4. You never get a response object, so the reply’s text and its token counts are not returned to you. parse_quiz_reply catches nothing; the project decides what to do about an invalid reply.

The reply was cut off at the model’s token limit. Then parse raises openai.LengthFinishReasonError. Chapter 9 handles cut-off replies.

Build the two functions

quiz_messages(note_text) returns the messages for a quiz about a note:

[{"role": "system", "content": QUIZ_SYSTEM}, note_message(note_text, QUIZ_PROMPT)]

note_message is Chapter 2’s function: it puts the note inside <note> tags after the prompt. The scripted model matches the start of that message, QUIZ_PROMPT, exactly; a live model would accept any wording.

parse_quiz_reply(messages, *, client):

  1. Make one call: client.chat.completions.parse(model=MODEL, messages=messages, response_format=QuizSet). Pass nothing else.

  2. message = response.choices[0].message.

  3. If message.parsed is None, raise ValueError(f"The model refused: {message.refusal}").

  4. Otherwise return message.parsed.

QuizSet is imported from quiz_models.py at the top of the file.

Run it

Run python client_calls.py. The demonstration asks for a quiz about notes/03-lists.md and prints it from the QuizSet your function returned:

quiz = parse_quiz_reply(quiz_messages(note), client=client)
for question in quiz.questions:
    print(f"  {question.question} Answer: {question.choices[question.answer_index]}")

Then it calls the supplied last_request(), which fetches GET /v1/scripted/last-request: the last request body your sandbox sent to llm-api, exactly as the service received it. It prints that body’s response_format, which you never wrote:

A quiz through the parse helper:
parse_quiz_reply returned a QuizSet with 3 questions:
  For scores = [90, 72, 85], what is scores[-1]? Answer: 85
  What does scores.append(60) return? Answer: None
  Which line makes an independent copy of scores? Answer: backup = scores.copy()
The response_format the parse helper sent:
{
  "type": "json_schema",
  "json_schema": {
    "schema": {
      "$defs": {
        "QuizQuestion": {
          "properties": {
            "question": {
              "title": "Question",
              "type": "string"
            },
            "choices": {
              "items": {
                "type": "string"
              },
              "title": "Choices",
              "type": "array"
            },
            "answer_index": {
              "title": "Answer Index",
              "type": "integer"
            }
          },
          "required": [
            "question",
            "choices",
            "answer_index"
          ],
          "title": "QuizQuestion",
          "type": "object",
          "additionalProperties": false
        }
      },
      "properties": {
        "questions": {
          "items": {
            "$ref": "#/$defs/QuizQuestion"
          },
          "title": "Questions",
          "type": "array"
        }
      },
      "required": [
        "questions"
      ],
      "title": "QuizSet",
      "type": "object",
      "additionalProperties": false
    },
    "name": "QuizSet",
    "strict": true
  }
}

That is the json_schema request format. Its json_schema holds three things: the schema’s name, taken from the class name; "strict": true; and the schema itself, which is QuizSet.model_json_schema() from Chapter 4 with one addition. Every object in it now has "additionalProperties": false, which strict mode requires and which forbids keys the model does not declare. QuizQuestion sits under $defs, and "$ref": "#/$defs/QuizQuestion" points to it from the questions list.

After that, python client_calls.py goes on to Lesson 5’s demonstration, which uses only finished code, and stops at chat_turn, the project’s first function.

parse gets a reply in which answer_index is "abc". What does your code receive?

Why does quiz_messages use plain QUIZ_SYSTEM instead of Chapter 4's quiz_system_message(), which added the schema?

Task

In client_calls.py, replace the unfinished bodies of quiz_messages(note_text) and parse_quiz_reply(messages, *, client).

  • quiz_messages returns [{"role": "system", "content": QUIZ_SYSTEM}, note_message(note_text, QUIZ_PROMPT)], with no schema in the system message.

  • parse_quiz_reply makes one call, client.chat.completions.parse(model=MODEL, messages=messages, response_format=QuizSet), and reads message = response.choices[0].message.

  • If message.parsed is None, raise ValueError(f"The model refused: {message.refusal}"). Otherwise return message.parsed.

  • Catch nothing: a ValidationError from parse reaches the caller.

Run python client_calls.py to see the quiz and the request format the helper sent, then check your work.