Using the OpenAI Client · practice
Structured Output Through the Client
Chapter 4 got a validated quiz in four moves: ask for json_object, put the schema in the system message so the model knows the shape, validate the reply with QuizSet.model_validate_json, and send one corrective message when it failed. The client has a helper that does the first three in one call, and it uses a stricter request format that Chapter 4 could only mention.
The parse helper
response = client.chat.completions.parse(model=MODEL, messages=messages, response_format=QuizSet)
quiz = response.choices[0].message.parsed
client.chat.completions.parse takes the same arguments as create, plus response_format=QuizSet: the pydantic model class itself, not a dictionary. From that class it does three things:
It builds a JSON Schema from
QuizSet, asQuizSet.model_json_schema()did in Chapter 4, and sends it in the request’sresponse_formatasjson_schema, the stricter format. With"strict": true, a provider that supports it makes the model’s output follow the schema exactly.It sends the request as
createwould.It validates the reply’s text with
QuizSet, and puts the resultingQuizSetobject inresponse.choices[0].message.parsed.
Because the schema travels in the request format, the prompt no longer needs to carry it. quiz_messages goes back to the plain QUIZ_SYSTEM, without Chapter 4’s quiz_system_message(). QUIZ_SYSTEM still describes the quiz in words, which helps a live model and costs little.
As in Chapter 4, the scripted model does not read the schema: it answers the quiz prompt from its table, and SCRIPTED_MODEL.md says how it treats json_schema. A live model reads it, and with strict output it follows it.
When there is no quiz
Three things can happen instead of a quiz, and parse_quiz_reply handles each one differently.
The model refuses. A live model can decline a request, for example one it judges unsafe. Then message.parsed is None and message.refusal holds the model’s explanation as text. The scripted model never refuses. parse_quiz_reply turns a refusal into an error the caller cannot miss: raise ValueError(f"The model refused: {message.refusal}"). parsed is also None if a reply has no content at all. Then refusal is None too, and your error reads The model refused: None.
The reply does not validate. Then parse itself raises pydantic.ValidationError, the same error parse_quiz raised in Chapter 4. You never get a response object, so the reply’s text and its token counts are not returned to you. parse_quiz_reply catches nothing; the project decides what to do about an invalid reply.
The reply was cut off at the model’s token limit. Then parse raises openai.LengthFinishReasonError. Chapter 9 handles cut-off replies.
Build the two functions
quiz_messages(note_text) returns the messages for a quiz about a note:
[{"role": "system", "content": QUIZ_SYSTEM}, note_message(note_text, QUIZ_PROMPT)]
note_message is Chapter 2’s function: it puts the note inside <note> tags after the prompt. The scripted model matches the start of that message, QUIZ_PROMPT, exactly; a live model would accept any wording.
parse_quiz_reply(messages, *, client):
Make one call:
client.chat.completions.parse(model=MODEL, messages=messages, response_format=QuizSet). Pass nothing else.message = response.choices[0].message.If
message.parsed is None, raiseValueError(f"The model refused: {message.refusal}").Otherwise return
message.parsed.
QuizSet is imported from quiz_models.py at the top of the file.
Run it
Run python client_calls.py. The demonstration asks for a quiz about notes/03-lists.md and prints it from the QuizSet your function returned:
quiz = parse_quiz_reply(quiz_messages(note), client=client)
for question in quiz.questions:
print(f" {question.question} Answer: {question.choices[question.answer_index]}")
Then it calls the supplied last_request(), which fetches GET /v1/scripted/last-request: the last request body your sandbox sent to llm-api, exactly as the service received it. It prints that body’s response_format, which you never wrote:
A quiz through the parse helper:
parse_quiz_reply returned a QuizSet with 3 questions:
For scores = [90, 72, 85], what is scores[-1]? Answer: 85
What does scores.append(60) return? Answer: None
Which line makes an independent copy of scores? Answer: backup = scores.copy()
The response_format the parse helper sent:
{
"type": "json_schema",
"json_schema": {
"schema": {
"$defs": {
"QuizQuestion": {
"properties": {
"question": {
"title": "Question",
"type": "string"
},
"choices": {
"items": {
"type": "string"
},
"title": "Choices",
"type": "array"
},
"answer_index": {
"title": "Answer Index",
"type": "integer"
}
},
"required": [
"question",
"choices",
"answer_index"
],
"title": "QuizQuestion",
"type": "object",
"additionalProperties": false
}
},
"properties": {
"questions": {
"items": {
"$ref": "#/$defs/QuizQuestion"
},
"title": "Questions",
"type": "array"
}
},
"required": [
"questions"
],
"title": "QuizSet",
"type": "object",
"additionalProperties": false
},
"name": "QuizSet",
"strict": true
}
}
That is the json_schema request format. Its json_schema holds three things: the schema’s name, taken from the class name; "strict": true; and the schema itself, which is QuizSet.model_json_schema() from Chapter 4 with one addition. Every object in it now has "additionalProperties": false, which strict mode requires and which forbids keys the model does not declare. QuizQuestion sits under $defs, and "$ref": "#/$defs/QuizQuestion" points to it from the questions list.
After that, python client_calls.py goes on to Lesson 5’s demonstration, which uses only finished code, and stops at chat_turn, the project’s first function.
parse gets a reply in which answer_index is "abc". What does your code receive?
Why does quiz_messages use plain QUIZ_SYSTEM instead of Chapter 4's quiz_system_message(), which added the schema?
Task
In client_calls.py, replace the unfinished bodies of quiz_messages(note_text) and parse_quiz_reply(messages, *, client).
quiz_messagesreturns[{"role": "system", "content": QUIZ_SYSTEM}, note_message(note_text, QUIZ_PROMPT)], with no schema in the system message.parse_quiz_replymakes one call,client.chat.completions.parse(model=MODEL, messages=messages, response_format=QuizSet), and readsmessage = response.choices[0].message.If
message.parsedisNone, raiseValueError(f"The model refused: {message.refusal}"). Otherwise returnmessage.parsed.Catch nothing: a
ValidationErrorfromparsereaches the caller.
Run python client_calls.py to see the quiz and the request format the helper sent, then check your work.