Structured Output · practice
Ask for JSON and Check It
Lesson 1 ended with a plan: ask the model for JSON, because a program can check JSON. This lesson sends that request and checks what comes back, with the tools you used on JSON documents in Python in Practice.
Ask for a JSON object
The chat endpoint takes one more field for this, response_format:
"response_format": {"type": "json_object"}
With it, the service asks the model for a reply whose content is a single JSON object. The content still arrives as a string, inside the same response document as always. It is JSON text, so your code decodes it with json.loads, as Python in Practice did with documents from web services. MODEL_SERVICE.md describes the field.
OpenAI’s API, and many services that follow it, add one condition: the word JSON has to appear somewhere in the messages, or they reject the request. The quiz’s system message, QUIZ_SYSTEM in prompts.py, says it, and it also says what the object should hold:
QUIZ_SYSTEM = (
"You write multiple-choice quiz questions for a Python course. Reply with "
"JSON only: an object with a questions list. Each question has question, "
"choices (three or four strings) and answer_index (the position of the "
"correct choice, counting from 0)."
)
QUIZ_PROMPT = "Write three multiple-choice questions about the note below."
The request is about a course note, so the user message is QUIZ_PROMPT followed by the note. Chapter 2’s note_message(note_text, question) builds exactly that: the question, a blank line, and the note between <note> tags. It is imported at the top of structured.py.
The scripted model recognizes only these exact texts. A live model would accept any wording that asks for the same thing, but the scripted model matches the prompt, so use the constants as they are.
Build quiz_request
quiz_request(note_text) returns the whole request body:
{
"model": MODEL,
"messages": [
{"role": "system", "content": QUIZ_SYSTEM},
note_message(note_text, QUIZ_PROMPT),
],
"response_format": {"type": "json_object"},
}
It only builds the body. Sending it is send_body’s job, from Chapter 1.
What json_object promises, and what it does not
json_object promises that the content is valid JSON. It says nothing about which keys the object has, or what is inside them. A model that replies {"quiz": "Sorry, no questions today"} has kept that promise. So before a program uses a reply, it checks the shape, just as Python in Practice checked documents from a web service before using them: is it a dictionary, does it have the key, is the value a list?
import json
data = json.loads('{"questions": [{"question": "Q?", "choices": ["a", "b"]}]}')
print(isinstance(data, dict), "questions" in data)
print(isinstance(data["questions"], list))
print("answer_index" in data["questions"][0])
This one passes the first three checks and fails the last: its question has no answer_index.
Build read_quiz_json
read_quiz_json(text) decodes the reply and checks it, from the outside in:
data = json.loads(text). Text that is not JSON makesjson.loadsraisejson.JSONDecodeError. That error is a kind ofValueError, so let it through unchanged.If
datais not a dictionary, or has noquestionskey, raiseValueErrorwith a message that namesquestions.If
data["questions"]is not a list, raiseValueError.For each item in the list: if it is not a dictionary, raise
ValueError; if it lacksquestion,choicesoranswer_index, raiseValueErrornaming the missing key.Return the list.
Check the outside before the inside. "questions" in data means something different for a list or a string, and data["questions"] fails with a TypeError or KeyError that tells the caller nothing about the reply. A ValueError whose message names the missing key, such as ValueError("A question has no answer_index key."), says what was wrong with the reply.
Run it
Run python structured.py. After Lesson 1’s output, the demonstration reads notes/03-lists.md, sends quiz_request for it with send_body, and prints the question of each item that read_quiz_json returns:
body = quiz_request(note)
for item in read_quiz_json(answer_text(send_body(body))):
print(f" {item['question']}")
The questions in a JSON reply, checked by read_quiz_json:
For scores = [90, 72, 85], what is scores[-1]?
What does scores.append(60) return?
Which line makes an independent copy of scores?
read_quiz_json given a number instead of text:
TypeError: the JSON object must be str, bytes or bytearray, not int
A QuizQuestion from sample_question:
Run stopped at the next function to build: sample_question is built in Lesson 4.
These are the same three questions as Lesson 1’s text quiz, now read from named keys instead of found by line layout. The scripted model returns this quiz for any note: it does not read the note, and SCRIPTED_MODEL.md says so. A live model writes questions about the note it is given. The two lines about a number are for Lesson 3.
What the checks do not cover
Your checks make sure every key is there. They do not look at the values. This reply passes read_quiz_json without complaint:
'{"questions": [{"question": "Q?", "choices": ["a", "b"], "answer_index": "abc"}]}'
A quiz app that later does item["choices"][item["answer_index"]] fails there with a TypeError, far from the reply that caused it. Checking every value’s type with more isinstance calls would work, but the checks grow with every field. The next lessons give you a way to state the types once and have them checked for you.
A request asks for {"type": "json_object"}. What can your code rely on in the reply's content?
Why does read_quiz_json check that the decoded value is a dictionary before it checks for the questions key?
Task
In structured.py, replace the unfinished bodies of quiz_request(note_text) and read_quiz_json(text).
quiz_request returns this dictionary:
"model":MODEL"messages": a list of two messages:{"role": "system", "content": QUIZ_SYSTEM}, thennote_message(note_text, QUIZ_PROMPT)"response_format":{"type": "json_object"}
read_quiz_json decodes text with json.loads and returns its questions list. Raise ValueError with a message naming what is wrong when:
the decoded value is not a dictionary, or it has no
questionskey (namequestionsin the message);questionsis not a list;an item of the list is not a dictionary, or it lacks one of the keys
question,choicesandanswer_index(name the missing key).
Text that is not JSON makes json.loads raise json.JSONDecodeError, which is a kind of ValueError, so let it through. Run python structured.py to see the questions from a JSON reply, then check your work.