Structured Output · practice
Try Again Once When the Output Is Invalid
Validation tells you when a reply is wrong and exactly how. A program that gives up at that point wastes a call that was nearly right. The usual fix is to ask again, once, and tell the model what was wrong.
Show the model its reply and what was wrong
A model remembers nothing between requests, as Chapter 2 showed: every request carries the whole conversation. So to ask for a correction, you send the conversation as it now stands: the original messages, the model’s own invalid reply as an assistant message, and a user message that says what was wrong with it.
[
{"role": "system", "content": "You write multiple-choice quiz questions ..."},
{"role": "user", "content": "Write three multiple-choice questions about the note below. ..."},
{"role": "assistant", "content": '{"questions": [{"question": "What does len() return?", ..., "answer_index": "abc"}]}'},
{"role": "user", "content": "Your last reply was invalid: questions.0.answer_index: Input should be a valid integer, unable to parse string as an integer"},
]
The model now sees what it wrote and the precise problem, which is exactly what error_lines produces. That is why error_lines gives short lines rather than the whole printed error: the printed form has a documentation link and details meant for programmers, and every character you send is a token you pay for.
The feedback starts with INVALID_PREFIX from prompts.py, Your last reply was invalid:. A live model understands any wording. The scripted model only checks that the last user message starts with that text, and then sends the corrected reply for the prompt before it. SCRIPTED_MODEL.md describes this rule and the prompts that are wrong on purpose the first time.
Build retry_request
retry_request(body, bad_text, error) takes the body that was sent, the reply text that failed, and the ValidationError, and returns the body to send next:
Copy the body with
body.copy(), somodel,response_formatand any other key come along.Give the copy a new
messageslist:body["messages"]plus two more messages, the assistant’sbad_textand the user’s feedback.The feedback is
f"{INVALID_PREFIX} "followed by the lines oferror_lines(error)joined with"; ".
body.copy() copies the dictionary, but the copy’s "messages" still refers to the same list as the original’s. Appending to retry["messages"] would change the caller’s list too. body["messages"] + [...] builds a new list and leaves the original alone, which is what you want: the caller may still need its body as it was.
Why only once
If the corrected reply is also invalid, stop. Two failures in a row say the request is the problem, not bad luck: the prompt is unclear, the shape is too hard for this model, or the note confuses it. A loop that keeps asking can run for a long time on a problem that more asking will not fix.
It also costs. Each retry sends the whole conversation again, now longer by the bad reply and the feedback, so a retry costs more input tokens than the first try. One retry fixes the occasional slip. More than that is paying to hide a problem you should look at.
Run it
Run python structured.py. The demonstration sends give me broken json, a prompt the scripted model answers wrongly on purpose the first time, with json_object. It shows the reply, builds the retry with your retry_request, prints the feedback it carries, sends it and validates the answer:
bad_text = answer_text(send_body(broken))
try:
parse_quiz(bad_text)
except ValidationError as error:
retry = retry_request(broken, bad_text, error)
A reply that fails validation, and one retry:
First reply: {"questions": [{"question": "What does len() return?", "choices": ["The number of items", "The last item", "The first item"], "answer_index": "abc"}]}
Feedback sent: Your last reply was invalid: questions.0.answer_index: Input should be a valid integer, unable to parse string as an integer
After the retry: QuizSet(questions=[QuizQuestion(question='What does len() return?', choices=['The number of items', 'The last item', 'The first item'], answer_index=0)])
The quiz for notes/03-lists.md:
Run stopped at the next function to build: make_quiz is built in Lesson 9.
The first reply had "abc" where an integer belongs. The retry carried that reply and one line of feedback, and the second reply validated as a QuizSet. The project puts these steps together.
Why does the retry include the invalid reply as an assistant message?
The retry's reply is invalid too. What should the program do?
Task
In structured.py, replace the unfinished body of retry_request(body, bad_text, error).
Return a copy of body whose "messages" is a new list: the messages of body, then
{"role": "assistant", "content": bad_text}, the reply that failed;{"role": "user", "content": ...}, where the content isf"{INVALID_PREFIX} "followed by the lines oferror_lines(error)joined with"; ".
Every other key of body, such as model and response_format, stays as it was. The caller’s body and its messages list must not change. Run python structured.py to see an invalid reply corrected after one retry, then check your work.