0%

Chapter 4

Structured Output

Text Is Hard for a Program to Use

So far every reply has been text for a person to read. Your study chat prints it, and you read it. This chapter is about replies that a program reads instead: a quiz app that needs the question, the choices and the right answer as separate values, so it can show them, check an answer and keep score. Getting those values out of a model’s reply reliably is called structured output.

Your project for this chapter

Open structured.py in the project editor. It starts with these imports:

import json

import requests
from pydantic import ValidationError

from conversation import note_message
from llm_basics import MODEL, answer_text, send_body
from prompts import INVALID_PREFIX, QUIZ_PROMPT, QUIZ_SYSTEM, QUIZ_TEXT_PROMPT
from quiz_models import QuizQuestion, QuizSet

conversation.py is Chapter 2’s finished code and llm_basics.py is Chapter 1’s. As before, checking your work uses its own copies of them, so an edit there cannot change a grade, but it can stop the demonstration from working. Two files are new. prompts.py holds the fixed prompt texts this chapter sends. quiz_models.py holds the quiz’s data models; you read it in Lesson 4 and finish it in Lesson 5. pydantic, the library those models come from, is already installed in your sandbox, and requirements.txt lists the version.

Every request in this chapter waits for the whole reply, the way Chapters 1 and 2 did. A program cannot do anything with half a quiz, so streaming would gain nothing here.

Below the imports is one finished function, numbered_lines, and ten unfinished ones, one or two per lesson. python structured.py runs the file, and the demonstration at the bottom calls each function as soon as it exists.

Ask for a quiz as text

prompts.py holds the first prompt:

QUIZ_TEXT_PROMPT = "Write three quiz questions about Python lists."

In the Shell, run python structured.py now, before you change anything. The demonstration sends that prompt as the only message, prints the reply, and then prints what numbered_lines finds in it:

A quiz written as text:
1. What does scores[-1] give for scores = [90, 72, 85]?
a) 90  b) 72  c) 85
Answer: c

2. What does scores.append(60) return?
a) The new list  b) None  c) 60
Answer: b

3. Which line makes an independent copy of scores?
a) backup = scores.copy()  b) backup = scores  c) backup = scores.append(0)
Answer: a
What numbered_lines finds in it:
['What does scores[-1] give for scores = [90, 72, 85]?', 'What does scores.append(60) return?', 'Which line makes an independent copy of scores?']
The questions in a JSON reply, checked by read_quiz_json:
Run stopped at the next function to build: quiz_request is built in Lesson 2.

The scripted model answers this prompt with three numbered questions, a line of lettered choices under each, and an Answer: line. numbered_lines is a quick parser written for that layout:

def numbered_lines(text):
    found = []
    for line in text.split("\n"):
        if line[:1].isdigit() and line[1:3] == ". ":
            found.append(line[3:].strip())
    return found

It splits the reply into lines and keeps each line that starts with a digit followed by a period and a space, without those three characters. On the scripted reply it finds exactly the three questions. Taking the choices and answers out of the other lines would be a few more string methods of the same kind.

The same prompt to a live model

On 2026-11-08 the same prompt went to openai/gpt-oss-20b, the model behind pl-live, as the only message. This is its reply, unedited:

**Quiz: Python Lists**

1. **Basic Operations**  
   Which of the following statements correctly appends the value `42` to the end of the list `my_list`?  
   a) `my_list + 42`  
   b) `my_list.append(42)`  
   c) `my_list.insert(0, 42)`  
   d) `my_list = my_list + [42]`

2. **Indexing and Slicing**  
   Given `numbers = [10, 20, 30, 40, 50]`, what will `numbers[1:4]` return?  
   a) `[20, 30, 40]`  
   b) `[10, 20, 30]`  
   c) `[20, 30, 40, 50]`  
   d) `[30, 40, 50]`

3. **List Comprehension**  
   Which list comprehension will produce the list `[1, 4, 9, 16]`?  
   a) `[i*i for i in range(1, 5)]`  
   b) `[i**2 for i in [1, 2, 3, 4]]`  
   c) `[(i-1)**2 for i in range(1, 5)]`  
   d) Both a) and b) are correct.

A person reads that easily. It is arguably a better quiz. Now give it to numbered_lines:

['**Basic Operations**', '**Indexing and Slicing**', '**List Comprehension**']

The live model numbered a topic heading for each question, in bold Markdown (the ** around it), and put the question itself on the next line, indented. So the lines that start with a number are the headings. The choices are indented too, they have four letters instead of three, and there is no answer anywhere in the reply. A second request with the same prompt, a few seconds later, gave another layout again: bold **A)** choices, a *Question:* label on its own line, and a code block in the third question.

A wrong answer is worse than a crash

Look at what numbered_lines did with the live reply. It did not raise an error. It returned a list of three strings, exactly the type the rest of a program expects. A quiz app built on it would show three headings as questions, offer no choices and have no answers to check against, and nothing would tell you why. An error stops a program at the line that went wrong. A plausible wrong value travels on and does its damage somewhere else.

The scripted model always uses one layout, which is why the parser worked on it. A live model writes text for people, and it may lay out the same content differently every time. A person adapts without noticing. A program does exactly what it was written to do.

What a program needs

A program needs a reply with a shape it can rely on: the same named parts every time, each holding the kind of value it expects, and a way to tell when a reply does not have that shape. You know a format built for exactly this from Python in Practice: JSON. A JSON object has named keys, its values have types such as strings, numbers and lists, and json.loads either decodes it or raises an error.

Writing a cleverer text parser is a losing race. The next live reply can use a layout you did not plan for, and the parser will again return something wrong without complaint. The fix is on the other side: ask the model for a format that a program can check.

The plan for this chapter

Lesson 2 asks the model for JSON and checks its keys by hand. Lessons 3 to 5 introduce the tools to describe the shape you want: type hints, and a pydantic model, a class that lists each field and its type. Lesson 6 uses that pydantic model to check a reply in one step, Lesson 7 sends the shape along with the request, and Lesson 8 asks again, once, when a reply does not fit. The project turns any course note into a checked quiz and prints it.

What did numbered_lines return for the live reply, and why?

What does a program need from a reply that a person reading it does not?

Would a stricter text parser, one that also understands bold headings and indented questions, fix the problem?