0%

Using the OpenAI Client · capstone

Capstone project: Project: Move the Study Assistant onto the Client

The study assistant has two features that talk to the model: the streaming chat from Chapter 3 and the quiz from Chapter 4. Both still use requests. This project moves them onto the client, using the functions you built in Lessons 2 to 4:

client = make_client()
ask("What is a Python list?", client=client)
reply, history = chat_turn([], "Explain Python lists in detail.", client=client)  # streams
quiz = make_quiz(note_text, client=client)  # a QuizSet, through the parse helper

ask is done already. The other two are short, because the hard parts are in stream_chat and parse_quiz_reply.

chat_turn

Chapter 3’s chat_turn prepared the messages, streamed the reply with open_stream and stream_reply, and returned the reply with a new history. Only the middle changes:

StepChapter 3This project
Prepareprepare_messages(history, question)the same
Streamstream_reply(open_stream(...))stream_chat(messages, client=client)
Return(reply, add_turn(messages[:-1], question, reply))the same

chat_turn(history, question, *, client):

  1. messages = prepare_messages(history, question).

  2. reply = stream_chat(messages, client=client). This prints the reply as it arrives.

  3. Return (reply, add_turn(messages[:-1], question, reply)).

chat_turn prints nothing itself and catches nothing. When the stream breaks, the StreamBroken from stream_chat goes straight to the caller, no history is returned, and the half reply never enters one, exactly as in Chapter 3.

make_quiz

Chapter 4’s make_quiz sent the quiz request, validated the reply, and on a ValidationError sent one retry. The parse helper now does the sending and the validating, so make_quiz keeps only the retry.

make_quiz(note_text, *, client):

  1. messages = quiz_messages(note_text).

  2. Return parse_quiz_reply(messages, client=client).

  3. If that raises ValidationError, build one feedback message,

    {"role": "user", "content": f"{INVALID_PREFIX} " + "; ".join(error_lines(error))}
    

    call parse_quiz_reply once more with messages + [feedback], and return its quiz.

error_lines is Chapter 4’s function, imported from structured.py, so the feedback reads the same as in Chapter 4: one short line per problem, such as questions.0.answer_index: Input should be a valid integer, unable to parse string as an integer.

Three things are not retried, and reach the caller unchanged:

  • A second ValidationError. One retry, as in Chapter 4: if the model fails twice, the request needs fixing, and every retry costs tokens.

  • A refusal. parse_quiz_reply raises a plain ValueError for it. Catch the error by its name, except ValidationError as error:. ValidationError is a kind of ValueError, so except ValueError: would catch the refusal too. It would then reach error_lines, which fails with AttributeError because a plain ValueError has no errors() method, and the caller would get that crash instead of the model’s explanation. The reverse is not true: except ValidationError does not catch a plain ValueError.

  • Errors from the client, such as openai.InternalServerError. Those are Chapter 9’s business.

What this retry cannot do

Chapter 4’s retry showed the model its own invalid reply as an assistant message, then said what was wrong with it. This one cannot: parse raises before it returns anything, so the reply’s text never reaches your code. A live model gets the complaint without the reply it refers to, which makes the feedback harder to act on.

The trade is usually a good one. With strict json_schema, a provider that supports it makes the model’s output follow the schema, so an invalid reply is rare. The retry mostly guards against models and providers without strict support. The scripted model shows the retry path anyway, because the prompt give me broken json answers invalidly once whatever the request format, and it decides by how the newest user message starts, not by what came before it.

Run it

Run python client_calls.py. The demonstration holds a two-turn chat, makes a quiz for the lists note, and then makes a quiz whose first reply is invalid. For that one it prints the last message of the last request your sandbox sent, using last_request() from Lesson 4, so you can see the feedback your make_quiz added:

The study chat on the client:
You: What does len() return?
len() returns the number of items in a sequence or collection, such as the characters in a string or the elements in a list.
You: Explain Python lists in detail.
A list is an ordered collection that can hold any mix of values. You create one with square brackets, such as scores = [90, 72, 85]. Each item has a position, starting at 0, so scores[0] is 90 and scores[-1] is the last item. Slicing takes a range: scores[0:2] gives a new list with the first two items. Lists are mutable, which means you can change them after creating them: scores.append(60) adds an item to the end, scores.remove(72) removes the first matching value, and scores[1] = 75 replaces an item in place. Because a list is changed in place, two names that refer to the same list both see the change. Use list.copy() when you need an independent list. len(scores) tells you how many items it holds.
The history now holds 5 messages.
A quiz from make_quiz:
3 questions. The first: For scores = [90, 72, 85], what is scores[-1]?
A quiz whose first reply is invalid:
The second request ended with: Your last reply was invalid: questions.0.answer_index: Input should be a valid integer, unable to parse string as an integer
1 question: What does len() return?

The chat replies came from stream_chat inside your chat_turn, as they arrived. The history holds the study system message and two whole turns.

The demonstration shows the happy paths and one retry. Before you check your work, think through what it does not show: a stream that breaks halfway, a second invalid reply, and a refusal. If you get stuck on one, ask Monty for a hint about that case without asking for the whole function.

Where this goes next

The study assistant now talks to the model only through the client, and every function that calls the model takes the client as an argument. Chapter 6 uses the same client to let the model ask your program to run a function, such as looking up a note.

make_quiz catches ValueError instead of ValidationError. The model refuses to write the quiz. What happens?

Chapter 4's retry sent the invalid reply back as an assistant message before the feedback. Why does this project's retry send only the feedback?

Task

In client_calls.py, replace the unfinished bodies of chat_turn(history, question, *, client) and make_quiz(note_text, *, client).

chat_turn:

  • messages = prepare_messages(history, question), then reply = stream_chat(messages, client=client).

  • Return (reply, add_turn(messages[:-1], question, reply)).

  • Print nothing yourself and catch nothing: a StreamBroken reaches the caller and no history is returned.

make_quiz:

  • messages = quiz_messages(note_text), and return parse_quiz_reply(messages, client=client).

  • On ValidationError, caught by that name, call parse_quiz_reply once more with messages + [{"role": "user", "content": f"{INVALID_PREFIX} " + "; ".join(error_lines(error))}] and return its quiz.

  • A second ValidationError, a refusal’s ValueError and errors from the client all reach the caller, with no further call.

Run python client_calls.py to see the chat and both quizzes, then check your work.