0%

When Things Go Wrong · practice

Notice a Cut-off Answer

In Lesson 1’s demonstration, the length line was the only one that raised nothing. The call succeeded, and the response looked like any other. But its finish_reason was "length", which means the model stopped because it reached max_tokens, not because it was done.

Incomplete, even when it looks fine

Explain Python lists in detail. under the length scenario gets this content:

A list is an ordered collection that can

Here the sentence obviously stops in the middle. Often it is less obvious: a cut-off list of steps can end after a complete step, and cut-off JSON can be missing its last question or its closing brace. A program that only reads content passes such a reply on as if it were the whole answer.

A reasoning model, such as the one behind pl-live, can be cut off before it writes anything. It spends part of max_tokens thinking, and if the thinking uses up the limit, the visible content is "". The empty-length scenario imitates that.

So a program checks response.choices[0].finish_reason before it trusts a reply. "stop" means the model finished, and "tool_calls" means it is asking for tools. Both are complete. "length" is not.

ReplyCutOff

errors.py supplies an error for this case. It works like Chapter 3’s StreamBroken: you raise it with the text that did arrive, and whoever catches it reads partial_text:

raise ReplyCutOff(text)
except ReplyCutOff as error:
    print(error.partial_text)

check_finished(response) raises ReplyCutOff with the reply’s content when the finish reason is "length". A reply that asked for tools, or one with no text, has content None, so pass "" instead in that case: partial_text is always a string. For any other finish reason, check_finished returns nothing.

A cut-off reply belongs to Lesson 1’s “fix the request” group. Sending the same request again gets the same cut-off reply, so it is not retried. The fix is a higher max_tokens or a shorter request, which is a decision for the person or program that built the request.

The parse helper does the same

Chapter 5’s parse_quiz_reply sent the quiz through the client’s parse helper. The helper checks finish_reason itself: on "length" it raises openai.LengthFinishReasonError before it tries to validate anything. check_finished gives every other call the same protection, with an error of the course’s own.

Run it

When check_finished is written, run python chapter9_demo.py. The demonstration sends Explain Python lists in detail. through call_once under length and under empty-length, passes each response to check_finished, and then asks Chapter 5’s parse_quiz_reply for a quiz about notes/03-lists.md under length:

A reply under length:
  ReplyCutOff. The partial text was 'A list is an ordered collection that can'
A reply under empty-length:
  ReplyCutOff. The partial text was ''
A quiz through the parse helper under length:
  LengthFinishReasonError
The same question until the budget refuses it:
Run stopped at the next function to build: check_budget is built in Lesson 6.

Each cut-off reply is now an error with a name, instead of a short answer that looks complete.

check_finished raised ReplyCutOff. Why is the same request not simply sent again?

Under empty-length, the reply's content is "" and its finish_reason is "length". What should check_finished do?

Task

In reliable.py, write check_finished(response). When response.choices[0].finish_reason is "length", raise ReplyCutOff with the reply’s content, or with "" when the content is None. For any other finish reason, return nothing. Use the supplied ReplyCutOff from errors.py. Run python chapter9_demo.py to see the cut-off replies caught, then check your work.