Streaming · practice
When a Stream Breaks Halfway
At the end of Lesson 6, the demonstration asked the scripted model to cut a stream short, and your stream_reply printed three pieces before requests raised an error. That is what this lesson fixes. A stream is a connection that stays open for the whole reply, and connections break: a network drops, a service restarts, a proxy gives up. Your reader has to notice, and it has to say so clearly.
The danger is a reply that looks finished
Suppose a stream breaks after A list is an ordered collection that can hold . The text is already on the screen. If stream_reply returned it like any other reply, the caller would have no way to tell it from a complete answer. A chat would store it in the history as what the model said, and a program that acts on the reply would act on half of it. Here the half sentence is obviously cut off, but the break can just as well fall after a full sentence, where the missing part is invisible.
So the rule for this lesson: stream_reply returns only a complete reply. This chapter uses a strict rule for that: a reply counts as complete only when "[DONE]" arrived. The finish chunk shows the model stopped writing, but [DONE] is the one signal that the service sent everything. A reader that insists on it can never mistake a cut-off stream for a finished one. Anything else is an error.
Two ways a stream ends early
The connection breaks. The service had promised more data and the connection closed without it. requests notices this while you are reading lines and raises requests.exceptions.ChunkedEncodingError from inside the for loop over response.iter_lines(). That is what Lesson 6’s demonstration showed. ChunkedEncodingError is one of the requests.RequestException family you caught in Python in Practice. The error’s name comes from HTTP, not from the stream’s JSON chunks. A streamed response is sent in blocks that end with a marker block, and requests raises this error when the connection closes before the marker arrives. Here the word “chunked” is about blocks of the HTTP response, not about your data: documents.
The lines run out. The connection closes cleanly but [DONE] never came, so the loop simply ends. requests sees nothing wrong, because as far as the connection is concerned the response is over. Only your code knows that the stream format promised an end marker.
The scripted model can produce the first case on demand, with the request header X-Scripted-Scenario: broken-stream. It cannot produce the second, so the demonstration shows that one with a saved stream, described below.
The error to raise: StreamBroken
errors.py supplies an exception for this:
from errors import StreamBroken
You create it with the text that did arrive, raise StreamBroken(text), and the code that catches it reads that text back from the exception’s partial_text attribute:
try:
reply = stream_reply(response)
except StreamBroken as error:
print(f"StreamBroken. The partial text was {error.partial_text!r}")
This is how you read attributes of existing exceptions in Python in Practice. You do not need to read or change errors.py: you only raise its exception and read its attribute. Passing the partial text along lets a caller decide what to do with it, for example show it marked as incomplete, or log it, without anyone mistaking it for a whole reply.
Change stream_reply
Keep everything from Lesson 6 and add two things:
Remember whether
"[DONE]"arrived. Setfinished = Falsebefore the loop, and set it toTruejust before thebreakat"[DONE]". After the loop, iffinishedis stillFalse, the lines ran out early: print one newline and raiseStreamBroken("".join(pieces)).Turn a broken connection into the same error. Put the
forloop inside atryblock, and addexcept requests.exceptions.ChunkedEncodingError:that prints one newline and raisesStreamBroken("".join(pieces)).
A complete stream still prints its newline and returns the joined pieces, as before. The newline before each raise ends the half-printed line, so whatever the caller prints next starts on a fresh line.
Catch only ChunkedEncodingError. The try block covers json.loads and your own functions as well, and a mistake there, such as a chunk that is not valid JSON or a typo in a key, is not a broken connection. If except Exception: turned those into StreamBroken, a bug in your code would be reported as a network problem, and the real error would be hidden. Narrow except clauses, as in Python in Practice, let everything else reach the caller unchanged.
A stream that goes silent in the middle for longer than the read timeout raises a different error, requests.exceptions.ConnectionError, not ChunkedEncodingError. stream_reply does not catch it, so it reaches the caller without the partial text. Handling it comes later, with retries.
Run it
Run python streaming.py. The broken stream comes first. The demonstration sends Explain Python lists in detail. with the scenario header:
response = open_stream(long_body, headers={"X-Scripted-Scenario": "broken-stream"})
The scripted model sends the comment, the first chunk and three pieces, then closes the connection. This time stream_reply turns that into StreamBroken, and the demonstration catches it and prints the partial text with !r, so you can see the space at its end:
A stream that breaks halfway:
A list is an ordered collection that can hold
StreamBroken. The partial text was 'A list is an ordered collection that can hold '
A stream that ends without [DONE]:
Lists keep their order,
StreamBroken. The partial text was 'Lists keep their order, '
The streaming study chat:
You: What does len() return?
Run stopped at the next function to build: chat_turn is built in Lesson 8.
The second case is the saved stream. The demonstration builds a response from two data: lines that it wrote out itself, with no [DONE] after them:
saved = requests.Response()
saved.raw = io.BytesIO(
b'data: {"choices": [{"index": 0, "delta": {"content": "Lists keep "}}]}\n\n'
b'data: {"choices": [{"index": 0, "delta": {"content": "their order, "}}]}\n\n'
)
requests.Response() creates an empty response object, and setting its raw to those bytes gives iter_lines() something to read, exactly as if a service had sent them and then closed the connection cleanly. Your stream_reply reads both pieces, runs out of lines, and raises StreamBroken because finished is still False.
A stream breaks after half the reply. Why should stream_reply raise StreamBroken rather than return the half it received?
A stream sends its pieces, the finish chunk and the usage chunk, and then the connection closes cleanly without data: [DONE]. By this chapter's rule, what should your stream_reply do?
stream_reply catches requests.exceptions.ChunkedEncodingError and nothing else. One chunk in a stream is not valid JSON, so json.loads raises an error. What happens?
Task
In streaming.py, change stream_reply(response) so it never returns part of a reply as if it were whole.
If
requests.exceptions.ChunkedEncodingErroris raised while you read the lines, print one newline and raiseStreamBroken("".join(pieces)).If the lines run out before
"[DONE]"arrives, do the same.A stream that reaches
"[DONE]"still prints the pieces, then one newline, and returns the whole reply.Catch no other exception: other errors must reach the caller unchanged.
StreamBroken is already imported from errors.py. Run python streaming.py to see both kinds of broken stream reported, then check your work.