0%

Streaming · practice

Print as It Arrives, Keep the Whole Answer

You have every step for one line: decode it, sort it with parse_line, and take its text with chunk_text. This lesson puts them in a loop that shows the reply as it arrives and also returns the whole reply as one string, because a program usually needs both. The person reads the pieces as they come, and the program keeps the whole answer, for example to add it to a history.

Printing pieces on one line

print ends what it prints with a newline. For pieces of one sentence that is wrong: every piece would land on its own line. The end argument replaces the newline with whatever you give it, and end="" means nothing at all:

for piece in ["Lists keep ", "their order, ", "so the first item stays first."]:
    print(piece, end="")
print()

The pieces join into one line because each piece already ends with the space it needs. The print() after the loop prints only the newline that ends the reply, so whatever is printed next starts on a fresh line.

There is one more thing to know about printing without newlines. Python does not always write text to the screen the moment you print it. When it prints to a , it collects text in a buffer and writes the buffer out when a line is complete. A piece without a newline can sit in that buffer until a later newline arrives, which for a streamed reply means at the end, and the reader sees nothing until then. flush=True tells print to write the buffer out immediately:

print(piece, end="", flush=True)

With it, each piece appears as soon as it is printed. Use both arguments for every piece.

To see the difference once you have built stream_reply below, remove flush=True, then run python streaming.py again. The long reply should appear all at once when it ends instead of piece by piece. Put the argument back afterward. Checking your work does not look at flush=True, but a reader in a terminal does.

Keep the pieces too

Printing shows the reply, but then the text is gone from your program. Keep each piece in a list as well, and join the list at the end. You met str.join in Python Fundamentals: "".join(pieces) puts the pieces together with nothing between them, which is right here because the spaces are already inside the pieces.

Build stream_reply

stream_reply(response) takes an open response from open_stream and does this:

  1. Start an empty list for the pieces.

  2. For each line from response.iter_lines(): decode it with .decode("utf-8") and pass it to parse_line.

    • When the result is None, skip to the next line.

    • When it is "[DONE]", stop the loop with break. Anything after the end marker is ignored.

    • Otherwise decode the JSON with json.loads, and get the piece with chunk_text. If the piece is not empty, print it with print(piece, end="", flush=True) and add it to the list.

  3. After the loop, print one newline, then return "".join(pieces).

Check for "[DONE]" before you call json.loads. [DONE] is not JSON, and json.loads("[DONE]") raises an error.

Notice what the loop does not do: it does not stop at the chunk with a finish_reason. The usage chunk comes after that, and a reader that stopped early would miss the token counts Chapter 9 relies on. Some live models also put the last piece of text in the same chunk as the finish_reason. Reading on until "[DONE]" handles both.

Run it

Run python streaming.py. After the earlier sections, the demonstration opens a stream for Explain Python lists in detail. and hands it to your function:

reply = stream_reply(open_stream(long_body))
print(f"stream_reply returned {len(reply)} characters.")

The reply appears piece by piece, as it did in Lesson 1’s preview, and this time your own code prints it:

A long reply, streamed by stream_reply:
A list is an ordered collection that can hold any mix of values. You create one with square brackets, such as scores = [90, 72, 85]. Each item has a position, starting at 0, so scores[0] is 90 and scores[-1] is the last item. Slicing takes a range: scores[0:2] gives a new list with the first two items. Lists are mutable, which means you can change them after creating them: scores.append(60) adds an item to the end, scores.remove(72) removes the first matching value, and scores[1] = 75 replaces an item in place. Because a list is changed in place, two names that refer to the same list both see the change. Use list.copy() when you need an independent list. len(scores) tells you how many items it holds.
stream_reply returned 709 characters.
A stream that breaks halfway:
A list is an ordered collection that can hold 
requests raised ChunkedEncodingError: Response ended prematurely
Run stopped: stream_reply does not handle a broken stream yet. Lesson 7 changes that.

The returned string has all 709 characters of the reply: what was printed and what was returned are the same text.

The last four lines are the next lesson’s problem. The demonstration asked the scripted model to cut the connection after three pieces. Your function printed those pieces, then requests raised an error in the middle of the loop. Lesson 7 decides what stream_reply should do about that.

What does this print?

for piece in ["Sets ", "drop ", "duplicates."]:
    print(piece)

Your stream reader prints with print(piece, end="") and no flush=True. In the terminal, the reply appears all at once at the end instead of piece by piece. Why?

Why must stream_reply check for "[DONE]" before calling json.loads?

Task

In streaming.py, replace the unfinished body of stream_reply(response).

  • Read the lines with response.iter_lines(), decode each with .decode("utf-8"), and pass it to parse_line.

  • Skip a line when parse_line returns None. Stop reading at "[DONE]", and ignore anything after it.

  • For every other line, get the piece with chunk_text(json.loads(text)). If the piece is not empty, print it with print(piece, end="", flush=True) and keep it.

  • After the loop, print one newline and return all the pieces joined into one string.

Run python streaming.py to watch your function stream a long reply, then check your work.