0%

Streaming · practice

Recognize the Lines That Matter

Your stream now arrives as decoded lines: a comment, blank lines, data: lines with JSON, and data: [DONE]. Only some of them carry the reply. This lesson writes the rule that sorts them, as one function that looks at a single line and says what to do with it.

Three answers for every line

parse_line(line) takes one decoded line and returns one of three things:

  • None: skip this line. That covers a blank line, a comment (a line that starts with :), and any other line that does not start with data:. Server-sent events allow a few other kinds of line, such as event: ... or id: .... The scripted model never sends them, but other services can, and a reader that only wants the data skips them.

  • "[DONE]": stop. The stream is complete.

  • Otherwise, the text after data:, with the spaces around it removed. For a chunk, that is its JSON text, ready for json.loads in the next lesson.

Returning None for “nothing here” is a common Python habit. The caller can then write if text is None: continue and move on to the next line.

Checking how a string starts

Two string methods do most of the work. str.startswith tells you whether a string begins with some text:

line = 'data: {"choices": []}'
print(line.startswith("data:"))
print(": keep-alive".startswith("data:"))
print("".startswith("data:"))

An empty string does not start with data:, so blank lines fall into the skip case without a separate check.

str.removeprefix returns the string without the text at its start, if it is there, and leaves the rest alone:

line = 'data: {"choices": []}'
print("[" + line.removeprefix("data:") + "]")
print("[" + line.removeprefix("data:").strip() + "]")

The square brackets show where each string starts and ends. removeprefix leaves the space that followed the colon, and .strip() removes it. Like every string method, both return a new string and change nothing.

Two details from the format

The space after data: is optional. The scripted model writes data: {...} with a space, but the server-sent events format allows data:{...} as well, and some services send it. Check for data: without the space, then strip whatever spaces follow. Slicing off six characters with line[6:] works only when the space is there.

Remove the prefix only from the start. A reply can contain the text data: itself, for example a model explaining this very format, so it can appear inside the JSON too. line.replace("data:", "") would remove every copy, including the ones inside the reply. removeprefix only looks at the start. strip("data:") is wrong in another way: it treats its argument as a set of characters to remove from both ends, not as one prefix.

The end marker needs no special case. data: [DONE] starts with data:, and the text after it, stripped, is "[DONE]". You can still check for it explicitly if that reads more clearly to you.

The format also lets one event spread over several data: lines, which a reader joins with newlines. The scripted model and OpenAI-style services send one JSON document per line, so parse_line treats each data: line as one event.

Build parse_line

Write parse_line(line) with the rule above. Then run python streaming.py. The demonstration passes every decoded line of the Say hello in a short sentence. stream through it and prints what comes back with repr(), which shows a string in quotes, so you can tell None from a string:

What parse_line makes of each line:
None
None
'{"id": "chatcmpl-scripted-ff5ff3bedeae40fb85e5e4dc", "object": "chat.completion.chunk", "created": 1794141686, "model": "pl-scripted", "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": null}]}'
None
'{"id": "chatcmpl-scripted-ff5ff3bedeae40fb85e5e4dc", "object": "chat.completion.chunk", "created": 1794141686, "model": "pl-scripted", "choices": [{"index": 0, "delta": {"content": "Hello there, it "}, "finish_reason": null}]}'
None
'{"id": "chatcmpl-scripted-ff5ff3bedeae40fb85e5e4dc", "object": "chat.completion.chunk", "created": 1794141686, "model": "pl-scripted", "choices": [{"index": 0, "delta": {"content": "is nice to "}, "finish_reason": null}]}'
None
'{"id": "chatcmpl-scripted-ff5ff3bedeae40fb85e5e4dc", "object": "chat.completion.chunk", "created": 1794141686, "model": "pl-scripted", "choices": [{"index": 0, "delta": {"content": "meet you."}, "finish_reason": null}]}'
None
'{"id": "chatcmpl-scripted-ff5ff3bedeae40fb85e5e4dc", "object": "chat.completion.chunk", "created": 1794141686, "model": "pl-scripted", "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]}'
None
'{"id": "chatcmpl-scripted-ff5ff3bedeae40fb85e5e4dc", "object": "chat.completion.chunk", "created": 1794141686, "model": "pl-scripted", "choices": [], "usage": {"prompt_tokens": 8, "completion_tokens": 9, "total_tokens": 17}}'
None
'[DONE]'
None
The text of each chunk, in brackets:
Run stopped at the next function to build: chunk_text is built in Lesson 5.

The first None is the comment, and every other None is a blank line. Each JSON text has lost its data: prefix. The demonstration printed the line after '[DONE]' too, the last blank line, only because it prints every line it gets. A real reader stops at [DONE], as yours will in Lesson 6.

The service sends only a few kinds of line, so try the other cases in the before you check your work. parse_line sends nothing over the network, so you can call it directly. Start Python with python and try a line with no space after the colon, and one that is not data at all:

from streaming import parse_line
print(parse_line('data:{"choices": []}'))
print(parse_line("event: message"))

The first should print the JSON text and the second None.

What should parse_line(": data: hello") return?

line is 'data: {"text": "data: arrives in lines"}'. Which expression gives exactly the JSON text, for this line and for every other data line?

Task

In streaming.py, replace the unfinished body of parse_line(line). It receives one decoded line, a str.

  • Return None for a line that does not start with data:. That includes blank lines, comments (lines that start with :) and lines such as event: message.

  • Return "[DONE]" for the end marker, data: [DONE].

  • For any other data: line, return the text after data: with the spaces around it removed. The space after the colon may be missing, and the text itself can contain data:, which must stay as it is.

Run python streaming.py to see what your function makes of each line, then check your work.