0%

An LLM Call Is an HTTP Request · practice

Read the Answer Out of the Response

The response document from Lesson 4 holds the answer, but a program that prints the whole document is not much use. This lesson pulls out the one string you usually want: the text the model wrote.

Walk down to the answer

Here is the document again, trimmed to the parts that lead to the answer:

response = {
    "model": "pl-scripted",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "A list is an ordered, changeable collection of values.",
            },
            "finish_reason": "stop",
        }
    ],
}

Read it from the outside in, one step per level:

choices = response["choices"]
first = choices[0]
message = first["message"]
print(message["content"])

choices is a list because some APIs can return several alternative answers to one request. This course always asks for one, so the answer is in choices[0]. Inside it, message is the model’s reply written as a chat message, with the role assistant, and content is its text.

Once each step makes sense, you can write the same walk in one expression:

print(response["choices"][0]["message"]["content"])

Why the answer stopped

Next to message sits finish_reason, which says why the model stopped writing:

  • "stop": the model finished its answer.

  • "length": the model ran into its token limit and was cut off mid-answer.

A cut-off answer is incomplete, and it can even be empty. Some live models think through a problem in tokens you never see before they start writing the answer. If that thinking uses up the whole limit, the response has finish_reason "length" and content that is empty or null. pl-scripted ignores the limit, so you will only meet this with a live model. Chapter 9 handles cut-off answers properly. For now, know that the text you read might not be the whole answer, and that finish_reason is how you find out.

When there is no text at all

content is not always a string. In Chapter 6, a model that wants to call one of your functions sends a reply whose content is null in JSON, which arrives in Python as None. Code that expects a string, such as print(len(answer)), then fails.

So answer_text(response) returns the content string unchanged, or "" when content is None. Your code can then always treat the answer as text, and an empty string still means “the model wrote nothing”.

Use plain indexing for the walk. If a document arrives without choices at all, the lookup raises KeyError. Let it. A document that shape is not a model reply, and an error that names the missing key is easier to diagnose than an empty answer that hides the problem.

Run python llm_basics.py after you finish. The demonstration now also prints:

Answer: A list is an ordered, changeable collection of values. You write one with square brackets, like [1, 2, 3], and you can add, remove and change items after creating it.
Finish reason: stop

A live model's response has finish_reason "length" and content "". What happened?

Task

In llm_basics.py, replace the unfinished body of answer_text(response).

Return response["choices"][0]["message"]["content"] unchanged when it is a string, and "" when it is None. Do not catch errors: a document without choices should raise KeyError.

Run python llm_basics.py to see the answer on its own, then check your work.