Using the OpenAI Client · practice
The Same Call Through the Client
In Chapter 1 one question took two functions. ask_model built the body and posted it, and answer_text dug the answer out of the decoded reply. With the client it is one call and one line to read the answer. The body that reaches llm-api is the same.
The same request, written two ways
This is the Chapter 1 way, with everything send_body and answer_text did written out:
body = {"model": MODEL, "messages": [{"role": "user", "content": question}]}
response = requests.post(f"{BASE_URL}/chat/completions", json=body, timeout=TIMEOUT)
response.raise_for_status()
document = response.json()
answer = document["choices"][0]["message"]["content"]
And this is the same request through the client:
response = client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": question}])
answer = response.choices[0].message.content
client.chat.completions.create follows the path of the endpoint, /chat/completions. Each keyword argument becomes one field of the request body, so model=MODEL and messages=[...] send exactly the body Chapter 1 built. The client adds the base URL, the timeout and the status check.
Attributes instead of keys
The client decodes the reply for you and gives it back as an object. The values are the same as in the JSON document; you reach them with dots instead of square brackets:
| Chapter 1, a decoded dictionary | Chapter 5, the client’s object |
|---|---|
document["choices"][0]["message"]["content"] | response.choices[0].message.content |
document["choices"][0]["finish_reason"] | response.choices[0].finish_reason |
document["usage"]["total_tokens"] | response.usage.total_tokens |
choices is still a list, so [0] stays. Every key in between becomes an attribute, the way you read error.response on an exception in Python in Practice and question.choices on a pydantic model in Chapter 4. The objects the client returns are in fact pydantic models, so response.model_dump() gives you the whole reply back as a dictionary if you ever want to print or save it.
Reading a key that does not exist raised KeyError. Reading an attribute that does not exist raises AttributeError. And writing response["choices"] out of habit raises TypeError, because the object cannot be indexed with a string.
Errors come from the client
Because the client checks the status itself, a failed request never reaches your code as a response. It raises one of the client’s own errors, all in the openai package:
a
500from the service raisesopenai.InternalServerError;a service that cannot be reached raises
openai.APIConnectionError;a wait longer than the timeout raises
openai.APITimeoutError, a kind ofAPIConnectionError.
ask catches none of them. As with send_body, the caller decides what a failure means, and Chapter 9 builds the code that does.
Build ask
ask(question, *, client) in client_calls.py:
Make one call:
client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": question}]). Pass nothing else.Read
response.choices[0].message.content.Return it, or
""when it isNone, asanswer_textdid.
The client parameter comes after the *, so callers must name it: ask("What is a list?", client=client). It has no default. That is the testing seam from Chapters 1 to 4, with one change: there, session=None meant “use requests”; here every caller passes a client. The demonstration passes make_client(). Checking your work passes a fake client that records each call and returns a prepared reply, so ask must use the client it is given and never build its own.
Run it
Run python client_calls.py. The demonstration asks one question:
print(ask("What is a Python list?", client=client))
One question through the client:
A list is an ordered, changeable collection of values. You write one with square brackets, like [1, 2, 3], and you can add, remove and change items after creating it.
A streamed reply:
Run stopped at the next function to build: stream_chat is built in Lesson 3.
That is the scripted model’s reply for this question, the same one Chapter 1’s ask_model got.
Try it live (optional)
If your account has live access, send the same question to the live model. Paste this in show_progress directly below the line print(ask("What is a Python list?", client=client)), and run python client_calls.py:
response = client.chat.completions.create(model="pl-live", messages=[{"role": "user", "content": "What is a Python list?"}])
print(response.choices[0].message.content)
print(response.usage.total_tokens, "tokens")
Text you send to pl-live leaves Python Land, so send nothing private. If your account has no live access you get openai.PermissionDeniedError, the client’s error for a 403. openai.RateLimitError is a 429: you reached the limit for now. openai.APITimeoutError is a normal outcome for a live model, which can think for longer than the course’s timeout. Remove the lines again when you are done.
Chapter 1 read the finish reason as document["choices"][0]["finish_reason"]. How do you read it from the client's reply?
The service answers ask's request with status 500. What reaches the code that called ask?
Task
In client_calls.py, replace the unfinished body of ask(question, *, client).
Make one call:
client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": question}]), and pass nothing else.Return
response.choices[0].message.content, or""when it isNone.Catch nothing: errors from the client reach the caller.
Use the client you are given; do not build one inside ask. Run python client_calls.py to see the answer, then check your work.