0%

Using the OpenAI Client · practice

The Same Call Through the Client

In Chapter 1 one question took two functions. ask_model built the body and posted it, and answer_text dug the answer out of the decoded reply. With the client it is one call and one line to read the answer. The body that reaches llm-api is the same.

The same request, written two ways

This is the Chapter 1 way, with everything send_body and answer_text did written out:

body = {"model": MODEL, "messages": [{"role": "user", "content": question}]}
response = requests.post(f"{BASE_URL}/chat/completions", json=body, timeout=TIMEOUT)
response.raise_for_status()
document = response.json()
answer = document["choices"][0]["message"]["content"]

And this is the same request through the client:

response = client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": question}])
answer = response.choices[0].message.content

client.chat.completions.create follows the path of the endpoint, /chat/completions. Each keyword argument becomes one field of the request body, so model=MODEL and messages=[...] send exactly the body Chapter 1 built. The client adds the base URL, the timeout and the status check.

Attributes instead of keys

The client decodes the reply for you and gives it back as an object. The values are the same as in the JSON document; you reach them with dots instead of square brackets:

Chapter 1, a decoded dictionaryChapter 5, the client’s object
document["choices"][0]["message"]["content"]response.choices[0].message.content
document["choices"][0]["finish_reason"]response.choices[0].finish_reason
document["usage"]["total_tokens"]response.usage.total_tokens

choices is still a list, so [0] stays. Every key in between becomes an attribute, the way you read error.response on an exception in Python in Practice and question.choices on a pydantic model in Chapter 4. The objects the client returns are in fact pydantic models, so response.model_dump() gives you the whole reply back as a dictionary if you ever want to print or save it.

Reading a key that does not exist raised KeyError. Reading an attribute that does not exist raises AttributeError. And writing response["choices"] out of habit raises TypeError, because the object cannot be indexed with a string.

Errors come from the client

Because the client checks the status itself, a failed request never reaches your code as a response. It raises one of the client’s own errors, all in the openai package:

  • a 500 from the service raises openai.InternalServerError;

  • a service that cannot be reached raises openai.APIConnectionError;

  • a wait longer than the timeout raises openai.APITimeoutError, a kind of APIConnectionError.

ask catches none of them. As with send_body, the caller decides what a failure means, and Chapter 9 builds the code that does.

Build ask

ask(question, *, client) in client_calls.py:

  1. Make one call: client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": question}]). Pass nothing else.

  2. Read response.choices[0].message.content.

  3. Return it, or "" when it is None, as answer_text did.

The client parameter comes after the *, so callers must name it: ask("What is a list?", client=client). It has no default. That is the testing seam from Chapters 1 to 4, with one change: there, session=None meant “use requests”; here every caller passes a client. The demonstration passes make_client(). Checking your work passes a fake client that records each call and returns a prepared reply, so ask must use the client it is given and never build its own.

Run it

Run python client_calls.py. The demonstration asks one question:

print(ask("What is a Python list?", client=client))
One question through the client:
A list is an ordered, changeable collection of values. You write one with square brackets, like [1, 2, 3], and you can add, remove and change items after creating it.
A streamed reply:
Run stopped at the next function to build: stream_chat is built in Lesson 3.

That is the scripted model’s reply for this question, the same one Chapter 1’s ask_model got.

Try it live (optional)

If your account has live access, send the same question to the live model. Paste this in show_progress directly below the line print(ask("What is a Python list?", client=client)), and run python client_calls.py:

response = client.chat.completions.create(model="pl-live", messages=[{"role": "user", "content": "What is a Python list?"}])
print(response.choices[0].message.content)
print(response.usage.total_tokens, "tokens")

Text you send to pl-live leaves Python Land, so send nothing private. If your account has no live access you get openai.PermissionDeniedError, the client’s error for a 403. openai.RateLimitError is a 429: you reached the limit for now. openai.APITimeoutError is a normal outcome for a live model, which can think for longer than the course’s timeout. Remove the lines again when you are done.

Chapter 1 read the finish reason as document["choices"][0]["finish_reason"]. How do you read it from the client's reply?

The service answers ask's request with status 500. What reaches the code that called ask?

Task

In client_calls.py, replace the unfinished body of ask(question, *, client).

  • Make one call: client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": question}]), and pass nothing else.

  • Return response.choices[0].message.content, or "" when it is None.

  • Catch nothing: errors from the client reach the caller.

Use the client you are given; do not build one inside ask. Run python client_calls.py to see the answer, then check your work.