An LLM Call Is an HTTP Request · practice
A Request Can Carry a Body
A model call is an HTTP request. The service is llm-api, its base URL is http://llm-api:8080/v1, and you talk to it with requests, the same way you talked to the practice API in Python in Practice. Open llm_basics.py in the project editor. It has the constants you need and five unfinished functions. In this lesson you build the first two.
Start with a request you already know
The service can tell you which models it offers. A GET to /models returns this document:
{"object": "list", "data": [{"id": "pl-scripted", "object": "model", "owned_by": "python-land"}, {"id": "pl-live", "object": "model", "owned_by": "python-land"}]}
The model names are the id values inside the data list. list_models() fetches that document and returns the ids in order, so for this document it returns ["pl-scripted", "pl-live"]:
def list_models(*, session=None):
http = requests if session is None else session
response = http.get(f"{BASE_URL}/models", timeout=TIMEOUT)
response.raise_for_status()
ids = []
for model in response.json()["data"]:
ids.append(model["id"])
return ids
Every line here is something you have written before. session=None is the seam from Python in Practice Chapter 11: normally the function uses requests, and a check can hand it a fake session that records the request instead of sending it. timeout=TIMEOUT bounds the wait, using the constant at the top of the file, (3.05, 10): up to 3.05 seconds to connect and up to 10 seconds for each wait while the reply arrives. raise_for_status() turns an error status into an exception before any JSON is read.
Every function in this course that talks to llm-api follows this pattern, so it is worth typing it out once rather than pasting it.
A POST carries a body
A GET puts its whole question in the URL. That works for “which models are there?”, but a model call needs to send the model a prompt, often a long one, plus settings. That goes in the request’s body: data sent along with the request, after the URL and headers. A request that carries a body to be processed is usually a POST.
The chat endpoint, /chat/completions, takes a JSON body. JSON is the text format you decoded with .json() in Python in Practice; here you send it the other way. A small body looks like this as a Python dictionary:
body = {"model": "pl-scripted", "messages": [{"role": "user", "content": "Hello"}]}
requests turns a dictionary into a JSON body for you when you pass it as json=:
response = http.post(f"{BASE_URL}/chat/completions", json=body, timeout=TIMEOUT)
json=body does two jobs: it converts the dictionary to JSON text, and it adds a Content-Type: application/json header so the service knows how to read it. The status check and .json() work exactly as they do after a GET.
send_body(body) is that POST, wrapped in the same pattern as list_models: choose requests or the injected session, POST the body with json=body and timeout=TIMEOUT, raise for an error status, and return the decoded JSON reply unchanged.
See what the model received
When pl-scripted does not recognize a prompt, it answers with an echo: a short report of what it received. Hello is not one of its prepared questions, so the body above gets an echo back.
You run your code in a python llm_basics.py and press Enter. The terminal keeps running until you close it, so after each edit you can run the same command again; the up arrow brings it back. Check my work is a button in the Shell tab’s toolbar.
Once both functions work, run python llm_basics.py. The demonstration at the bottom of the file lists the models, prints the body it sent, then prints the decoded reply:
Models: ['pl-scripted', 'pl-live']
Sent: {"model": "pl-scripted", "messages": [{"role": "user", "content": "Hello"}]}
Received:
{
"id": "chatcmpl-scripted-91b01cf4fa744410ab443153",
"object": "chat.completion",
"created": 1794134874,
"model": "pl-scripted",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "[pl-scripted echo] No scripted reply matches this prompt, so here is what I received.\nMessages: 1\nSystem message: (none)\nLast user message: Hello"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 2,
"completion_tokens": 37,
"total_tokens": 39
}
}
Your id and created will differ; they change on every call. Find the line that starts with "content". Each \n in it is a line break, so the echo says: one message, no system message, and the last user message was Hello. That matches the message you sent: one message, no system message, and the text Hello. Lesson 5 shows you how to pull that text out with Python. For now, notice that you can check what reached the model without guessing.
After the output you will see a line saying Run stopped at the next function to build. That is expected: the demonstration calls each function you have built so far and stops at the first one you have not. You will see it again in the next lessons, with a later function named each time.
If the command prints Could not reach llm-api instead, the service is down or restarting. Try again in a few minutes. Checking your work does not need it.
What does json=body do in http.post(url, json=body, timeout=TIMEOUT)?
Task
In llm_basics.py, replace the unfinished bodies of list_models and send_body.
list_models(*, session=None)makes one GET tof"{BASE_URL}/models"withtimeout=TIMEOUT, callsraise_for_status(), and returns theidof every item in the document’sdatalist, in order.send_body(body, *, session=None)makes one POST tof"{BASE_URL}/chat/completions"withjson=bodyandtimeout=TIMEOUT, callsraise_for_status(), and returns the decoded JSON reply unchanged.
Both use requests when session is None and the given session otherwise. In the Shell tab, run python llm_basics.py to see the echo, then check your work.