0%

An LLM Call Is an HTTP Request · practice

A Request Can Carry a Body

A model call is an HTTP request. The service is llm-api, its base URL is http://llm-api:8080/v1, and you talk to it with requests, the same way you talked to the practice API in Python in Practice. Open llm_basics.py in the project editor. It has the constants you need and five unfinished functions. In this lesson you build the first two.

Start with a request you already know

The service can tell you which models it offers. A GET to /models returns this document:

{"object": "list", "data": [{"id": "pl-scripted", "object": "model", "owned_by": "python-land"}, {"id": "pl-live", "object": "model", "owned_by": "python-land"}]}

The model names are the id values inside the data list. list_models() fetches that document and returns the ids in order, so for this document it returns ["pl-scripted", "pl-live"]:

def list_models(*, session=None):
    http = requests if session is None else session
    response = http.get(f"{BASE_URL}/models", timeout=TIMEOUT)
    response.raise_for_status()
    ids = []
    for model in response.json()["data"]:
        ids.append(model["id"])
    return ids

Every line here is something you have written before. session=None is the seam from Python in Practice Chapter 11: normally the function uses requests, and a check can hand it a fake session that records the request instead of sending it. timeout=TIMEOUT bounds the wait, using the constant at the top of the file, (3.05, 10): up to 3.05 seconds to connect and up to 10 seconds for each wait while the reply arrives. raise_for_status() turns an error status into an exception before any JSON is read.

Every function in this course that talks to llm-api follows this pattern, so it is worth typing it out once rather than pasting it.

A POST carries a body

A GET puts its whole question in the URL. That works for “which models are there?”, but a model call needs to send the model a prompt, often a long one, plus settings. That goes in the request’s body: data sent along with the request, after the URL and headers. A request that carries a body to be processed is usually a POST.

The chat endpoint, /chat/completions, takes a JSON body. JSON is the text format you decoded with .json() in Python in Practice; here you send it the other way. A small body looks like this as a Python dictionary:

body = {"model": "pl-scripted", "messages": [{"role": "user", "content": "Hello"}]}

requests turns a dictionary into a JSON body for you when you pass it as json=:

response = http.post(f"{BASE_URL}/chat/completions", json=body, timeout=TIMEOUT)

json=body does two jobs: it converts the dictionary to JSON text, and it adds a Content-Type: application/json header so the service knows how to read it. The status check and .json() work exactly as they do after a GET.

send_body(body) is that POST, wrapped in the same pattern as list_models: choose requests or the injected session, POST the body with json=body and timeout=TIMEOUT, raise for an error status, and return the decoded JSON reply unchanged.

See what the model received

When pl-scripted does not recognize a prompt, it answers with an echo: a short report of what it received. Hello is not one of its prepared questions, so the body above gets an echo back.

You run your code in a , as in Python in Practice. Open the Shell tab next to Project and choose Start Bash terminal. Type python llm_basics.py and press Enter. The terminal keeps running until you close it, so after each edit you can run the same command again; the up arrow brings it back. Check my work is a button in the Shell tab’s toolbar.

Once both functions work, run python llm_basics.py. The demonstration at the bottom of the file lists the models, prints the body it sent, then prints the decoded reply:

Models: ['pl-scripted', 'pl-live']
Sent: {"model": "pl-scripted", "messages": [{"role": "user", "content": "Hello"}]}
Received:
{
  "id": "chatcmpl-scripted-91b01cf4fa744410ab443153",
  "object": "chat.completion",
  "created": 1794134874,
  "model": "pl-scripted",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "[pl-scripted echo] No scripted reply matches this prompt, so here is what I received.\nMessages: 1\nSystem message: (none)\nLast user message: Hello"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 2,
    "completion_tokens": 37,
    "total_tokens": 39
  }
}

Your id and created will differ; they change on every call. Find the line that starts with "content". Each \n in it is a line break, so the echo says: one message, no system message, and the last user message was Hello. That matches the message you sent: one message, no system message, and the text Hello. Lesson 5 shows you how to pull that text out with Python. For now, notice that you can check what reached the model without guessing.

After the output you will see a line saying Run stopped at the next function to build. That is expected: the demonstration calls each function you have built so far and stops at the first one you have not. You will see it again in the next lessons, with a later function named each time.

If the command prints Could not reach llm-api instead, the service is down or restarting. Try again in a few minutes. Checking your work does not need it.

What does json=body do in http.post(url, json=body, timeout=TIMEOUT)?

Task

In llm_basics.py, replace the unfinished bodies of list_models and send_body.

  • list_models(*, session=None) makes one GET to f"{BASE_URL}/models" with timeout=TIMEOUT, calls raise_for_status(), and returns the id of every item in the document’s data list, in order.

  • send_body(body, *, session=None) makes one POST to f"{BASE_URL}/chat/completions" with json=body and timeout=TIMEOUT, calls raise_for_status(), and returns the decoded JSON reply unchanged.

Both use requests when session is None and the given session otherwise. In the Shell tab, run python llm_basics.py to see the echo, then check your work.