0%

An LLM Call Is an HTTP Request · capstone

Capstone project: Project: Ask One Question from the Command Line

You have four tested functions in llm_basics.py: send_body posts a request, ask_model turns a question into a chat request, answer_text reads the answer, and call_cost prices the call. This project puts them behind a command you can run from the :

$ python ask.py "What is a Python list?"
A list is an ordered, changeable collection of values. You write one with square brackets, like [1, 2, 3], and you can add, remove and change items after creating it.
Tokens: 6 in, 42 out. Cost at live prices: $0.00000564

Every part of it is something you have built or used before. The work is putting the parts together so the command behaves well, including when the model service does not answer.

What ask.py does

Create ask.py next to llm_basics.py and import what you need from it. The file defines one function, main(argv=None, *, session=None):

  1. Parse the with argparse. There is one positional argument, question. Passing argv to parse_args(argv) means a check can call main(["What is a Python list?"]) directly, while a real run with argv=None reads the terminal’s arguments, as in Python in Practice.

  2. Call ask_model with the question, passing session=session along so a check can hand in a fake.

  3. Print the answer with answer_text.

  4. Print one more line with the token counts from the response’s usage and the cost from call_cost at LIVE_PRICES, in exactly this form, with the cost to eight decimal places:

    Tokens: 6 in, 42 out. Cost at live prices: $0.00000564
    
  5. Return 0.

The module ends with the line that runs main only when the file is executed as a program:

if __name__ == "__main__":
    raise SystemExit(main())

raise SystemExit(main()) turns the number main returns into the program’s exit status. Importing ask from another module or a check runs nothing, because __name__ is then "ask".

When the request fails

The model service can be down, slow, or answer with an error status. All of these surface as exceptions from requests: a connection error, a timeout, or the HTTPError that raise_for_status() raises inside send_body. They share one parent class, requests.RequestException, so one except clause around the ask_model call catches all of them.

When that happens, print one line to standard error instead of a traceback:

Model request failed: <the error>

ask.py needs import requests for the exception class and import sys for print(..., file=sys.stderr), which writes to standard error. Then return 1. Catch only requests.RequestException, and only around the request. A KeyError or TypeError from a mistake in your own code should still stop the program with its traceback, because hiding it would make the mistake much harder to find.

Run it

In the terminal, run the command with a question in quotes, so the shell passes it as one argument:

python ask.py "What is a Python list?"
python ask.py "What does len() return?"
python ask.py "Tell me a joke."

The first two are prepared questions. The third gets the scripted echo, so the answer is four lines long and the token counts change with it. Run python ask.py with no question and argparse prints a usage message and exits with status 2 before any request is made.

If you get stuck, ask Monty for a hint about one step, such as reading the token counts out of usage, without asking for the whole file.

Why does main catch requests.RequestException rather than Exception?

Task

Create ask.py with main(argv=None, *, session=None):

  • Parse one positional argument, question, with argparse, from argv.

  • Call ask_model(question, session=session) from llm_basics.

  • Print the answer from answer_text, then print Tokens: {I} in, {O} out. Cost at live prices: ${cost:.8f}, where I and O are the response’s prompt_tokens and completion_tokens and cost is call_cost of its usage at LIVE_PRICES. Return 0.

  • If the request raises requests.RequestException, print Model request failed: {error} to standard error and return 1. Catch nothing else.

  • End the file with if __name__ == "__main__": raise SystemExit(main()), so importing ask runs nothing.

Try python ask.py "What is a Python list?" in the , then check your work.