An LLM Call Is an HTTP Request · capstone
Capstone project: Project: Ask One Question from the Command Line
You have four tested functions in llm_basics.py: send_body posts a request, ask_model turns a question into a chat request, answer_text reads the answer, and call_cost prices the call. This project puts them behind a command you can run from the
$ python ask.py "What is a Python list?"
A list is an ordered, changeable collection of values. You write one with square brackets, like [1, 2, 3], and you can add, remove and change items after creating it.
Tokens: 6 in, 42 out. Cost at live prices: $0.00000564
Every part of it is something you have built or used before. The work is putting the parts together so the command behaves well, including when the model service does not answer.
What ask.py does
Create ask.py next to llm_basics.py and import what you need from it. The file defines one function, main(argv=None, *, session=None):
Parse the
with argparse. There is one positional argument,question. Passingargvtoparse_args(argv)means a check can callmain(["What is a Python list?"])directly, while a real run withargv=Nonereads the terminal’s arguments, as in Python in Practice.Call
ask_modelwith the question, passingsession=sessionalong so a check can hand in a fake.Print the answer with
answer_text.Print one more line with the token counts from the response’s
usageand the cost fromcall_costatLIVE_PRICES, in exactly this form, with the cost to eight decimal places:Tokens: 6 in, 42 out. Cost at live prices: $0.00000564Return
0.
The module ends with the line that runs main only when the file is executed as a program:
if __name__ == "__main__":
raise SystemExit(main())
raise SystemExit(main()) turns the number main returns into the program’s exit status. Importing ask from another module or a check runs nothing, because __name__ is then "ask".
When the request fails
The model service can be down, slow, or answer with an error status. All of these surface as exceptions from requests: a connection error, a timeout, or the HTTPError that raise_for_status() raises inside send_body. They share one parent class, requests.RequestException, so one except clause around the ask_model call catches all of them.
When that happens, print one line to standard error instead of a traceback:
Model request failed: <the error>
ask.py needs import requests for the exception class and import sys for print(..., file=sys.stderr), which writes to standard error. Then return 1. Catch only requests.RequestException, and only around the request. A KeyError or TypeError from a mistake in your own code should still stop the program with its traceback, because hiding it would make the mistake much harder to find.
Run it
In the terminal, run the command with a question in quotes, so the shell passes it as one argument:
python ask.py "What is a Python list?"
python ask.py "What does len() return?"
python ask.py "Tell me a joke."
The first two are prepared questions. The third gets the scripted echo, so the answer is four lines long and the token counts change with it. Run python ask.py with no question and argparse prints a usage message and exits with status 2 before any request is made.
If you get stuck, ask Monty for a hint about one step, such as reading the token counts out of usage, without asking for the whole file.
Why does main catch requests.RequestException rather than Exception?
Task
Create ask.py with main(argv=None, *, session=None):
Parse one positional argument,
question, withargparse, fromargv.Call
ask_model(question, session=session)fromllm_basics.Print the answer from
answer_text, then printTokens: {I} in, {O} out. Cost at live prices: ${cost:.8f}, whereIandOare the response’sprompt_tokensandcompletion_tokensandcostiscall_costof itsusageatLIVE_PRICES. Return0.If the request raises
requests.RequestException, printModel request failed: {error}to standard error and return1. Catch nothing else.End the file with
if __name__ == "__main__": raise SystemExit(main()), so importingaskruns nothing.
Try python ask.py "What is a Python list?" in the