Capstone · practice
Failures, Budget and Exit Codes
The three commands work, and each one either returns its text or raises. Nothing yet decides what the person sees when something fails, or what exit code the program ends with. That is the job of the program’s boundary, the code between the main(argv=None) grown up: the same shape, with every failure the study assistant can meet sorted into one of a few exit codes.
You write three functions, then the line that runs the program.
From cause to exit code
Since Chapter 3 you have written code that raises a specific error for each kind of failure. Here they all meet, and each gets a code:
1, the run failed. The agent stopped without an answer (
ValueErrorfromask_command), an answer or a quiz was still invalid after its retry (ValidationError, a kind ofValueError, orask’s ownValueError), a quiz reply had no text (ValueError), a reply was cut off atmax_tokens(ReplyCutOff), a stream broke halfway (StreamBroken), or a model error was not worth retrying or kept failing aftercall_model’s retries (openai.APIError, which also covers a service that cannot be reached).3, the budget ran out.
BudgetExceeded, from any call of the run,ask’s retry included. This is the person’s own limit, set with--budget, so it has its own code: a script that runs the study assistant can tell “give it more budget” from “it failed”.4, the note does not exist.
FileNotFoundErrorfromquiz_command. The person mistyped an id.2, the command line was wrong.
argparsehandles this before any of your code runs, as Lesson 1 showed.
Anything else, such as a TypeError or a KeyError, is a bug in the program. It is not caught, so Python prints its traceback, which is what you want when the program itself is wrong.
The picture sorts every outcome. The command line is checked first, so a usage error never reaches run_command. After that, each except clause catches its kind of failure and gives it a code, and anything none of them names ends in a traceback.

error_line
The person sees one line on standard error: Error: and a short description. For most errors the description is the first line of str(error): The budget ran out before agent step 2., The reply was cut off at the max_tokens limit., or an openai error’s text, which is one line. Take it with str(error).split("\n")[0].
A pydantic ValidationError is different. The first line of its text is only a heading, such as 1 validation error for CitedAnswer, which says nothing about what was wrong. Chapter 4’s error_lines(error) gives one short line per problem, so for a ValidationError, error_line returns the first of those, such as (whole reply): Invalid JSON: expected ident at line 1 column 2.
run_command
run_command(args, *, client, budget, sleep, stdin) runs one command and returns its exit code:
try:
if args.command == "ask":
print(ask_command(args.question, client=client, budget=budget, sleep=sleep))
...
return 0
except BudgetExceeded as error:
print(f"Error: {error_line(error)}", file=sys.stderr)
return 3
Inside try, check args.command and call that command: print what ask_command(args.question, ...) returns for "ask", print what quiz_command(args.note_id, args.count, ...) returns for "quiz", and call chat_command(..., stdin=stdin) for "chat", which prints as it goes. Pass client, budget and sleep to each, then return 0.
Then one except clause per code, in this order, each printing f"Error: {error_line(error)}" with file=sys.stderr and returning its code: BudgetExceeded returns 3, FileNotFoundError returns 4, and (ValueError, ReplyCutOff, StreamBroken, openai.APIError) returns 1. Nothing goes to standard output on a failure: the commands return their text only when they succeed, so the print never ran. For chat, whatever streamed before a break is already on the screen, and stays there.
print(..., file=sys.stderr) looks up sys.stderr when it runs, so when a grader captures standard error, your line lands in the capture.
main
main(argv=None, *, client=None, notes_dir=NOTES_DIR, stdin=None, sleep=time.sleep):
args = build_parser().parse_args(argv). Withargv=None,argparsereads the real command line. A usage error leaves throughargparse’s ownSystemExit(2)with its usage and error lines, as in Python in Practice; do not catch it.use_settings(notes_dir), so the course modules use this program’s model and notes folder.If
clientisNone,client = build_client(). IfstdinisNone,stdin = sys.stdin.budget = {"limit": args.budget, "used": 0}. This is the run’s one budget: the agent’s steps,ask’s retry, a quiz and its retry, and every turn of a chat all draw on this dictionary.Return
run_command(args, client=client, budget=budget, sleep=sleep, stdin=stdin).
main takes client, notes_dir, stdin and sleep so that tests and graders can run the whole program with a fake client, a folder of their own notes, typed input and no waiting. It does not take the output streams: the commands print, and a streamed reply is printed piece by piece inside chat_turn, so tests capture what is printed instead, with contextlib.redirect_stdout and redirect_stderr, as Python in Practice’s graders did.
Finally, replace the comment at the end of the file with the guard:
if __name__ == "__main__":
raise SystemExit(main())
SystemExit(code) ends the program with that exit code, so whatever main returns becomes the program’s. Importing the file runs nothing, which is how stages.py and the graders use it.
Run it
When everything is written, run python stages.py. Lesson 6’s block calls main once for each exit code. Each call prints the argument list and then the exit code; between them is what the program wrote, with standard error and standard output on the same screen:
Lesson 6: main and its exit codes
--- main(['ask', 'How do I make an independent copy of a list?'])
Use scores.copy() or list(scores). Both build a new list, so a change made through one name no longer shows through the other.
Sources: 03-lists#4
exit code 0
--- main(['--budget', '1000', 'ask', 'How do I make an independent copy of a list?'])
Error: The budget ran out before agent step 2.
exit code 3
--- main(['quiz', '99-magic'])
Error: There is no note called 99-magic.
exit code 4
--- main(['quiz', '03-lists', '--count', '9'])
usage: study_assistant.py quiz [-h] [--count N] NOTE_ID
study_assistant.py quiz: error: argument --count: must be a whole number from 1 to 5
exit code 2
Under the tool-loop scenario:
--- main(['ask', 'How do I make an independent copy of a list?'])
Error: The agent stopped without an answer (stuck).
exit code 1
Under the broken-stream scenario:
--- main(['chat'])
A list is an ordered collection that can hold
Error: The stream stopped before the reply was complete.
exit code 1
Under the rate-limit:2 scenario:
--- main(['ask', 'How do I make an independent copy of a list?'])
(waiting 1.0 s)
(waiting 1.0 s)
Use scores.copy() or list(scores). Both build a new list, so a change made through one name no longer shows through the other.
Sources: 03-lists#4
exit code 0
With --budget 1000, step 1’s estimate was 146 + 800 = 946 tokens, which fit. The first step used 159, leaving 841, and step 2’s request also carried the 763-character search result, so its estimate was 1,136.75, which did not. Under tool-loop the model asks for the same search again, and Chapter 7’s loop detection stops the agent. Under broken-stream the chat printed the start of a reply, then failed. Under rate-limit:2, stages.py passes Chapter 9’s sleep_and_say, which says how long it waits, and call_model waited out two 429 replies; stages.py first resets the scripted model’s counter. stages.py then goes on to Lesson 8’s eval report.
If llm-api is down, python stages.py stops at Lesson 2’s block with Could not reach llm-api, because that block calls run_agent directly. main itself handles it: in the python study_assistant.py ask "..." prints Error: Connection error. and ends with exit code 1, like any other model error.
Once the guard is there, try the commands in the terminal. echo $? shows the exit code of the command before it:
$ python study_assistant.py --budget 1000 ask "How do I make an independent copy of a list?"
Error: The budget ran out before agent step 2.
$ echo $?
3
$ python study_assistant.py quiz 99-magic
Error: There is no note called 99-magic.
$ echo $?
4
Why does BudgetExceeded get exit code 3 instead of the 1 every other failed run gets?
A mistake inside ask_command raises a KeyError. What does the program do?
Task
In study_assistant.py, write three functions and end the file with the guard.
error_line(error): for a ValidationError, return the first of error_lines(error); for any other error, the first line of str(error).
run_command(args, *, client, budget, sleep, stdin): inside try, run the command args.command names. For "ask", print what ask_command(args.question, ...) returns; for "quiz", print what quiz_command(args.note_id, args.count, ...) returns; for "chat", call chat_command(..., stdin=stdin). Pass client, budget and sleep to each. Return 0. Then catch, in this order, printing f"Error: {error_line(error)}" to sys.stderr each time:
BudgetExceeded: return3;FileNotFoundError: return4;ValueError,ReplyCutOff,StreamBrokenoropenai.APIError: return1.
Catch nothing else.
main(argv=None, *, client=None, notes_dir=NOTES_DIR, stdin=None, sleep=time.sleep): args = build_parser().parse_args(argv), then use_settings(notes_dir). Use build_client() when client is None and sys.stdin when stdin is None. Create one budget = {"limit": args.budget, "used": 0} and return run_command(args, client=client, budget=budget, sleep=sleep, stdin=stdin).
Replace the last comment with if __name__ == "__main__": and raise SystemExit(main()) under it. Run python stages.py to see every exit code, try the exit result yourself in the Shell, then check your work.