A Small Agent · practice
Stop When the Budget Is Used Up
The step limit guarantees that run_agent ends. It says nothing about what the run costs. Six small steps and six steps that each carry three whole notes count the same, and the second run can use many times the tokens of the first.
Where the tokens go
Every reply reports its size in response.usage, as in Chapter 1: prompt_tokens for what was sent, completion_tokens for what came back, and total_tokens for both. A provider bills by those numbers.
Here is the errors question again, step by step, as the scripted model counts it:
| Step | What the request carries | total_tokens | Running total |
|---|---|---|---|
| 1 | system message and question | 48 | 48 |
| 2 | the same, plus the assistant message and the list of notes | 89 | 137 |
| 3 | the same, plus the assistant message and the whole errors note | 454 | 591 |
The third step costs more than the first two together, because the request carries everything the conversation has collected. That is the second risk from Lesson 1: every round costs more than the last, and a long run grows quickly.
The scripted model’s counts are an approximation. As SCRIPTED_MODEL.md says, it counts the characters of the message texts and divides by four. A real provider counts its own tokens and also charges for the tool descriptions, which the scripted model ignores. The numbers are good enough to learn the rule with. The assistant messages that ask for tools have no text content, so the scripted model counts them as nothing.
Stop at a budget
run_agent gets another keyword-only parameter, max_tokens_total, after max_steps. Inside it:
Before the loop, start a running total:
used = 0.After each call, add the reply’s tokens:
used += response.usage.total_tokens.A reply with no tool calls still returns
"answered".A reply that asks for tools is checked against the budget first. If
used >= max_tokens_total, return{"answer": "", "stop_reason": "budget", "messages": messages}.Then the step limit, as before.
The budget is checked before the step limit. When both are reached by the same reply, the reason says the money ran out, which is the more important thing for the caller to know.
Notice what the check cannot do. The size of the next call is not known until it has been made, so the loop can only look at what was already used. A run can end over its budget, because the reply that crossed the line has already been paid for. The total counts what was really used, including that reply. And as with the step limit, an answer that crosses the budget is still returned: it is already paid for, and it is what the run was for.
Chapter 9 replaces this count with a budget that every model call shares and checks before each call, and the tests in Chapter 10 use that final version of run_agent.
Run it
Run python agent.py. The demonstration now passes max_tokens_total=2000 to the earlier runs, and adds the errors question with max_tokens_total=100:
A budget of 100 tokens:
stop_reason: budget
The conversation has 4 messages.
Run stopped at the next function to build: run_agent returns a trace from Lesson 5.
The arithmetic is the table above. After step 1 the total is 48, below 100, so the loop runs list_notes and goes on. After step 2 it is 137, which is 37 over the budget, so the loop stops without reading the note. The four messages are the system message, the question, and one round of tools. With a budget of 2000 the same question answers, at 591.
max_tokens_total is 100. After a reply that asks for tools, the running total is exactly 100. What does run_agent do?
Why can a run end with a total above max_tokens_total?
Task
In agent.py, give run_agent a keyword-only parameter max_tokens_total, after max_steps.
Keep a running total, starting at 0, and add each reply’s
response.usage.total_tokensto it.A reply with no tool calls still returns
"answered", even when it took the total past the limit.After a reply that asks for tools, if the total has reached
max_tokens_total, return{"answer": "", "stop_reason": "budget", "messages": messages}. Check this before the step limit, and do not run the calls.
Run python agent.py to see a budget of 100 tokens stop the errors question, then check your work.