0%

A Small Agent · practice

Write Down Every Step

run_agent now ends for one of three reasons, and the result says which. It does not say what happened on the way: which tools ran, what they returned, and which step used the tokens. When a run ends with "budget", the first question is where the budget went.

A trace

A trace is a list with one entry per step, built while the loop runs and returned with the result. Each entry is a dictionary:

{"step": 2, "tokens": 89, "tool_calls": [...], "results": [...]}
  • "step" is the step number, counting from 1.

  • "tokens" is that reply’s usage.total_tokens.

  • "tool_calls" is what read_tool_calls returned for that reply: a list of dictionaries with id, name and arguments, or [] for an answer.

  • "results" holds the content of each tool message, in the order of the calls.

An answer step has two empty lists. A step that stopped the loop lists the calls it asked for, with an empty "results" list, because none of them ran. That is useful in itself: the trace of a run stopped by its budget shows what the model wanted to do next.

Why not print as you go?

You could add print calls inside the loop instead. It would work in the demonstration, and nowhere else. A trace is data, so the caller decides what to do with it: print it for a person, check it in a test, save it to a file, or add up its tokens. The checks for this lesson read the trace exactly that way. A loop that prints decides all of that for every caller, and its output cannot be tested without capturing the screen.

The supplied print_trace(trace) is one such caller. It prints a line per step and a line per call:

Step 2 (89 tokens)
  read_note {"note_id": "07-errors"} -> 1325 characters

A call that has a result shows the result’s length; a call without one shows (not run); a step with no calls shows answered.

Keep the trace

In run_agent:

  1. Before the loop, start a new list: trace = []. Create it inside the function, so every run gets its own.

  2. After reading the calls, build the entry with "step" set to your step counter, "tokens" set to response.usage.total_tokens, "tool_calls" set to calls and "results" set to [], and append it to trace. Do this before any of the stop checks, so the step that stops the loop is recorded too.

  3. Where the loop adds the tool messages to messages, also append each message’s content to the entry’s "results". The entry is still the dictionary you appended, so appending to entry["results"] changes the one in trace.

  4. Add "trace": trace to every dictionary run_agent returns.

Run it

Run python agent.py. The demonstration prints the traces of the errors question and of the tool-loop run with max_steps=3:

The trace of the errors question:
Step 1 (48 tokens)
  list_notes {} -> 140 characters
Step 2 (89 tokens)
  read_note {"note_id": "07-errors"} -> 1325 characters
Step 3 (454 tokens)
  answered
The trace of the tool-loop run with max_steps=3:
Step 1 (48 tokens)
  list_notes {} -> 140 characters
Step 2 (83 tokens)
  list_notes {} -> 140 characters
Step 3 (118 tokens)
  list_notes {} (not run)
stop_reason: max_steps
Run stopped at the next function to build: run_calls is built in Lesson 6.

The first trace shows the growth from Lesson 4: the third step carries the whole 1325-character note. The second shows what the step limit stopped. Steps 1 and 2 asked for the same thing and got the same 140 characters back; step 3 asked again and was not run. The model learned nothing new after step 1, and the loop paid for two more steps anyway. Lesson 7 uses the trace to notice that.

A run stops with "budget" after a reply that asks for read_note. What does the last trace entry hold?

Why does run_agent return a trace instead of printing each step?

Task

In agent.py, make run_agent keep a trace and return it under "trace" with every stop reason.

  • Start each run with a new empty list.

  • After each reply, append {"step": n, "tokens": response.usage.total_tokens, "tool_calls": calls, "results": []}, where n counts from 1 and calls is what read_tool_calls returned.

  • When the calls run, add each tool message’s content to that entry’s "results", in order.

  • An answer step has two empty lists. A step that stopped the loop keeps its calls and has empty "results".

Run python agent.py to print two traces with print_trace, then check your work.