0%

Chapter 7

A Small Agent

From One Tool Call to a Loop

Chapter 6 ended with answer_with_tools: one request, one round of tool calls, one more request, and an answer. It gives up when the model asks for a tool a second time, because it cannot know how many rounds a question needs. This chapter removes that limit and replaces it with limits of your own.

Your project for this chapter

Open agent.py in the project editor. It imports from three course files:

from client_calls import MODEL, make_client, scenario_client
from prompts import TOOLS_SYSTEM
from tools import (
    TOOL_DESCRIPTIONS,
    answer_with_tools,
    assistant_turn,
    ...
)

tools.py is Chapter 6’s finished code, supplied next to it together with the earlier files it imports: client_calls.py, structured.py, quiz_models.py, conversation.py, llm_basics.py and errors.py. You do not need to change any of them. As before, checking your work uses its own copies, so an edit there cannot change a grade, but it can stop the demonstration from working.

The top of agent.py is finished code that later lessons ask you to read: a third tool, save_flashcard, which writes a flashcard file, with its description and a check on the file name; two dictionaries of tools, READ_TOOLS and AGENT_TOOLS; and a few helpers, describe_tools, approve_all, refuse_all and print_trace. Below them are four unfinished functions: run_agent, run_calls, repeated_call and stop_reason. Most of the chapter’s work happens in run_agent, which grows a little in every lesson. python agent.py runs the file, and the demonstration at the bottom calls each of your functions as soon as it exists. The demonstration code below your functions is not part of the course. One helper there, has_parameter, uses Python’s inspect module to see which parameters your run_agent has so far, so the demonstration can call the version you have. You do not need to read it.

One round is not enough

In the Shell, run python agent.py now, before you change anything. The demonstration asks answer_with_tools a question from Chapter 6’s notes, then sends the same first request itself and prints the tool calls in the reply:

print(answer_with_tools("What does the note on errors say about finally?", client=client))
One round of tools, from Chapter 6:
I needed more than one round of lookups to answer that.
The first reply asked for:
[{'id': 'call_1_1', 'name': 'list_notes', 'arguments': '{}'}]
The agent loop:
Run stopped at the next function to build: run_agent is built in Lesson 2.

answer_with_tools returned NEEDS_MORE_LOOKUPS. The first reply shows why. The question says “the note on errors”, not 07-errors, so the model did not know the note’s id. It asked for list_notes first. Given the list, it asked for read_note with 07-errors: a second round of tools, which answer_with_tools refuses to run.

The scripted model is playing this out from a table, as SCRIPTED_MODEL.md shows, but a live model does the same kind of thing all the time. It looks something up, reads the result, and only then knows what to look up next. A program cannot tell in advance how many rounds that takes.

The loop

The fix is to keep going:

  1. Send the conversation, with the tools.

  2. If the reply asks for nothing, it is the answer. Stop.

  3. Otherwise, run every call it asked for, add the assistant message and the tool results to the conversation, and go back to step 1.

A program that runs this loop is what people usually mean by an agent: the model chooses the next step, and your program carries it out and reports back. Nothing new happens inside a round. Each one is exactly the round you built in Chapter 6, with read_tool_calls, assistant_turn and tool_messages. The only new thing is that the program goes round again.

For the errors question, the loop makes three requests:

RequestThe conversation it sendsThe reply asks for
1system message, questionlist_notes
2the same, plus the assistant message and the list of notesread_note with 07-errors
3the same, plus the assistant message and the note’s textnothing: it is the answer

Three risks

A loop that keeps going until the model is satisfied has three problems, and the rest of the chapter deals with them one at a time.

First, the model decides when to stop. The loop above only ends when a reply asks for nothing. A model that keeps asking, whether because of a bug in a tool, a confusing result or just bad luck, keeps the program running. Lesson 3 adds a limit on the number of steps.

Second, every round costs more than the last. Each request carries the whole conversation, and the conversation grows by a note’s text every round, so the third request above is many times bigger than the first. Lesson 4 counts the tokens and stops at a budget, and Lesson 5 writes down what each step did and cost.

Third, a tool can change something. Reading a note is harmless. save_flashcard writes a file, and the model chooses its name and contents. Lesson 6 makes the program ask a person before any such tool runs.

Lesson 7 adds one more check for the first risk: a model that asks for exactly the same call twice will not learn anything new the second time. The project puts all four stop checks in one function, in a fixed order.

Why did answer_with_tools return NEEDS_MORE_LOOKUPS for "What does the note on errors say about finally?"

A basic agent loop sends a request, runs the tools the reply asks for, sends the results back and repeats. With no other checks, what makes it stop?

An agent loop stops only when a reply asks for no tools. The model keeps asking for read_note with 07-errors on every turn. What is the risk?