0%

Chapter 8

Answering from Your Notes

The Model Doesn't Know Your Notes

Chapter 7’s agent answers from the notes by reading them: it asks for read_note, your program sends the whole note back, and the model writes its answer from that text. Without the note in the request, the model has nothing of yours to go on. This chapter makes that idea precise. Your program finds the few passages of the notes that fit a question, sends only those, asks the model to answer from them and name them, and then checks that it did. Answering from passages the program supplies is called a grounded answer.

Your project for this chapter

Open retrieval.py in the project editor. Below Path and openai, it imports from three course files:

from answer_models import CitedAnswer
from client_calls import MODEL, make_client
from prompts import GROUNDED_SYSTEM

client_calls.py is Chapter 5’s finished code. agent.py, Chapter 7’s finished agent, is supplied next to it with every earlier file it imports, including Chapter 6’s tools.py; this chapter’s code does not import it, but the project builds the search tool that agent will use. As before, checking your work uses its own copies of these files, so an edit there cannot change a grade, but it can stop the demonstration from working. prompts.py now also holds GROUNDED_SYSTEM, the system message you read in Lesson 5. answer_models.py is new and almost empty: you write a pydantic model in it in Lesson 6.

Below the imports are six constants that later lessons explain, one finished function, load_notes, twelve unfinished functions and one empty placeholder, SEARCH_NOTES_DESCRIPTION = {}. load_notes returns a dictionary from each note id, such as 03-lists, to the note’s text, read as UTF-8 from the notes folder, in the order of the ids. python retrieval.py runs the file, and the demonstration at the bottom calls each function as soon as it exists.

A question about one detail

On 2026-11-08 this question went to openai/gpt-oss-20b, the model behind pl-live, as the only message:

What example list does the Python Land lists note use, and what values are in it?

This is its first reply, unedited:

In the ordinary Python‐language reference (the “Python Land” documentation) the *Lists* section uses a very small example list to illustrate how lists are made and how they’re indexed.  
The example list is simply  

```python
[1, 2, 3]
```

and it contains the three integers **1, 2** and **3**.  The list is used throughout the section to demonstrate indexing, slicing, mutating, and other list operations.

It reads well, it is sure of itself, and it is wrong. Two more samples of the same question named other lists, [1, 2, 3, 4, 5, 6, 7, 8, 9] and ['spam', 'ham', 'eggs'], and three samples of a question about the functions note named a greet or hello_world function. The functions note defines average. The course keeps all six raw replies, unedited.

What the note says

In the Shell, run python retrieval.py now, before you change anything. The demonstration loads the notes with load_notes and prints the lists note:

load_notes read 12 notes. The lists note:
# Lists

A list is an ordered collection of values. You write one with square brackets, such as `scores = [90, 72, 85]`. A list can hold values of any type, including a mix of types and other lists.

Each item has a position, called its index, starting at 0. `scores[0]` is the first item and `scores[-1]` is the last. Reading an index that does not exist raises `IndexError`. A slice such as `scores[0:2]` returns a new list with the items from position 0 up to, but not including, position 2.

Lists are mutable, so you can change them after creating them. `scores.append(60)` adds an item to the end, `scores.insert(0, 100)` adds one at a position, `scores.remove(72)` removes the first matching value, and `scores[1] = 75` replaces an item in place. These methods change the list and return `None`, so `scores = scores.append(60)` loses the list.

Because a list is changed in place, two names that refer to the same list both see every change. Use `scores.copy()` or `list(scores)` when you need an independent list. Adding two lists with `+` also builds a new list and leaves both originals unchanged.

`len(scores)` tells you how many items a list holds, and `value in scores` tells you whether a value is one of them.
Run stopped at the next function to build: note_chunks is built in Lesson 2.

The first paragraph has the answer: the note’s example list is scores = [90, 72, 85]. The last line is the usual stop line: the demonstration stops at the first function you have not written yet.

Why the model got it wrong

A language model learned from a large amount of public text, collected before it was trained. These twelve notes were not part of it, and nothing in the question contained them. So the model did what it always does: it wrote the most likely continuation of the text it was given. An example list in a lists tutorial is very often [1, 2, 3], so that is what it wrote. The answer is plausible, not checked. And nothing in the reply tells you which parts came from somewhere real.

The scripted model is no help here either. It answers the prompts in its table and echoes anything else. Neither model knows your notes until your program sends them.

Why not send every note

The simplest fix is to paste all twelve notes into every request. They hold 14,943 characters, about 3,700 tokens by the scripted model’s count, so every question would pay for all of them, even a question that needs one paragraph. A request also has to fit the model’s context window, which Chapter 2 introduced, and a real collection of documents is far larger than twelve short notes. And a model given a lot of text can answer less reliably from the one paragraph that matters than from that paragraph alone.

The plan: retrieve, label, cite, check

Retrieval picks the few passages that fit the question and sends only those:

  1. Split each note into passages, called chunks. Here a chunk is a paragraph (Lesson 2).

  2. Score each chunk against the question by the words they share (Lesson 3), and keep the best few (Lesson 4).

  3. Label each chunk, such as 03-lists#1 for the first paragraph of the lists note, and send the chunks after the question. The system message tells the model to answer only from them and to list the labels it used (Lesson 5).

  4. Check the reply in your program: it must be valid JSON in the expected shape, and every label it cites must be one you sent (Lesson 6).

  5. Abstain when no chunk fits well enough: answer “I don’t know” without asking the model at all (Lesson 7).

The project puts these together in answer_from_notes, and turns the search into a tool, search_notes, for the agent.

Step 4 is what makes the answer checkable. The model can still write something wrong, but a citation of a passage that was never sent is caught by your program every time, and a person can read the cited passage to check the answer against it.

This chapter scores chunks by the words they share with the question, using the standard library. Chapter 12 explains embeddings, the method larger systems use to find passages that mean the same thing in different words.

Why did the live model say the lists note uses [1, 2, 3]?

Why not paste all twelve notes into every request?

The model replies with an answer and a list of labels it says it used. What can your program check without trusting the model?