0%

Chapter 2 · practice

Conversations and Instructions

A Model Remembers Nothing

In Chapter 1 you read that a model keeps nothing between calls. This lesson shows it happening, then fixes it the only way it can be fixed: your program sends the conversation again with every request.

Your project for this chapter

Open conversation.py in the project editor. It starts with an import:

from llm_basics import MODEL, answer_text, ask_model, send_body

llm_basics.py is Chapter 1’s finished code, supplied next to it, so ask_model, send_body and answer_text work as you built them. You do not need to change that file. Checking your work uses its own copy of it, so an edit there cannot change a grade, but it can stop the demonstration from working.

Below the import are two constants you will use later in the chapter and eight unfinished functions. As in Chapter 1, python conversation.py runs the file, and the demonstration at the bottom calls each function as soon as it exists.

Two calls that do not know each other

In the Shell, run python conversation.py now, before you change anything. The demonstration sends My name is Sam. with ask_model, then sends What is my name? with a second, separate ask_model call:

Two separate calls:
  You: My name is Sam.
  Model: Nice to meet you, Sam.
  You: What is my name?
  Model: I don't know your name. You haven't told me in this conversation.
One call that carries the history:
Run stopped at the next function to build: remembering_messages is built in Lesson 1.

The last line is the same stop line as in Chapter 1: the demonstration stops at the first function you have not built yet. The second request carried one message, What is my name?, and nothing else. From the model’s side, that request is the whole conversation. The first request happened, but nothing about it reached the second one.

A real model behaves the same way. The scripted model gives this exact reply because it was written to, but a live model sent only What is my name? has no way to know the answer either.

Send the history

To let the model answer, the second request has to contain the first exchange as well as the new question. A chat request’s messages list holds the conversation in order, oldest first:

messages = [
    {"role": "user", "content": "My name is Sam."},
    {"role": "assistant", "content": "Nice to meet you, Sam."},
    {"role": "user", "content": "What is my name?"},
]
for message in messages:
    print(message["role"], "said:", message["content"])

What you typed goes in as a user message, as in Chapter 1. The model’s earlier reply goes back in as an assistant message, so the model can see what it said before. Lesson 2 looks at these roles, and a third one, more closely.

This list is the conversation’s history. The model does not store it anywhere. Your program keeps it and sends all of it every time, and the model reads it fresh on every call. That is how every chat app you have used appears to remember.

Build remembering_messages

remembering_messages() returns exactly that three-message list: the user’s My name is Sam., the assistant’s Nice to meet you, Sam., and the user’s What is my name?, in that order and with that exact text.

Return a new list on every call. Writing the list inside the function body does that, because Python builds a fresh list each time the line runs. If the function returned one list stored at the top of the file instead, every caller would share it, and a caller that appended a message would change what the next caller gets. Lesson 3 relies on histories that do not change behind your back.

Run python conversation.py again. The demonstration sends your list with send_body:

response = send_body({"model": MODEL, "messages": remembering_messages()})

and this time the model answers from the history:

One call that carries the history:
  Model: Your name is Sam.
A system message first:
Run stopped at the next function to build: system_message is built in Lesson 2.

The scripted model has a prepared reply for What is my name? that it only gives when an earlier user message says my name is sam. SCRIPTED_MODEL.md lists that rule. It is a stand-in for what a real model does by reading the history, so you can see the difference without a live model.

A chat app shows a long conversation, and the model's newest reply refers to something you said ten messages ago. What made that possible?

In the history you send, the model's earlier reply Nice to meet you, Sam. appears as a message with which role?

Task

In conversation.py, replace the unfinished body of remembering_messages().

Return a new list of three messages, in this order:

  1. {"role": "user", "content": "My name is Sam."}

  2. {"role": "assistant", "content": "Nice to meet you, Sam."}

  3. {"role": "user", "content": "What is my name?"}

Run python conversation.py to see the model answer from the history, then check your work.