0%

Taking It Home

Where to Go Next

You started this course with one HTTP request and a reply you read by hand. You finish it with a program that calls a model through a client, streams its answers, turns them into validated objects, lets the model call your functions in a loop with limits, grounds its answers in your notes and proves its sources, fails well, stays inside a budget, is tested without a network, and runs against a real provider with a key nobody else can see. That is most of what an application built on a language model is made of.

Four directions lead on from here. Each builds on something you already did by hand.

Retrieval by meaning: embeddings

Chapter 8’s retrieval compares words. A question and a paragraph score a point for each word they share, after the stop words are removed. You saw the price of that rule twice. In Chapter 8, tuples did not match tuple. In Chapter 10’s eval, the question about filtering with a list comprehension missed its note, because comprehension does not match comprehensions. The program knew nothing about meaning; it compared spelling.

An embedding is a way to compare meaning instead. A model turns a piece of text into a long list of numbers, chosen so that texts about the same thing end up with similar lists. tuples and tuple land close together, and so do “duplicate a list” and “make a copy”. Retrieval then works as yours does, with a different score: embed every chunk once, embed the question, and keep the chunks whose numbers are closest to the question’s. Many providers offer an embeddings endpoint next to chat completions, and some models for it run locally. Once there are many documents, a vector store keeps the numbers and finds the closest ones quickly.

Everything around the score stays as you built it: chunks with labels, the best few in the prompt, citations checked against what was sent, and “I don’t know” when nothing is close enough. Chapter 10’s eval is how you would find out whether embeddings retrieve better than words on your own notes.

Agent frameworks

Libraries for building agents write the loop for you: they send the request, read the tool calls, run your functions, send the results back and keep a trace of each step. That is Chapter 7’s run_agent, with its step limit, its budget, its approval gate for tools that change things and its check for a repeated call. Because you wrote that loop yourself, you know what a framework must do on your behalf and where to look when it misbehaves: how many steps it allows, what it spends, which tools run without asking, and what it does when the model asks for the same thing twice.

Evaluation tools

Chapter 10’s eval was ten questions, scored for retrieval and for citations, run once. Evaluation tools do the same thing at a larger scale: more cases, several runs of each to see how much a live model varies, and graders that are themselves models, asked to judge whether an answer is right when no simple rule can tell. The habit is the one you practiced in Chapter 11: measure a change before you keep it, and trust only differences that are bigger than the noise.

The Model Context Protocol

In Chapter 6 you described each function to the model with a name, a description and a JSON schema of its arguments, and your program ran the call when the model asked for it. The Model Context Protocol, MCP, is a shared standard for exactly that. A program that offers tools, called a server, describes them once in a standard form, and any application that speaks the protocol can list them and call them. It is Chapter 6’s descriptions and dispatch, agreed on by many programs, so a tool written once can be used by many assistants.

Whichever you pick, the same questions you learned to ask still apply: what does the model see, what can it make my program do, how do I know the answer is right, what happens when a call fails, and what does it cost.

Why did the course's retrieval miss the paragraph about tuple for a question about tuples, and what would an embedding change?

What does an agent framework do that run_agent did by hand?

What does the Model Context Protocol standardize?