The Browser You Are About to Model

Every client you have pointed at your server so far does exactly what it is told. wire.py sends the bytes you type. pl_sec.py sends the cookie it is handed. A test builds a request dict and calls handle().

A browser is a different kind of client. It decides for itself which cookies go along with a request. It decides whether a request is sent at all. And it decides whether the page that started a request may read the answer. A person opens a page or clicks a button, and the browser makes those three decisions without asking anyone.

Your service already counts on those decisions. In Chapter 4 you put Secure, SameSite=Lax, and the __Host- prefix on the session cookie, and each of them is an instruction to a browser. In Chapter 7 the service got a page with a Delete button on every book. The only thing that authorizes a delete from that page is the session cookie, which a browser attaches by its own rules.

This chapter and the next are about those rules. There is one obstacle: this site has no browser. Your workspace is a terminal and an editor, and nothing in it can load a page.

What you will build, and what it costs

So you write the deciding part of a browser yourself. The file mini_browser.py gets one small function per rule: what the origin of a URL is, which cookies go along with a request, and, in Chapter 9, whether a page may read a response. A supplied function, visit(), calls your rules one after another for a single request, and your tests carry its requests to handle() in memory.

That is a model, and a model costs something. A real browser follows the Fetch standard, the HTML standard, and the cookie standard, RFC 6265bis. Together they run to thousands of lines. Your functions will come to a few dozen. Each rule you write is based on one of those standards, and each lesson says what its rule leaves out or simplifies. Here are the largest gaps, before any code:

• No public suffix list. A real browser uses that list to decide which hosts belong together as one site. The model uses a shorter rule, and Lesson 2 explains both.

• No Domain attribute and no expiry. A cookie in the model belongs to one exact host, and it stays until the server removes it.

• No frames, no redirects, no caches. The model looks at one page and one request at a time.

• No null origins. A real browser gives some pages an origin that is written null. The model knows http and https URLs and refuses everything else.

• Two finer rules about headers, which Chapter 9 names when it reaches them.

• No differences between browsers. The model gives one answer to every question. Where real browsers give different answers, the lesson says so.

Keep one more limit in mind. When a test in these chapters passes, your server has passed against the model. Nothing on this site runs it against a real browser.

Four names

A browser rule compares where a page came from with where a request is going. So the model needs a few places, and the course uses the same four from here to the capstone:

All four end in .example, an ending that is reserved for examples and never leads to a real server. Nothing in this course looks these names up. Inside the model they are text that your functions take apart and compare.

An origin

The first thing a browser works out about a URL is its origin: the scheme, the host, and the port. Nothing else in the URL counts.

https://reading.example:8443/books?status=done
|___|   |_____________| |__||________________|
scheme       host       port  path and query

|__________________________|
         the origin

In https://reading.example:8443/books?status=done, the scheme is https, the host is reading.example, and the port is 8443. Those three make the origin, which is written https://reading.example:8443. The path /books and the query status=done are not part of it.

Most URLs name no port. urlsplit from HTTP with Python shows what is there and what is not. Run this block:

The first line of output ends in None, and so does the last: urlsplit reports no port when the URL names none. The origin rule fills the gap. A missing port means the default port of the scheme, 443 for https and 80 for http. So the first two URLs have the same origin, and it is written without the port: https://reading.example. The third names another port and is another origin. The fourth has another scheme and is another origin too.

Two URLs share an origin only when all three parts agree.

from urllib.parse import urlsplit

for url in ["http://reading.example:8080/x", "https://app.reading.example/books"]:
    parts = urlsplit(url)
    print(parts.scheme, parts.hostname, parts.port)

An origin is the unit a browser protects, and Lesson 4 says what that protection covers. Nothing was sent anywhere in this lesson. In the next one you write the origin as a function, and you add a coarser unit that cookies go by.