Resolve a Relative Location

A server that moves a book answers with a header such as location: /books/2. That is not a URL a client can connect to: it has no scheme, no host and no port. It is a reference relative to the URL the client asked for, and the client has to resolve it against that URL before it can send anything. HTTP allows this on purpose (RFC 9110, section 10.2.2). A server could build a full URL from the Host line (Chapter 2, Lesson 4), but it would have to copy that value into every location, and a proxy in front of it may have rewritten it; a relative value lets the client, which knows the URL it asked for, do the resolving. The rules for resolving are the ones every browser uses for links, written down in RFC 3986, section 5. This lesson writes the two functions the follower in Lesson 4 needs: one that resolves a location, and one that splits the result into what http.client asks for.

The urljoin function

You do not write those rules yourself. urljoin, from the same urllib.parse module as urlsplit, takes the URL that was requested and the location value, and returns the absolute URL they name together. It is already imported at the top of wire.py. Here are the common shapes of location, each resolved against one request URL:

from urllib.parse import urljoin

asked = "http://books.test/v0/books/2?x=1"
print(urljoin(asked, "/books/2"))
print(urljoin(asked, "../books/2"))
print(urljoin(asked, "?page=2"))
print(urljoin(asked, "//other.test/x"))
print(urljoin(asked, "http://books.test/books/2"))

The output, line by line:

http://books.test/books/2
http://books.test/v0/books/2
http://books.test/v0/books/2?page=2
http://other.test/x
http://books.test/books/2

A value starting with / replaces the whole path and query, and keeps the scheme, host and port. A value starting with // replaces the host too. A value starting with ? keeps the path and replaces only the query. An absolute URL is returned as it is. Everything else, such as ../books/2 or 3, is resolved like a file name relative to the folder the request URL points into: the path up to and including its last /. For /v0/books/2 that folder is /v0/books/, so .. steps up to /v0/ and books/2 lands back on /v0/books/2.

The folder rule has one surprise: whether the request URL ends in a slash changes the folder.

from urllib.parse import urljoin

print(urljoin("http://books.test/v0/books/2", "/books/2"))
print(urljoin("http://books.test/books", "2"))

The second line is http://books.test/2, not http://books.test/books/2. /books has no slash at the end, so its last segment is treated like a file name, and 2 replaces it. A server that means /books/2 should say so with a value that starts with /, which is what the reading list will do.

urljoin neither decodes nor encodes anything: %C3%A6 in a location stays %C3%A6 in the result, ready to go into a request line.

Each location against its own URL

A redirect can lead to another redirect. Each location is resolved against the URL that returned it, which after the first hop is no longer the URL you started with. A follower keeps a current URL, resolves each new location against it, and then makes the result the current URL.

Only http here

urljoin resolves any URL, including one whose scheme is https or ftp. A location may well point there: a server can send location: https://books.test/books/2. wire.py speaks plain HTTP only, as build_request has since Chapter 2, and following that redirect would mean sending the request without the encryption the server asked for. So resolve_location checks the scheme of the resolved URL, and raises ValueError when it is anything other than http. Chapter 9 explains what https adds and why the site cannot show it.

Host, port, target

http.client does not take a URL. HTTPConnection takes a host and a port, and conn.request takes a method and a request target. So the follower needs one more step: split an absolute URL into those three parts. You did exactly this in Chapter 2, spread across build_request and fetch:

• urlsplit(url).hostname is the host.

• urlsplit(url).port is the port, or None when the URL names none; then it is 80, the default for http.

• The target is the path, then ? and the query when there is one. An empty path, as in http://books.test, becomes /.

• The fragment, the part after #, is left out. It is never sent.

A browser that follows a redirect carries the fragment of the old URL over to the new one, so the page still scrolls to the same place. Your client sends no fragment at all, so it has nothing to carry.

Your task

In wire.py, write two functions:

1. resolve_location(request_url, location) returns urljoin(request_url, location), and raises ValueError when the result’s scheme is not http.

2. url_parts(url) returns a dict with "host", "port" and "target" for an http URL, with port 80 when the URL names none, / for an empty path, the query kept, and no fragment.

Together they take about eleven lines. Try them:

python -c "
import wire
print(wire.resolve_location('http://books.test/v0/books/2?x=1', '/books/2?x=1'))
print(wire.resolve_location('http://books.test/books/2', '3'))
print(wire.url_parts('http://books.test:8080/books/2?x=1#top'))
"

The first two lines are http://books.test/books/2?x=1 and http://books.test/books/3, and the dict has host books.test, port 8080 and target /books/2?x=1. Nothing from the terminal is graded.

Then press Check my work. The client side can now turn any location into a request. The server side has nothing to send yet: the next lesson makes the reading list answer its old paths with redirects.