Headers the Server Received

Every request you have sent carries one header line, and you did not write it: HOST_LINE, Host: practice-api:8080, supplied in Chapter 1 with a promise that this chapter would explain it. In this lesson you build that line from the address, write any other header lines a caller asks for, refuse the ones that would break the request, and send one header to practice-api’s /v1/headers route, which reports what it received.

Why Host exists

When your client connects to practice-api on port 8080, the name is first resolved to an IP address, as in Networks with Python. The connection itself carries only that address and the port. The name you started from is gone by the time the first byte arrives.

That matters because one address often answers for many names. A single server can host books.test, shop.test, and news.test at once, all resolving to the same IP address and all on port 80. When a request arrives, the server has to know which of them the client meant, and nothing in the connection says. The Host line says it: the request names the server it is meant for, and the server picks the site to answer from.

So HTTP/1.1 requires every request to carry exactly one Host line, and a server must refuse a request without one with 400 Bad Request. The web server in front of practice-api should do exactly that, with a short HTML page instead of JSON. If a request without Host ever gets the usual 200 instead, it reached the service directly, which tolerates the omission; HTTP/1.1 still requires the line, and your client always sends it.

The value of Host copies the host and port of the URL, exactly as the URL wrote them:

A URL that names no port gets a Host line with no port, even though the client connects to port 80. That is why Lesson 2 kept the None from urlsplit: your host_header(host, port) takes port=None to mean “the URL named no port” and writes the host alone. Because 80 is the default port for http, Host: books.test and Host: books.test:80 name the same server, and servers accept either; your host_header writes the port whenever it is given one.

Header syntax

Every header line has the same shape: a name, a colon, then the value, ending in \r\n. A space after the colon is customary, and your client always writes one:

X-Practice-Name: http-with-python

The name is one word: the first colon is where it ends, so it cannot hold a colon, and it cannot hold a space, a tab, or a line break either. HTTP’s full rule for names is stricter still, and also forbids characters such as /, (, ,, and @; your client refuses the ones that would break the request. The value is the rest of the line, and it may hold spaces and colons; it can never hold \r or \n, because those end the line.

When a field has several values, the client writes one line per value, with the same name each time. That is the shape of Chapter 1’s parse_head, read the other way round. There, the lines X-Trace: a and x-trace: b became {"x-trace": ["a", "b"]}; here, a caller who passes {"X-Tag": ["a", "b"]} gets two lines:

X-Tag: a
X-Tag: b

Going from the dict to the lines is two loops, one over the names and one over each name’s values:

A caller who passes a single string, {"Accept": "application/json"}, means one line. format_headers accepts both forms: a value that is a str is treated as a list of one, so the inner loop never walks through a string letter by letter.

A line break in a value

Header values often come from somewhere else: a user’s name, a title, a value read from a file. Suppose one of them holds a line break:

The caller meant one header, X-Note, with an odd value. The server reads two: X-Note: hello, and X-Admin: yes, a line nobody wrote on purpose. This is header injection. If any part of a value can be chosen by someone else, they can add any header they like, or end the head early with an empty line and start a body. The fix is simple and strict: refuse any name or value that holds \r or \n, and raise ValueError before anything is sent.

Your task

Four parts of wire.py are this lesson’s, in this order.

1. host_header(host, port) returns the Host line, ending in \r\n: Host: practice-api:8080\r\n for ("practice-api", 8080), and Host: books.test\r\n for ("books.test", None).

2. format_headers(headers) returns a str with one Name: value\r\n line per value. Loop over the names and their values as in the example above. When a value is a str, which you can test with isinstance(values, str), treat it as a list of one. Raise ValueError for an empty name; for a name that holds a space, a colon, a tab, \r, or \n; and for a value that holds \r or \n. An empty dict gives "".

3. Change make_request to make_request(method, target, headers=None). Build the head from the request line, then host_header(ADDRESS[0], ADDRESS[1]) in place of HOST_LINE, then format_headers(headers) when headers is not None, then the blank line. Host goes first, as HTTP recommends. Remove HOST_LINE from the top of the file, so nothing can add a second Host line by accident.

4. Give fetch a headers=None parameter after the *, next to connect and timeout, and pass it on to make_request.

First, read your own bytes. In the terminal:

python -c "import wire; print(wire.make_request('GET', '/v1/headers', headers=wire.PRACTICE_HEADERS))"

b'GET /v1/headers HTTP/1.1\r\nHost: practice-api:8080\r\nX-Practice-Name: http-with-python\r\n\r\n'

These are the exact bytes your client sends: the request line, your Host line built from ADDRESS, the practice header, and the empty line. Check each one. The live runs in this chapter pass headers=wire.PRACTICE_HEADERS, as README.md says; python wire.py on its own sends only Host.

What the server received

Now send the request. /v1/headers answers with the X-Practice-Name header it received, and with nothing else: it does not report Host or any other header. practice-api also sits behind a small web server that passes each request on to it, and that front server is set up to write its own Host line for the service, so even a route that reported Host would not show yours. For Host and every other line, your own request bytes above are the record of what you sent. For X-Practice-Name, the server’s reply is a second record:

python -c "import wire; print(wire.fetch('GET', '/v1/headers', headers=wire.PRACTICE_HEADERS)['body'])"

You should see {"header":"X-Practice-Name","value":"http-with-python"}: the route reflects that one header, whatever casing you sent its name in, and no other. If the body looks different, compare it with your request bytes above; those are what you sent.

The body is printed as bytes, so it ends in \n, the newline practice-api puts after every JSON body.

Last, send two values for the practice header. format_headers writes them as two lines, and practice-api refuses a repeated practice header:

python -c "import wire; r = wire.fetch('GET', '/v1/headers', headers={'X-Practice-Name': ['a', 'b']}); print(r['status'], r['body'])"

You should see 400 with the message "X-Practice-Name must not be repeated": the server saw two lines with the same name, which is what a repeated field is on the wire. If you see something else, your request bytes still hold the two lines; compare them with the reply. Nothing in this run is checked; it is there for you to see.

Then press Check my work.

If a check fails, ask Monty to compare the failing message with your format_headers loop and say which rule it breaks, without writing the code.

Your client now writes every line of a request itself, but fetch still sends only to practice-api, and only a target you built separately. The next lesson puts the pieces together in one function that takes any method, any URL, any headers, and a body.