Read the Status Line and Headers

fetch now returns the head of a reply as bytes. A program cannot do much with bytes: it needs the status code as a number, to decide whether the request worked, and the headers by name, to find out how long the body is. This lesson writes parse_head(head), which turns the head into a dictionary:

{"version": "HTTP/1.1", "status": 200, "reason": "OK", "headers": {"content-type": ["application/json"], ...}}

The status line

The first line of the head is the status line. Like a request line, it has three parts separated by spaces: the version, the status code, and the reason phrase, a short description for people to read. The reason may contain spaces of its own, so cutting at every space would break it apart. split takes a second argument for exactly this, maxsplit, as in Lesson 2: with 2, it cuts at the first two spaces only, and the rest of the line stays in one piece.

print("HTTP/1.1 429 Too Many Requests".split(" ", 2))

Three rules decide whether a status line is one parse_head accepts:

• The version starts with HTTP/.

• The code is exactly three digits. Check with isascii() and isdigit() together: isdigit() alone also accepts characters such as ², which int cannot convert.

• The reason may be empty. A server may send HTTP/1.1 299 with nothing after the last space, or HTTP/1.1 299 with no last space at all. In the first case split(" ", 2) gives three items, the last one ""; in the second it gives two. Either way the reason is "".

Anything else is not a status line, and parse_head raises ValueError. A code that passes the checks becomes an int, so a caller can compare it with 200 or 404.

Bytes to text

The head arrives as bytes, and split(" ", 2) needs text. Decode the whole head with "iso-8859-1" before splitting it. That encoding maps every possible byte to exactly one character, so decoding a head never fails, whatever a server sends. A head is meant to be plain ASCII, and for ASCII bytes the result is the same as with any other common encoding. Python’s own http.client reads heads the same way.

Header lines

After the status line, split the head at \r\n and treat each line as a header line: a name, a colon, and a value. Split each line at the first colon only, line.split(":", 1), because a value may contain colons of its own:

Then tidy both parts. strip() removes the spaces around the value, which HTTP allows. lower() turns the name into lowercase, because header names are case-insensitive: Content-Length, content-length, and CONTENT-length are the same header. The requests library you used in Python in Practice lets you look up a response header in any casing; your parser gets the same effect by storing every name in one casing. A line without a colon is not a header line, and parse_head raises ValueError for it.

The same name may appear more than once, and a later line must not erase an earlier one. So the dictionary maps each name to a list of values, and each line appends its value to the list for its name, starting a new list the first time a name appears. A reply with Vary: a and then vary: b gives {"vary": ["a", "b"]}.

Your task

In wire.py, write parse_head(head):

1. decode head with "iso-8859-1" and split it at "\r\n";

2. split the first line with split(" ", 2); raise ValueError unless it has at least two parts, the first starting with "HTTP/" and the second exactly three ASCII digits; the reason is the third part if there is one, and "" otherwise;

3. for every other line, raise ValueError if it holds no colon; otherwise split it at the first colon, lowercase and strip the name, strip the value, and append the value to the list for that name;

4. return {"version": ..., "status": ..., "reason": ..., "headers": ...}, with the status as an int.

Leave everything else in wire.py as it is. fetch still returns the head as bytes, so try parse_head on a live head from the Shell tab:

python -c "import wire; print(wire.parse_head(wire.fetch('GET', '/v1/items/1')))"

The first three entries come from the status line. Your status line should read HTTP/1.1 200 OK. If it reads HTTP/1.0 200 OK, the reply came straight from the service, which answers in HTTP/1.0; parse_head accepts both, and Chapter 9 explains what the version changes. In the headers, content-type should hold ['application/json; charset=utf-8'] and content-length should hold ['53'], the size of this item’s body, which fetch has not read yet. If your headers include server and date, those lines come from the small web server that stands in front of practice-api; if a reply you receive has neither, it came from the service directly, and nothing in this course depends on them.

Then press Check my work. The next lesson uses content-length to read exactly the body that follows the head.