Read Response Headers

The status class tells a client whose move it is. Some moves need more than that. After a 429, how long should the client wait? Is this body JSON at all, or an HTML error page from a server in between? The answers arrive in the response headers. This lesson adds the two headers a decision most often needs to the record get_json returns: Content-Type and Retry-After.

Two ways to read a header

Your record already holds every header in the dict you built from getheaders(): lowercase names, each with a list of values. The response object also has a method for one header at a time:

response.getheader("Retry-After")

getheader(name) finds the name in any casing, so "retry-after", "Retry-After", and "RETRY-AFTER" all find a line the server wrote as Retry-After: 1. It returns the value as a string, or None when the header is absent; a second argument, as in getheader("Retry-After", "0"), replaces that None. If the name appears more than once, getheader joins the values into one string with ", " between them.

Both ways are case-insensitive, and each fits a different job. For a header that carries one value, such as Content-Type or Retry-After, getheader gives you that value directly. For a header that may repeat, where you need each value on its own, the dict keeps them apart: X-Trace: a and X-Trace: b stay ["a", "b"] there, while getheader("X-Trace") returns "a, b". Joining is safe only when a comma cannot appear inside a value, which is not true of every header; Chapter 7 meets one where it is not.

What you should not do is look the name up in a dict built from the server’s own casing. A server may write retry-after or RETRY-AFTER, and HTTP says field names are case-insensitive, so {"Retry-After": "1"}.get("Retry-After") works only with a server that cased the name your way.

Retry-After is text

Retry-After tells the client when to ask again. It holds either a number of seconds, such as Retry-After: 120, or a date, such as Retry-After: Fri, 31 Dec 1999 23:59:59 GMT. Because it may be a date, the value is not always a number, and get_json keeps it as the text that arrived. Turning it into a wait, and deciding whether to try again at all, belongs to whoever calls get_json.

Python in Practice’s rule for 429 still holds here: it is a signal to stop. get_json reports what the server said and returns; it does not sleep and send the request again.

Content-Type has parameters

Content-Type names the format of the body. practice-api sends Content-Type: application/json; charset=utf-8. The part before the ; is the media type, application/json; what follows is a parameter, here the character encoding of the text. Another server might send application/json with no parameter, or application/json;charset=UTF-8 with no space. Lesson 4 uses the media type to decide whether to decode a body as JSON. For now, keep the whole value as it arrived.

Look at a 429

The rate-limit scenario answers by the attempt number you send: attempts 1 and 2 get a 200, and attempts 3 to 10 get a 429. It remembers nothing between requests, so attempt 3 is always a 429. Among the header lines of that reply are these, which practice-api sends itself:

Content-Type: application/json; charset=utf-8
Retry-After: 1
X-RateLimit-Limit: 2
X-RateLimit-Remaining: 0

Your task

1. In wire.py, add two keys to the record get_json returns: "content_type", from response.getheader("Content-Type"), and "retry_after", from response.getheader("Retry-After"). Read both inside the with block, keep each as the text that arrived, and leave None when the header is absent. Keep everything Lesson 1 wrote.

2. Look at a live 429:

python -c "import wire; r = wire.get_json('/v1/scenarios/rate-limit?attempt=3'); print(r['status'], r['content_type'], r['retry_after'])"

It should print 429 application/json; charset=utf-8 1. Try attempt=2 as well: the 200 that comes back carries no Retry-After, so its retry_after is None.

Then press Check my work. Your client now reads the status and the headers a decision needs, but it still treats every body the same way. The next lesson sorts every scenario by class and keeps the error bodies.