Let the Client Carry Cookies

Your server now hands out a session cookie and recognizes it when it comes back. In the session scenario, though, the helper copied the cookie from the set-cookie line into the next request itself. That copying is the client’s half of the job, the server’s work from Lesson 2 run in reverse: read Set-Cookie from a reply, keep the cookie, and send it in a Cookie header with the next request to the same server.

urllib, which you used for open_url in Chapter 6, does all three with two pieces:

• http.cookiejar.CookieJar() is the jar: an object that keeps cookies.

• urllib.request.HTTPCookieProcessor(jar) is a step for an opener. On every reply it reads each Set-Cookie line into the jar; on every request it adds a Cookie header with whatever the jar holds for that server.

Put the processor into an opener and every request through that opener carries the session, with no cookie code of your own:

jar = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(jar))

build_opener takes the steps you hand it and adds the usual ones, such as making the connection and following redirects, around them. In Chapter 6 you called it with no arguments.

One opener, one jar

The jar lives as long as the opener that uses it. Two requests sent through the same opener share one jar, so a cookie set by the first reply goes out with the second request. A second opener built with a new CookieJar() starts empty. So a login and the request after it must go through one opener: if the second request uses a fresh one, the server sees no cookie and has no idea the client just logged in.

A POST with no body

The login is a POST with nothing to send. A plain opener.open(url) sends a GET, so build a request object that says otherwise:

request = urllib.request.Request(base_url + "/login", data=b"", method="POST")

data=b"" is an empty body. For it, urllib writes content-length: 0, exactly what your server’s 411 rule asks of every POST. It also adds content-type: application/x-www-form-urlencoded, the type of a form; /login reads no body, so it ignores that. Send the request with opener.open(request, timeout=timeout) in a with statement, as open_url did, and keep response.status. Then open base_url + "/me" through the same opener, with the same timeout, and decode its body with json.loads(response.read().decode("utf-8")), inside the with block.

A 4xx or 5xx reply makes opener.open raise urllib.error.HTTPError, as Chapter 6 showed. open_url turned that into a record, because its caller wanted any reply. login_and_me does not catch it: when the login fails, there is no session to read, and the caller should hear about the failure itself. In Lesson 5, a wrong password will raise exactly that error, with code 401, and /me will never be asked.

What the jar decides

A jar does more than keep the latest cookie. Before each request it decides which of its cookies to send, from the attributes the server set:

• Domain. A cookie goes back only to the host that set it. A cookie from books.test is never sent to annex.test.

• Path. Path=/ sends it with every path; a cookie with Path=/books would not go with /me.

• Max-Age. The jar drops the cookie once that many seconds have passed. Max-Age=0 empties it at once.

• Secure. The jar keeps a Secure cookie, but sends it only over https://, never over http://.

That is why Lesson 1 left Secure off this course’s cookie: your server speaks http://, and a correct jar would keep a Secure session and never send it back.

All of this is urllib’s work, and you will not write any of it. What you write is the wiring: one jar, one processor, one opener for both requests, and a timeout on each.

Your task

In wire.py:

1. Write make_cookie_opener(jar): return an opener built with urllib.request.build_opener and one urllib.request.HTTPCookieProcessor(jar).

2. Write login_and_me(base_url, *, user=None, password=None, opener=None, timeout=TIMEOUT). When opener is None, build one with make_cookie_opener(http.cookiejar.CookieJar()); when an opener is given, use it. Send the POST to base_url + "/login", then the GET of base_url + "/me", both through that opener and both with timeout=timeout. Return {"login": <the login status>, "me": <the decoded /me body>}. Leave user and password unused until Lesson 5.

wire.py already has import http.cookiejar and import urllib.request. Write the names through the modules, as urllib.request.Request and http.cookiejar.CookieJar.

Then run:

python pl_http.py cookies

The helper builds an opener with your make_cookie_opener and a jar of its own, then adds one handler of its own to that opener: it sends the opener’s requests to your server on this machine, so the name books.test is never looked up. It calls your login_and_me("http://books.test", ...) with that opener and the first user and password from sample_state(), which your function ignores for now. Between the first line, which names your server, and the log of your server, you should see:

--- requests and replies ---
request 1  POST /login  ->  200 OK
request 2  GET /me  ->  200 OK
--- your client returned ---
login: 200
me: {'user': 'guest'}
--- requests and replies ---
--- the cookies in the jar pl_http gave your opener ---
session: 'domain books.test, path /, HttpOnly'

Two requests reached your server, and the second came back as guest, so the cookie traveled. The last line is the helper reading its jar afterward: one cookie, session, kept for books.test with path / and the HttpOnly flag.

Then press Check my work. It checks how your code wires the jar and the opener, not what the jar does inside; that part is urllib’s, and the helper just showed it working. Anyone can still log in as guest, though, so what the session records means little yet. Before a login can check a password, the password needs a way to travel in a header, which is the next lesson.