0%

Chapter 10 · practice

requests and httpx, Now That You Know

What a Session Remembers

In Python in Practice you sent one requests.get with a timeout, called raise_for_status(), and decoded the . Each call like that stands alone. In Chapter 7 you needed more for two requests in a row: a login and a /me that knew about it. You built make_cookie_opener, an opener with a jar, and sent both requests through it in login_and_me. In Chapter 9 you wrote fetch_many so that several requests could share one connection.

A requests.Session is that opener, done properly. It is one you send every request through, and it carries what one request leaves behind for the next. This chapter goes back to requests, and later httpx, now that you can name the mechanism behind each feature.

import requests

with requests.Session() as session:
    session.post("http://books.test/login", auth=("ana", "correct horse"), timeout=(1.0, 2.0))
    reply = session.get("http://books.test/me", timeout=(1.0, 2.0))

session.post and session.get take the same as requests.post and requests.get. There is also session.request("POST", url, ...), which takes the as text. auth=(user, password) writes the Authorization: Basic line for you; Lesson 2 looks at how.

What a session keeps

Three things a session keeps matter here, and each carries over from one call to the next.

Cookies. session.cookies is the jar. Its class, RequestsCookieJar, is a subclass of the http.cookiejar.CookieJar you used in Chapter 7, and it follows the same rules: it reads every Set-Cookie line in a reply and sends the cookies that match the host and path with each later request. You never copy a cookie by hand.

Default headers. session.headers holds the headers sent with every request. A new session already has four, which requests adds unasked:

User-Agent: python-requests/VERSION
Accept-Encoding: gzip, deflate, zstd
Accept: */*
Connection: keep-alive

VERSION stands for the version of requests installed here. The zstd is there because the Python these lessons were checked on, 3.14, can decode that compression; on an older Python the line ends at deflate. You add to them like a : session.headers["accept"] = "application/json". The names ignore case, as header names do, so that line replaces Accept: */*. Lesson 5 says what the four are for. Assigning a new dict to session.headers throws the four away; change the one you have.

Connections. A session keeps a pool of open connections, one per server. Your server speaks /1.1, so after a reply the connection stays open, and the session’s next request to the same server goes out on it. That is Chapter 9’s keep-alive, done for you.

Which of these does a requests.Session carry from one call to the next?

What a session does not keep

A timeout. Session has no timeout setting at all, and the obvious guess fails quietly: session.timeout = 2 stores an attribute that requests never reads, and the next call can wait without limit. The only way to bound a call is timeout= on that call, every time.

This course passes TIMEOUT = (1.0, 2.0), a pair, as in Python in Practice: at most 1 second to connect, then at most 2 seconds for each read. Both learner files define it at the top. A that takes a timeout argument passes it on with timeout=timeout.

session.timeout = 2, then session.get(url). How long can the call wait?

Close it

A session holds connections, so close it when you are done, with the same with statement you use for files and : with make_session() as session: closes the session when the block ends.

One detail of the library versions this course runs: closing the session ends a connection only once your program no longer holds any response that came over it. A response kept in a , or an HTTPError that carries one, keeps its connection open until it is gone, or until your server’s 2-second idle limit from Chapter 9 ends it. pl_http closes its sessions’ connections itself, so its runs never wait for that.

Your task

Two functions, in two files.

  1. In reading_client.py, make_session() returns a new requests.Session() whose default headers also say accept: application/json. Lesson 6 builds the client on it.

  2. In side_by_side.py, who_am_i(base_url, user, password, *, session, timeout=TIMEOUT) logs in and asks who you are, on the session it is given:

    • POST base_url + "/login" with auth=(user, password) and timeout=timeout;

    • response.raise_for_status(), so a refused login raises HTTPError before anything else is sent;

    • GET base_url + "/me" with timeout=timeout;

    • return response.json()["user"].

Use the session you are given for both requests. Do not make another, and do not call requests.post directly: the cookie from the login lives in that session’s jar, and only that session sends it back.

Then run the helper:

python pl_http.py requests-session

pl_http starts your server and gives who_am_i a session of its own. That session sends to your server on this machine through a handler pl_http mounts on it, so the name books.test is never looked up. Before you write anything, each call ends in raised NotImplementedError: not written yet: Lesson 1. After:

--- requests and replies ---
request 1  POST /login  ->  200 OK  (connection 1)
request 2  GET /me  (with a cookie line)  ->  200 OK  (connection 1)
--- your who_am_i() on pl_http's session ---
value: 'ana'
--- requests and replies ---
request 1  POST /login  ->  401 Unauthorized  (connection 2)
--- your who_am_i() on a second session, with a wrong password ---
raised HTTPError: 401 Client Error: Unauthorized for url: http://books.test/login
--- requests and replies ---
--- the cookies in the first session's jar ---
session: 'domain books.test, path /, HttpOnly'

Read it top to bottom. The /me request carried a cookie line you never wrote; the jar shows the cookie your server set, with the attributes from Chapter 7. Both requests of the first session went out on connection 1. The second session, with a wrong password, opened connection 2, got 401, and raise_for_status() turned that into an HTTPError, so /me was never asked. The result block ends with serve() returned after 2 connections; the server is closed.

In the transcript, POST /login and GET /me both say (connection 1). Why one connection for two requests?

Press Check my work. Lesson 2 maps the rest of requests’ features to the code you wrote by hand, and finds one place where auth= and your own Basic header disagree.

Task

Make a session that asks for , and log in through the session you are given.

  1. In reading_client.py, make_session() returns a new requests.Session() with accept: application/json added to its default headers. Add to the headers it already has (session.headers["accept"] = "application/json"); do not replace them with a new .

  2. In side_by_side.py, who_am_i(base_url, user, password, *, session, timeout=TIMEOUT):

    • sends POST base_url + "/login" with auth=(user, password) and timeout=timeout;

    • calls raise_for_status() on that reply, so a refused login raises HTTPError and nothing more is sent;

    • sends GET base_url + "/me" with timeout=timeout;

    • returns the user from that reply’s JSON.

Send both requests on session, with session.post, session.get, or session.request. Do not make a session of your own, do not call requests.post or requests.get, and do not pass a yourself: the session’s jar sends the login cookie back. Keep import requests as it is.

In the , python pl_http.py requests-session runs your who_am_i on a session pl_http mounts on your server, then on a second session with a wrong password, and prints the cookies in the first session’s jar.

Then press Check my work. It calls make_session() and reads the session’s headers. It runs who_am_i with a user, a password, a base , and a timeout you have not seen, on a real requests session whose requests are answered by a small server inside the check, in memory: the login must succeed, /me must carry the cookie, every request must carry the timeout you were given, and a wrong password must raise HTTPError before /me is asked. It also runs who_am_i on a session that only writes down each call, to see that the credentials go in auth= and no cookie is passed by hand. Nothing from your terminal is graded.