A Fingerprint for a Body

A client that kept yesterday’s book list wants to know one thing: is the list the server would send now the same as the one it holds? The direct way to find out is to compare the two bodies, and that needs the server’s body, which is the download the client was trying to avoid. What the two sides need is something much smaller that stands for the bytes: a value that is the same whenever the bytes are the same, and different whenever any byte differs. Then the client can send that value, and the server can compare it with the value of what it would send now.

That value is a fingerprint of the body, and Python computes one with a module from the standard library.

The hashlib module

hashlib turns any bytes into a digest: a short value of fixed size, computed from every byte. hashlib.sha256(body) reads the bytes in body and returns a hash object, and the object gives you its digest in two forms:

• .hexdigest() returns a str of 64 hexadecimal characters, the digits 0 to 9 and the letters a to f. Text like that can go into a header as it is.

• .digest() returns the same value as 32 raw bytes. Each byte is written as two hex characters in the other form, which is why 32 bytes become 64 characters.

SHA-256 is the name of the algorithm. Whatever the size of the input, an empty body or a whole library, the digest has the same size. Two properties make it useful here. The same bytes always give the same digest, on any machine and in any run of Python. And a change of a single byte gives a digest that has nothing visibly in common with the old one. The digest only identifies the bytes: nothing in its 64 characters lets anyone recover the body. Here are two bodies that differ by one space:

import hashlib

for body in [b'{"books": []}', b'{"books": [ ]}']:
    print(hashlib.sha256(body).hexdigest())

e72c4626ea5e221c238f01d50feff3d7007992fb3447610eaee3ee661310f6af
2d0190243f5b7e60660f91e4aace542f2e2a0de3c2dc923a01d05263c7333624

A JSON parser reads both bodies as the same data. The digests still differ, because a digest sees bytes, not meaning. That is the right behavior for a fingerprint of a reply: the client holds bytes, and the question is whether the server would send the same bytes.

hashlib.sha256 takes bytes, never a str. Your server’s bodies are already bytes, built by json_response with .encode("utf-8"), so they can go in directly.

import hashlib

print(len(hashlib.sha256(b"x").hexdigest()), len(hashlib.sha256(b"x").digest()))

A tag is quoted

In HTTP this fingerprint is called an entity tag, sent in a header named ETag. Its value has a fixed shape (RFC 9110, section 8.8.3): the tag’s characters between double quotes, and the quotes belong to the value. A server whose digest is e72c...f6af sends:

etag: "e72c4626ea5e221c238f01d50feff3d7007992fb3447610eaee3ee661310f6af"

A tag written exactly like that is a strong tag: it promises that two responses with the same tag have the same bytes. A tag can also start with W/, as in W/"e72c...". That marks a weak tag, which promises less: the two responses mean the same thing, though their bytes may differ, for example a page whose only change is a timestamp in a comment. Your server’s tag comes from the exact bytes, so it is strong, and it never sends W/. In Lesson 4 your server learns to read W/ when a client sends it back.

Not Python’s hash()

Python has a built-in hash() too, and it is tempting here. It returns a whole number for a value, and equal values in one program get equal numbers. But for str and bytes the number changes from one run of Python to the next. Run python -c 'print(hash(b"abc"))' twice in your terminal and the two numbers differ, unless Python was started with the environment variable PYTHONHASHSEED set to a fixed value, which pins the number. That is deliberate: it makes it harder for anyone to build input that slows down Python’s dictionaries.

A tag has to outlive one run. A client may keep a tag for hours while the server restarts, and an old tag must still name the same bytes when it comes back. A SHA-256 digest does, because it depends on nothing but the bytes.

Your task

In reading_server.py, write make_etag(body). It takes the body bytes and returns the strong tag as a str: a double quote, the SHA-256 hex digest of the body, and a double quote. reading_server.py already imports hashlib. One line does it.

Nothing calls make_etag yet; Lesson 3 puts its tag on every reply. You can try it now. Start Python in the terminal, in the project folder:

python

Then import your module and call the function on two bodies that differ by one byte:

>>> import reading_server
>>> reading_server.make_etag(b"Kindred")
'"213c649df845274f0590aae4a97f84eb72a8935662c6fb10c7a63bda2e69549f"'
>>> reading_server.make_etag(b"Kindred!")
'"fc29d1d0143aae9c48c2ab38f15c87e99a4345712343322b8d0d8924a63ce456"'

The outer single quotes are how Python shows a str; the double quotes inside them are part of the tag. One added byte gives an unrelated tag, and each tag is 66 characters with its quotes. Type exit() to leave Python, then press Check my work.

The same tools come back in a later course in this track, WebSockets with Python, which is planned. Its opening handshake hashes a key with hashlib and sends the digest() bytes in base64, the encoding you learned in Chapter 7. Here, the next lesson sends your tag on every reply your server gives to a read.