🏆Capstone project: Project: A Signing Module

This project finishes signing.py and gives it the tests that will guard it for the rest of the course. Chapter 5 builds tokens on this module, so it has to be right, and it has to stay right when someone edits it later.

The requirements:

• new_key(), sign(), and verify() behave as in Lesson 2.

• verify() makes its decision through hmac.compare_digest, on bytes.

• verify() returns False, and raises nothing, for a wrong tag made of any text a header can carry: a short one, an empty one, one with characters outside ASCII.

• tests/test_attacks.py holds four new attack tests with fixed names.

Four attacks, four tests

Each test plays one attacker and asserts that signing.verify(...) returns False, or in the last case that it compared the right way.

The forger has no key. They change the message and need a tag for it. The best they can compute is hashlib.sha256(message).hexdigest(), the plain digest of their new message. test_attack_forged_tag_is_refused hands exactly that to verify.

The truncator has seen a right tag and sends only its first characters, hoping the check looks no further. test_attack_truncated_tag_is_refused signs a message and cuts the tag short with a slice, such as tag[:8].

The outsider has a key of their own, made the same way as yours, and signs with it. test_attack_tag_from_another_key_is_refused makes two keys, signs with one, and verifies with the other. The message stays the same, so the key is the only thing that differs.

The fourth attacker is the one from Lesson 3, who learns from how the comparison works. That attack cannot be run here, because nothing on this site is timed. So the fourth test checks the thing that stops it. helpers.count_compare_digest() returns a dict whose "calls" entry counts every call of hmac.compare_digest from then on, while the real function still does the comparing. Sign a message first, then start the counter, then call verify with the right tag, and expect True and exactly one call. Start the counter last, right before the call you want to count. In later chapters the setup makes comparisons of its own.

That fourth test is different in kind from the other three. They test what an outsider could observe. This one looks inside and checks which function did the comparing. It is a test of how the code is written, not of what it lets through, and it is the only way to test this property without a clock.

Milestone 1: write the tests

Write the four tests below the comment for this lesson in tests/test_attacks.py. Reach the module as signing.new_key(), signing.sign(...), and signing.verify(...). Then run them:

python -m pytest -q

The run ends with 2 failed, 3 passed. Three of your new tests pass at once against the code from Lesson 2, which already refuses those forgeries. Two tests fail: the one from Chapter 1, which the top of the file lists as expected, and your comparison test, with assert 0 == 1. Zero calls: your verify still uses ==. If your verify already calls hmac.compare_digest, that test passes too.

To run one test, select it by a word from its name. The option -k keeps only the tests whose names contain that word:

python -m pytest -q -k forged

Milestone 2: make each test fail once

Three tests passed the first time you ran them. That is pleasant, and it proves little. A test that has never failed might be unable to fail.

So break the code on purpose and watch. In signing.py, change sign for a moment so that it returns hashlib.sha256(message).hexdigest() and leaves the key out. Run python -m pytest -q -k forged. The forged-tag test must fail now, because a forger’s plain digest verifies. Restore the line and see the test pass again. Do the same for the other two: a verify that compares only expected[:len(tag)] with the tag must make the truncated-tag test fail, and a sign that always uses one fixed key, such as b"k" * 32, must make the other-key test fail.

Check my work does the same thing with four broken versions of the module, one for each test, and expects each test to fail against its own.

Milestone 3: the comparison

Now make the last test pass. In verify, keep the recomputed tag in a variable, encode both tags with .encode("utf-8"), and return what hmac.compare_digest(...) returns for the two byte strings.

Run all the tests again with python -m pytest -q. The run ends with 1 failed, 4 passed, and the one failure is the Chapter 1 test. Then run the helper:

python pl_sec.py forged-message

Its last lines now read:

--- how verify() compared ---
hmac.compare_digest was called 5 times in 5 checks
--- result ---
5 of 5 forgeries refused

What does not change yet

SECURITY.md stays as it is. F2 is about the login, and the login in reading_server.py still compares a password with !=. This chapter built the tool. Chapter 3 uses it to close the row.

Press Check my work when all four tests pass. You now have two things every later chapter relies on: a tag that only the key holder can make, and a comparison that does not show where two values differ. Chapter 3 turns to the oldest secret in the service, the password.