0%

Testing and Refactoring Object-oriented Programs · practice

Protect Behavior with Regression Tests

A regression is behavior that used to be right and quietly stopped being right. A regression test is the test that would have caught it.

They arrive in a program two ways.

The first is a bug report. Something was wrong, you understood why, and before fixing it you wrote a test that failed for that exact reason. The test now stands guard over the fix, and it will fail again the day someone reintroduces it.

The second is the one this chapter needs. You are about to change code that currently works, and you want to know if you break it.

Write down what it does before you change it

Next lesson finds a design problem in report.py. The lesson after that fixes it, which means rewriting a working .

Rewriting working code is where behavior goes missing. Not the behavior you are thinking about, which you will check by hand. The other kind: the empty quiz, the perfect score, the exact spacing in a line someone else’s code is parsing.

So before touching it, pin down what it does today:

def test_the_full_report_for_the_chapter_quiz():
    attempt = build_chapter_attempt()

    lines = report_lines(attempt)

    assert lines == ["Mina: 2 of 5", "Grade: F"]

That test makes no claim about whether the current output is good. It only records that this is what the program produces right now. That is enough to make a refactoring safe, and it is why the test can be written before you know what the new design will look like.

Tests written for this purpose are sometimes called characterization tests, because they characterize what the code does rather than assert what it should do.

Cover the cases you would not think to check by hand

The obvious case is the one you will notice breaking. The edges are the ones that go quietly:

def test_an_empty_quiz_still_reports():
    quiz = Quiz("Empty")
    attempt = Attempt(quiz, "Mina")

    lines = report_lines(attempt)

    assert lines == ["Mina: 0 of 0", "Grade: F"]

Nothing about this scenario is interesting until someone rewrites report_lines and computes the fraction before checking whether there is anything to divide by. Then it crashes, in a case nobody tried, for a user with an empty quiz.

Lesson 5 tested that percentage guards against this. This test is a different claim: that the guard is still reachable from the report. Both can be true, then one stops being true, and only this test notices.

You are about to refactor a function whose current output you think is slightly wrong. What should the regression test assert?

Do not pin more than you mean to

A regression test can overreach. Assert the exact text of every line and you have frozen the wording, so a legitimate improvement to the report now “fails” and someone has to decide whether the test or the change is wrong.

There is no formula for this, only a question worth asking: if this test fails one day, will I be glad or annoyed? Glad means it was protecting something. Annoyed means it was describing something incidental.

The two lines of report_lines are worth pinning exactly. They are what the program prints, and the next two lessons are going to move the code that builds them.

Task

Write test_report_output.py, recording exactly what the report produces today.

Import what you need from quizapp.models and report, then write three tests:

  • test_the_full_report_for_the_chapter_quiz: Mina scores 2 out of 5, and the whole is ["Mina: 2 of 5", "Grade: F"].

  • test_a_perfect_attempt_reports_an_a: Mina answers both questions correctly, and the list is ["Mina: 5 of 5", "Grade: A"].

  • test_an_empty_quiz_still_reports: a quiz with no questions at all produces ["Mina: 0 of 0", "Grade: F"] and does not raise.

Assert on the whole list in one go, not line by line. These tests are the safety net for the refactoring in Lesson 9, so they should fail loudly if any part of the output moves.