0%

Testing and Refactoring Object-oriented Programs · practice

Read Failures and Notice Coupling

Run tests reports four different things, and telling them apart saves time.

Passed. Nothing to do.

Failed. An assertion was false. The behavior is not what the test said it should be, and the report names the test, the file, the line, and your message. This is the useful kind: it is the machine telling you something specific about your program.

Error. The test raised something that was not an assertion. A , an AttributeError, a name that does not exist. The test started but could not finish normally. The cause may be in its setup or in the application: a report that divides by zero for an empty quiz is an application bug. Read the to locate the operation that raised before deciding what to fix.

Collection error. A whole file could not be loaded: a , or an import of something that is not there. Nothing in that file ran. Fix this first, because until you do, the tests in that file are neither passing nor failing.

The order to work in follows from that: collection errors, then errors, then failures. A single missing import can produce a page of alarming output that all disappears at once.

Read the message before the traceback

A failure looks something like this:

test_a_four_out_of_five_attempt_is_a_b
  test_grades.py, line 14
  AssertionError: expected Grade: B, got Grade: C

The last line is the one that tells you what happened. The line number tells you where to look. The traceback above it, when there is one, tells you how execution arrived there, and is mostly useful when the failure is an error rather than a failed assertion.

This is why Lesson 5 insisted on a message in a looping test. AssertionError on its own sends you back to read the test. expected Grade: B, got Grade: C often means you never need to.

When a test is hard to write, the code is telling you something

Here is the second half of this lesson, and it is a design idea rather than a testing one.

To check that a 4-out-of-5 attempt gets a B, you have to: build a quiz, invent questions whose points add up to 5 with 4 of them reachable, add them, build an attempt, answer some of them correctly and not others, then call the report and look at the second line.

Six or seven lines of arrangement to check one , and most of it is not about grades at all. It is about getting the two numbers 4 and 5 into place through a chain of .

Compare what the grade line actually depends on: a score and a total. Two numbers. Everything else in that setup exists because the only way to reach the grade line is through a fully assembled Attempt.

That gap has a name. The grade line is coupled to Quiz, Question, and Attempt, although the idea it expresses needs none of them.

A test needs eight lines of setup to check one short string. What is the most useful conclusion?

Pain is evidence, not proof

Be careful with this idea, because it is easy to overapply.

Some setup is honest. Lesson 4 arranged a quiz and an attempt to test report_lines, and that was right: the behavior under test really was about those objects working together.

The signal is not “this test has setup”. It is setup that has nothing to do with what you are checking. Questions and responses are not part of the idea “80% is a B”. When they show up in the arrangement anyway, they are there to satisfy the code’s structure rather than the behavior’s meaning.

Notice it, write the test anyway, and let it stand as the record of the problem. Lesson 9 does something about it.

Task

Write test_grades.py, checking the grade line in two bands the chapter quiz cannot reach.

Import what you need, then write three tests:

  • test_a_four_out_of_five_attempt_is_a_b: build a quiz worth 5 points where the learner can score exactly 4, answer accordingly, and assert report_lines(attempt)[1] is "Grade: B".

  • test_a_seven_out_of_ten_attempt_is_a_c: the same for 7 out of 10 and "Grade: C".

  • test_the_letter_alone_needs_no_quiz: assert grade_letter(0.8) is "B" and grade_letter(0.7) is "C", with no quiz, question, or attempt anywhere in the test.

Write the first two the honest way, through the report. Notice how much of the arrangement is about reaching the grade line rather than about grades. The third test is the same claim without any of that, and it is deliberately the odd one out: it costs almost nothing to write and proves nothing about the report.