Test Public Behavior

Fundamentals I ended a chapter with loose assertions written underneath the code they checked:

assert double(4) == 8

That works, and it is still the heart of what you are about to do. What it lacks is somewhere to live. Assertions written under the code run every time the program runs, disappear when you tidy up, and stop at the first one that fails.

A test is the same assertion given a name and a home.

A test is a function whose name starts with test_

def test_a_correct_response_scores():
    question = Question("Which keyword starts a function?", "def", 2)

    assert question.is_correct("def")

That is the entire form. No new syntax: a function definition and an assertion, both of which you already know.

What is new is the convention. The name matters to something other than you. Python Land collects functions in your test files whose names begin with test_ and runs each one separately, so a failure in the first does not hide the rest.

Tests live in their own files, and those names are a convention too. A file counts as a test file when it is called tests.py, or starts with test_, or ends with _test.py. This chapter uses one file per idea: test_question.py holds the tests for Question.

Run tests is a separate button

The editor now has two actions.

Run executes your program, the way a person using it would.

Run tests imports your test modules and their dependencies, then runs every test_ function and reports each one. It does not start the application through its guarded entry point, but unguarded top-level code would still run during those imports. It also does not complete the lesson. Submit still does that, and Submit still has checks of its own that you cannot see.

So Run tests is genuinely yours. It exists to tell you whether the code does what you believe it does, before anyone else asks.

Test the behavior, not the machinery

Here is the distinction the rest of this chapter rests on.

Question promises something: give it a response, and it will tell you whether that response is correct. That promise is its public behavior, and it is what other code depends on.

Underneath, is_correct happens to call .strip() and .lower(). That is machinery. It could be rewritten tomorrow to compare differently, and as long as the promise holds, nothing that uses Question would notice.

A test that checks the promise:

def test_capitalization_does_not_matter():
    question = Question("Which keyword starts a function?", "def", 2)

    assert question.is_correct("DEF")

A test that checks the machinery:

def test_is_correct_calls_lower():
    ...

The first test survives a rewrite. The second one breaks during a rewrite that changed nothing anyone can observe. A test that fails when nothing is wrong is worse than no test, because you learn to ignore it.

Chapter 5 drew this line already, between the attributes a class publishes and the ones it keeps to itself. Tests are where that line starts paying you back.

Writing your first one

Everything in this chapter builds on one quiz application, already written and working. It deliberately uses the earlier incremental Quiz, which starts empty and receives questions through add(), because its state changes are useful testing material. Chapter 12’s capstone used a different contract: its runnable Quiz received a complete question list at construction and refused an empty one. Treat this chapter’s seed as a related application with an explicit different boundary, not as the same class silently changing its promise.

Your job this chapter is not to build the application. It is to find out what it actually promises, write those promises down as tests, and then use them to change the code safely.

Start with the smallest object in it.

Question takes a prompt, an answer, and a number of points. Its one method, is_correct, compares a response to the answer while ignoring surrounding spaces and capitalization.

Three promises are worth pinning down: a matching response is correct, a different response is not, and neither spacing nor capitalization changes the verdict.