Test Expected Exceptions

Chapter 12 argued that refusing to do something is part of what a method promises, not an accident. Question refuses to exist with fewer than one point. Attempt refuses to score a question that is not in its quiz.

Those refusals are behavior. They deserve tests as much as the successes do, and they need a different shape, because the thing you are checking is that the code stopped.

Why the obvious version does not work

def test_a_question_needs_points():
    Question("Which keyword?", "def", 0)

    assert ???

There is nothing to assert on. Either the call raised, in which case the test never reached the assertion, or it did not, in which case the promise was broken and the test still passes.

You could reach for try and except, and it works, but read what it takes:

def test_a_question_needs_points():
    try:
        Question("Which keyword?", "def", 0)
    except ValueError:
        return

    raise AssertionError("a question with 0 points should have been refused")

Five lines of plumbing around one line of behavior, and the return in the middle is the part that says “passed”. This is common enough to deserve a tool.

raises

Python Land gives you one:

from pythonland_test import raises


def test_a_question_needs_points():
    with raises(ValueError):
        Question("Which keyword starts a function?", "def", 0)

Read it as a sentence: this block raises ValueError. If it does, the test passes. If the block runs to the end without raising, the test fails. And if it raises something else entirely, that error comes straight through, because an unexpected error is a different problem from a failed expectation and should not be disguised as one.

raises comes from pythonland_test, a module this platform provides. You will not find it in a Python installed on your own machine. What it stands for is universal, though: every serious Python test tool has this, usually spelled almost exactly this way.

with, briefly

That with is new.

A with statement runs a block under something’s supervision. The object after with gets told when the block starts and when it ends, including when it ends because of an error, and it can decide what to do about that. raises uses the ending: it looks at how the block finished and judges it.

You will meet with again for files and connections, where the supervision is about cleaning up afterwards. Here the only thing to hold on to is the shape: with something, then an indented block, and the something gets a say in how the block turns out.

Checking which error, and what it says

The seeded quizapp/errors.py gives UnknownQuestionError a prompt attribute so a caller could report the problem properly. A test can look at it:

def test_answering_a_foreign_question_is_refused():
    quiz = Quiz("Review")
    stray = Question("Not in this quiz", "nope", 1)
    attempt = Attempt(quiz, "Mina")

    with raises(UnknownQuestionError) as caught:
        attempt.answer(stray, "nope")

    assert caught.value.prompt == "Not in this quiz"

Like except, raises accepts the named exception type and its subclasses. Use a specific subclass when that distinction is part of the promise.

as caught gives you a handle, and caught.value is the exception object itself, after the block has finished. From there it is an ordinary object: read its attributes, or pass it to str() for the message.

This is worth the extra line. “It raised something” is a much weaker promise than “it raised the error that names the question at fault”, and Chapter 12 built the second one on purpose.

Which refusals to pin down

The quiz application makes two, and they are of different kinds.

Question refuses invalid construction with a plain ValueError. It is a bad argument, nothing more, and Python already has a word for that.

Attempt.answer refuses a question from outside its quiz with UnknownQuestionError, which the application defined itself, because a caller might reasonably want to catch exactly this and report it. Test both, and for the second one, check that it carries the prompt.