Accept Only What You Expect

The reading list grew for this chapter. A book may now carry three more fields: the year it came out, how far you are with it, and a list of tags. A request body can look like this:

{
  "title": "Dune",
  "author": "Frank Herbert",
  "year": 1965,
  "status": "reading",
  "tags": ["desert", "classic"]
}

To store fields like these, create_book in reading_server.py changed as well. This code is new, and it came with this chapter’s project. You did not write it:

def create_book(request, state, owner):
    """Store the body as a new book owned by owner and return the 201 response."""
    data = json_body(request)
    book_id = state["next_id"]
    state["next_id"] += 1
    book = {"id": book_id, "owner": owner}
    for key in data:
        book[key] = data[key]
    state["books"][book_id] = book
    response = json_response(201, book)
    response["headers"]["location"] = f"/books/{book_id}"
    return response

The loop copies every key of the body into the book. That is convenient, because a new field needs no new code. It also invites a mistake. The server sets id and owner first and the loop runs after that, so a key in the body replaces what the server chose. replace_book, which serves PUT, has the same loop.

Write the attack first

Chapter 6 made the owner the one fact that every read and every write is checked against. So try to choose it. Ben has an account that may write, and a token of his own. He posts a book and adds "owner": "ana" to the body.

Write that as a test, test_attack_body_cannot_set_the_owner, below the Chapter 7 comment in tests/test_attacks.py:

1. Build the state with steps.keyed_state(), and get ben’s Authorization value with steps.bearer_for(state, "ben", "battery staple").

2. Post an honest book with that token and assert that the status is 201. Without this step, a server that refuses every book would pass your test.

3. Post {"title": "Planted", "author": "Ben", "owner": "ana"} with the same token. Assert that the status is 422, and that no book in state["books"] has the title Planted.

Then run it. The option -k owner selects the tests with owner in their name:

python -m pytest -q -k owner

The run ends with 1 failed, 21 deselected, and this is the line that failed:

E       assert 201 == 422

The server answered 201 and stored the book with "owner": "ana". It now shows up in Ana’s list when she asks for GET /books. Ben, who made it, gets 404 for it. The same loop lets a body choose the id a book shows, or add a field that nobody planned, such as "is_admin": true.

This weakness is called mass assignment: the server assigns a whole body to a record in one step, and so the client decides fields that were never the client’s to set.

The attack needs a valid token that may write: Ben is a user of the service, not a stranger. Nobody else works in your workspace, so the test plays both parts.

A list of what is refused always misses one

The first fix that comes to mind names the key that caused the trouble:

if "owner" in body:
    raise ValueError("owner is set by the server")

That stops the test’s attack and leaves id open. A list of what is refused is called a denylist. It lets through everything it does not name, and that includes fields that do not exist yet.

A list of what is accepted

Turn the list around. An allowlist names what is accepted, and everything else is refused. Your project has one, as a set:

ALLOWED_FIELDS = {"title", "author", "year", "status", "tags"}

The test is key not in ALLOWED_FIELDS. A field that is added to books later stays out of requests until someone decides to let it in.

validate_book(body) applies the list. It raises ValueError("the body must be a JSON object") for a body that is not a dict. Then it looks at each key of the body, in the body’s own order:

• for owner or id, it raises ValueError(f"{key} is set by the server");

• for any other key that is not in ALLOWED_FIELDS, it raises ValueError(f"{key} is not a field of a book").

When no key was refused, it builds a new dict, copies the body’s keys and values into it, and returns the new dict. It never returns the body itself, so a key the validator did not read cannot reach the book.

You have seen this defense before. In HTTP with Python, Chapter 11, Lesson 3, a Pydantic model dropped the fields it did not name, “so a client cannot choose a book’s id”. That dropping was an allowlist, and it was the defense against mass assignment.

The model dropped those fields without a word. Your function refuses them, for two reasons. An honest client that sends a misspelled key learns about its mistake. And a tampering attempt becomes a 422, which the log from Chapter 6 records with the user’s name. A model can refuse too: Pydantic’s setting for that is extra="forbid".

Your task

The test is written and red. Now, in reading_server.py:

1. Write validate_book(body) as described above.

2. In book_request_error(request), after the checks that are there, call validate_book(json_body(request)) inside try. When it raises ValueError, the error to return is json_error(422, str(problem)). Leave book_error in place: it still reports a missing title.

3. In create_book and in replace_book, loop over validate_book(json_body(request)) in place of the body.

The second change serves POST and PUT alike, so both write routes close in this lesson. The status follows HTTP with Python, Chapter 5, Lesson 5. A 400 says the body could not be read, and this one was read. A 415 says its media type is wrong, and this one is labeled application/json. A 403 says the account may not write, and Ben’s may. The body was understood, and one thing in it is not acceptable: that is 422.

Run python -m pytest -q -k owner again: 1 passed, 21 deselected. Without -k owner, the whole file ends with 22 passed.

This lesson changes a contract on purpose. In HTTP with Python, a book body with an extra field was accepted and the field was ignored. From now on such a body gets a 422 that names the field.

Then press Check my work. The right keys now arrive, and nothing else does. What sits under those keys is still unchecked: a title of any length, a year of any type. That is the next lesson.