Limit and Normalize Every Field

validate_book lets the right keys through. It says nothing yet about the values under them. Each of these three bodies comes back from it as a book:

{"title": "(200 spaces)", "author": "A"}
{"title": "T", "author": "A", "year": true}
{"title": "T", "author": "A", "tags": ["tag", "tag", (500 of them)]}

Send the second or the third to your server and it answers 201: a book whose year is true, and a book with 500 tags. Only the first is stopped, by the old book_error, which knows about blank titles and little else. HTTP with Python, Chapter 5 put a limit on the whole body, 10,000 bytes. All three bodies fit. Nothing limits a field inside a body that fits.

This lesson gives every field a rule, puts all the rules in validate_book, and retires book_error.

Normalize, then check, then use what you checked

Start with the title. Two rules come to mind: remove the white space around it with strip(), and allow at most 200 characters. The order of the two decides what gets stored.

The middle column gets both cases wrong, because it measured one text and stored another. So the rule has three steps, in this order:

1. Normalize. Bring the value into the one form you want to store.

2. Check the normalized value.

3. Use what you checked. The normalized value goes into the book. The value the client sent is not looked at again.

Two strings that look the same

Stripping is one kind of normalizing. Text needs a second kind. The sample book Café Society has an é in its title, and Unicode has two ways to write that letter: as one character, or as a plain e followed by a combining accent, a character that attaches itself to the letter before it. On screen the two look the same. In this block, ́ is the combining accent:

import unicodedata

typed = "Café Society"
stored = "Café Society"
print(typed == stored, len(typed), len(stored))
print(unicodedata.normalize("NFC", typed) == stored)

It prints False 13 12 and then True. The two titles are different strings of different lengths, so ==, in, and len all tell them apart, and a client may send either one.

unicodedata.normalize("NFC", text) returns the text in one agreed form, called NFC, in which a letter and its accent are joined into one character. unicodedata is part of the standard library and is already imported at the top of reading_server.py. This is the one new call of the lesson. For every text value, the first of the three steps becomes unicodedata.normalize("NFC", value).strip().

import unicodedata

a = "Café"
b = "Café"
print(a == b, len(a), len(b), unicodedata.normalize("NFC", a) == b)

A limit for every field

The numbers are constants near the top of the file: TEXT_LIMIT, FIRST_YEAR, LAST_YEAR, MOST_TAGS, and TAG_LIMIT, beside the set STATUSES. Two of the rules have a trap in Python.

The first is the year. JSON’s true arrives in Python as True, and isinstance(True, int) is True: a bool counts as an int, and True + 1998 is 1999. A test for int alone accepts it. So ask about bool first: if isinstance(year, bool) or not isinstance(year, int). The same line refuses the text "1999" and the float 1999.0.

The second is the status. status not in STATUSES looks complete. But a client can send a list, and ["done"] in STATUSES raises TypeError, because a list cannot be looked up in a set. A TypeError passes straight by your except ValueError, and the client’s mistake becomes a 500. Test isinstance(status, str) first. The status is not stripped: it is a code from a fixed list, not text a person types freely.

Each tag goes through the same three steps as a title, and the test for a repeat compares the normalized tags. So "x" and " x " are one tag twice, and so is one word sent in both Unicode forms.

Your task

In reading_server.py:

1. Write a helper first, clean_text(value, name, limit). It raises ValueError when value is not a str. It normalizes to NFC and strips. It raises ValueError when the result is shorter than 1 or longer than limit characters. Then it returns the result. name is for the messages, as in f"{name} must be text".

2. Grow validate_book. Keep the key checks of Lesson 1 at the top. Then build the new dict field by field, by the table: title and author must be there and go through clean_text; year, status, and tags are checked when they are there. Every refusal is a ValueError whose message names the field.

3. In book_request_error, remove the call of book_error. validate_book reports a missing title now. You can delete the function book_error too, because nothing else calls it.

Then try it in the terminal:

python -c "import reading_server; print(reading_server.validate_book({'title': '  Dune ', 'author': 'Frank Herbert', 'tags': [' sand ']}))"

It prints {'title': 'Dune', 'author': 'Frank Herbert', 'tags': ['sand']}: the book that will be stored is the one that was checked.

Two limits of all this. NFC makes text equal only where the difference was a letter joined with its accent or written apart. Letters from different alphabets that look alike stay different. A stronger form, NFKC, folds more and also changes what some text means, so a real service chooses with care. And a title may still hold any character, including a line break in the middle of it. Such a title passes every rule in this lesson.

Then press Check my work. The next lesson is about why a title that passed can still do harm.