Put Text in a Page, Not Markup

The reading list has a page now. GET / lists the books of the logged-in reader as HTML. The fourth part of python pl_sec.py injection stores a book with the title <b>Dune</b> and prints two lines of the page:

--- a page: a book titled <b>Dune</b>, and the rows GET / writes for it ---
POST /tokens, with ana's cookie       -> 200
POST /books, with the token           -> 201
GET /, with ana's cookie              -> 200
  b'<span class="title"><b>Dune</b></span> by <span class="author">Frank Herbert</span>'
  b'<button aria-label="Delete <b>Dune</b>">Delete</button>'
the title reached the page as markup: a browser would read <b> as a tag

validate_book accepted that title, and it was right to: the title is 11 characters of text. In a JSON answer it stays text. In the page, a parser reads it as HTML.

Read the page once

Open page.html. You write no HTML in this course, and five things are enough to read this file:

• A tag such as <li> starts an element, and </li> ends it. What stands between them is the element’s text.

• A tag can carry attributes: a name, an equals sign, and a value in double quotes, as in name="id".

• Five markers in braces: {user}, {title} twice, {author}, and {id}.

• Two lines are comments, which a parser skips. They hold the words row and end row. The lines between them are one book, and they are repeated for every book.

• The <form>, which has its own section below.

render_page() fills the markers through fill(), which uses str.format_map. It reads the template once and never reads a value again, so a title of {author} stays that text. Two replace() calls, one for each marker, would fill that title with the author’s name.

Text, not markup

html.escape() replaces five characters that an HTML parser can act on. Each becomes a code that a browser shows as the character and never acts on:

import html

print(html.escape("<b>Dune</b>"))

It prints <b>Dune</b>. A browser that reads this shows <b>Dune</b> on the screen and starts no element. The ampersand becomes &, the double quote ", and the single quote '.

import html

print(html.escape('<b>"Tom" & Jerry</b>'))

Escaping is enough in two places: text between tags, and the value of an ordinary attribute inside double quotes, such as value or aria-label. page.html uses only those two. The title appears in both: between the <span> tags, and inside the aria-label of the button, a label for screen readers. Inside the quotes, the dangerous character is the quote itself. A title of " onclick="x would end the value and start an attribute of its own. html.escape replaces quotes unless you pass quote=False, so do not pass it. It is not enough for a URL in an href, because javascript:alert(1) holds none of the five characters. It is not enough for an attribute that a browser reads as code, such as onclick or style, even inside double quotes: the browser undoes the escaping before it runs the value. And it is not enough for anything inside <script>, or for an attribute without quotes.

What this prevents is a script of the attacker’s that runs in a page your reader trusts. That attack is called cross-site scripting, XSS for short. This course cannot show you one, because no browser opens the page in your workspace. Content-Security-Policy is a second line of defense, and Chapter 10 adds it.

Once, and on the way out

Store the title as the client sent it, and escape it when you write HTML. Escaping on the way in would put < into your JSON answers, where nothing reads it as a name. And a value that is escaped twice shows its own escaping. The first pass turns & into &. The second pass escapes the & of that name too, and the reader sees & on the page.

So there is one place for it: render_page, where text becomes HTML. This is Lesson 3’s rule once more. The defense belongs where the text is used.

The delete form

Every row has a form with a hidden id and a button. When a reader presses the button, the browser sends POST /books/delete with content-type: application/x-www-form-urlencoded, the body id=1, and the session cookie.

That body has the shape of a query string, so parse_qs reads it and percent-decodes each value. form_fields(request) came with this chapter’s project, and it takes the first value of every field without a second look. You make it strict in two ways. The media type must be the form type. And a field that appears twice is refused, because code that checks the first value and acts on the last one is a classic way for two parts of a program to disagree.

One thing to notice before you go on: nothing in this request proves that the reader pressed the button on your page. Chapter 8 starts there.

Your task

In render_page, pass every value through html.escape() once, before fill(): the user, each title, each author, and each id as text, str(book["id"]).

Rewrite form_fields(request):

1. Raise ValueError("send the form as application/x-www-form-urlencoded") unless media_type(request) == FORM_TYPE.

2. Decode the body as UTF-8, and read it with parse_qs(text, keep_blank_values=True), so that an empty value is kept. You can also pass strict_parsing=text != "" and max_num_fields=20, which refuse a malformed body and a very long one. A body that is not UTF-8 makes decode raise UnicodeDecodeError, which is a ValueError too.

3. Raise ValueError(f"the field {name} is repeated") when a field has more than one value.

In delete_from_form, answer json_error(415, ...) first when the media type is not the form type, as the JSON routes do. Then call form_fields inside try, and answer json_error(400, str(problem)) when it raises ValueError.

helpers.form_request(target, fields, headers) builds a form post like the browser’s, and fields may be a list of pairs. Try a repeated field once, with a session cookie from steps.login_cookie:

python -c "import sys; sys.path.insert(0, 'tests'); import helpers, steps, reading_server; s = reading_server.sample_state(); c = steps.login_cookie(s, 'ana', 'correct horse'); r = helpers.form_request('/books/delete', [('id', '1'), ('id', '2')], {'Cookie': c}); print(reading_server.handle(r, s)['status'])"

It prints 400.

python -m pytest -q ends with 23 passed, and the scenario’s last part reads:

  b'<span class="title">&lt;b&gt;Dune&lt;/b&gt;</span> by <span class="author">Frank Herbert</span>'
  b'<button aria-label="Delete &lt;b&gt;Dune&lt;/b&gt;">Delete</button>'
the title reached the page as text
--- result ---
0 of 4 attacks worked

Then press Check my work. It reads your page with Python’s HTML parser, which is not a browser. The validator and four guards stand now. Two of the five have an attack test, and the project writes the other three.