Keep a File Name Inside Its Folder

A book can have a cover now. The covers are three small text files in the covers folder, and this chapter’s project serves them at GET /covers/{name}. The branch in route() starts like this:

            cover = pathlib.Path(state["covers_dir"]) / segments[1]
            if cover.is_file():
                body = cover.read_bytes()

It joins a name from the request to a folder and reads whatever file it finds there. Run python pl_sec.py injection and read the third part:

--- a path: a cover, and a name that climbs out of the folder ---
GET /covers/kindred.txt       -> 200  Kindred, cover notes.
GET /covers/..%2fSECURITY.md  -> 200  # Security findings: the reading list
a file that is not a cover was served: SECURITY.md, from your own project folder

The second request got the first line of SECURITY.md. The scenario stays inside your project folder on purpose, and nothing outside it is read. On a real server the same request reaches for a configuration file or a key. Reading a file outside the folder a route is meant to serve is path traversal.

How the name got there

In a path, .. means the folder above. A plain /covers/../SECURITY.md gets nowhere: that path has three segments, and the covers branch takes exactly two.

So the attack hides the slash as %2f. path_segments() splits the path at / first, and decodes each segment after:

def path_segments(target):
    """Return the decoded path segments of a request target."""
    segments = []
    for segment in urlsplit(target).path.split("/")[1:]:
        segments.append(unquote(segment))
    return segments

While the path is split, ..%2fSECURITY.md is one segment. Decoded, it is ../SECURITY.md. Splitting first is right, because an encoded slash can then never change which route a request takes. The name that arrives can still hold a slash, and joined to the folder it reads covers/../SECURITY.md.

Check the path, not the text

The first idea is to look at the text of the name. Here are three such checks, and the name that defeats each.

Refuse a name that holds ... An absolute name needs no dots, because joining it to a folder throws the folder away.

import pathlib

print(pathlib.PurePosixPath("/srv/covers") / "/etc/hosts")

Refuse a name that starts with a slash. Then ../SECURITY.md is back.

Join first, and accept the path when its text starts with the folder’s text. Now a folder named covers-old, next to covers, is open: the text /srv/covers-old/x.txt starts with /srv/covers.

Each of these checks asks about characters. The real question is about the file system: is this file inside that folder? pathlib can answer it. resolve() returns the absolute path with every .. worked out and every link followed. parents holds every folder above a path. So resolve the folder, resolve the joined path, and require the folder among path.parents. A link inside the folder that points at a file outside is resolved too, and refused.

One decode, in one place

The router already decoded the name. It is tempting to call unquote once more inside the function, to be safe. One of the covers shows what that costs. Its file name is rep ort.txt, with a space, so a client asks for /covers/rep%20ort.txt, and the router’s decode gives the file’s name. A request for /covers/rep%2520ort.txt names something else: %25 is an encoded percent sign, so the name that arrives is rep%20ort.txt, and no cover has that name.

from urllib.parse import unquote

once = unquote("rep%2520ort.txt")
print(once)
print(unquote(once))
print(unquote(unquote("%252e%252e%252fSECURITY.md")))

It prints rep%20ort.txt, then rep ort.txt, then ../SECURITY.md. The first line is the router’s result, the name the client asked for. A second decode turns it into rep ort.txt, and the server answers with a file the client did not name. It also works the other way: on a server where a file’s name holds a percent sign, a second decode changes that name, and the file can no longer be fetched.

The third line is worse. After the router’s decode, %252e%252e%252f is %2e%2e%2f: text with no dot and no slash in it. A check on the text finds nothing wrong, and then a second decode turns it into ../. The rule is the one from Lesson 2: check what you will use. The name is decoded once, by the router, and your function uses it as it arrives.

Your task

Write safe_cover_path(folder, name) in reading_server.py. It returns a path, or None:

1. Return None when "\x00" is in the name. /covers/a%00 delivers such a name, and resolve() raises ValueError for it.

2. base = pathlib.Path(folder).resolve(), and path = (base / name).resolve().

3. Return None unless base in path.parents.

4. Return None unless path.is_file(), so that the name of a folder gets a 404 too.

5. Return path.

A file in a folder inside covers passes all of these, and it is served. The rule is about the folder, not about the number of slashes in the name.

Then use it in the covers branch: cover = safe_cover_path(state["covers_dir"], segments[1]), and serve the file when cover is not None. The folder comes from the state, so a test can point the server at a folder of its own. You will do that in Lesson 8.

Ask for the cover with the space, and then for the name with the encoded percent sign:

python -c "import sys; sys.path.insert(0, 'tests'); import helpers, reading_server; print([reading_server.handle(helpers.make_request('GET', target), reading_server.sample_state())['status'] for target in ('/covers/rep%20ort.txt', '/covers/rep%2520ort.txt')])"

It prints [200, 404]. You can put name = unquote(name) at the top of your function for a moment and run the line again: [200, 200]. The second request now gets a file it did not ask for. Take the line out again.

python -m pytest -q still ends with 23 passed. The scenario’s third part now reads:

--- a path: a cover, and a name that climbs out of the folder ---
GET /covers/kindred.txt       -> 200  Kindred, cover notes.
GET /covers/..%2fSECURITY.md  -> 404  {"error": "no cover has that name"}
the name outside the folder was refused

The result line says 1 of 4 attacks worked. Then press Check my work. The one attack that is left puts a title into a page, and the next lesson stops it.