🏆Capstone project: Project: A Complete Frame Codec

Every rule of this chapter now lives in ws_frames.py, but in separate functions. Masking is in encode_frame and decode_frame, the control-frame limits in decode_frame, the close payload in parse_close_payload, and fragments are refused nowhere yet. A program that reads from a connection does not want to call five functions and check opcodes itself. It wants one call that answers “what did the other side just send?”. This project writes that call, read_message, and gives every refusal in the codec the close code a connection would send for it.

What read_message returns

read_message(read_exact, *, role, max_size=MAX_SIZE) reads exactly one frame through read_exact and returns one of five events, each a dict:

role says which side is reading. A "server" reads its client’s frames, so it calls decode_frame with require_mask=True; a "client" reads its server’s frames, with require_mask=False. max_size goes on to decode_frame unchanged.

Not everything that goes wrong is the peer breaking a rule. When the connection ends in the middle of a frame, read_exact raises ConnectionError, and read_message lets it through unchanged. Chapter 5 reports that ending as 1006, the code for a connection that ended without a close frame, which is a different story from a protocol error.

Give every refusal its code

When a peer breaks a rule, the reader closes the connection with a code that says why. Until now every refusal in the codec has been a ValueError with one message. Give each one its code too, as a first value before the message:

try:
    raise ValueError(1002, "opcode 3 is reserved")
except ValueError as exc:
    print(exc.args == (1002, "opcode 3 is reserved"))
    print(exc.args[0], exc)

An exception keeps the values it was created with in its args attribute, a tuple, so a handler that catches the error as exc finds the code in exc.args[0] and the reason in exc.args[1]. This block prints True, then 1002 (1002, 'opcode 3 is reserved'): printing a ValueError with two values shows the whole tuple. Chapter 5 reads exc.args[0] to choose the close frame it sends.

Here is every refusal on the read path, eleven already in the codec and three that read_message adds:

Most of the codec’s refusals are 1002, protocol error: the peer broke a framing rule. A frame over the size limit breaks no rule, so it gets 1009, message too big. A fragment breaks no rule either: as Lesson 5 explained, the learning codec does not reassemble fragmented messages, and 1003, unsupported data, is the least wrong code for that limit of the codec. The errors on the encoding side, in encode_frame, reply_to_ping and close_payload, keep their one message. They report a mistake in your own program, not something a peer sent, so no close code belongs to them.

exc = ValueError(1009, "too big")
print(exc.args[0], exc.args[1])

Specific before general

Invalid UTF-8 shows up in two places: a text payload, and the reason of a close. In both, .decode("utf-8") raises UnicodeDecodeError, and read_message turns it into 1007, invalid payload data. For a text frame, put the decode in a try block with except UnicodeDecodeError: and raise ValueError(1007, ...) from the handler. For a close frame, do the same around the call to parse_close_payload.

Name UnicodeDecodeError in that handler, not ValueError. UnicodeDecodeError is a kind of ValueError, so a handler for ValueError would also catch the 1002 refusals that parse_close_payload raises for a one-byte payload or a code that may not travel, and they would come out as 1007. Naming the specific exception lets those refusals pass through untouched with their own code.

Two roles, one function

The same frame can be fine for one role and a protocol error for the other. A masked frame is exactly what a server expects and exactly what a client must refuse, and the role argument is what decides. Everything after the direction rule is the same for both sides: the limits, the close payload, the UTF-8 check and the fragment refusal.

Fragments before content

read_message adds its checks after decode_frame has returned a frame, so by then every framing rule has been applied: the direction rule, the control-frame limits and the size limit. The fragment check comes next, before the opcode decides which event to build. A continuation frame has opcode 0, which belongs to none of the five events. And a text frame with FIN clear holds only part of a message, which may stop in the middle of a character that takes several bytes in UTF-8; decoding it first would refuse it with 1007, blaming the peer’s text, when the real reason is the codec’s own limit, 1003.

Your task

In ws_frames.py:

1. Work through the table above: change each refusal in parse_header_bytes, read_length, decode_frame and parse_close_payload from ValueError(message) to ValueError(code, message) with the code the table gives. Rows 7 and 8 may share one raise.

2. Write read_message(read_exact, *, role, max_size=MAX_SIZE): call decode_frame with require_mask set for the role and max_size passed on; raise ValueError(1003, ...) for a frame with FIN clear or opcode 0; return the event for each opcode, raising ValueError(1007, ...) for text or a close reason that is not UTF-8. Let ConnectionError through.

Press Run. The Lesson 6 section feeds a server the RFC’s masked “Hello”, an empty masked ping and a masked close 1000, and prints the three events your read_message returns:

Lesson 6: what a server reads
[{'type': 'text', 'data': 'Hello'}, {'type': 'ping', 'data': b''}, {'type': 'close', 'code': 1000, 'reason': ''}]

Then press Submit. The check first runs a few unseen cases from Lessons 1 to 4. Then it walks the table row by row, sending read_message one frame that breaks each rule, and names the first function whose refusal reaches the caller without the right code. Then it reads every kind of event for both roles, delivered whole, one byte and two bytes at a time, two frames back to back, broken frames that must be refused with their code and a str reason, an oversized frame that must be refused before its payload is read, and a connection that ends mid-frame.

The learning codec is now complete for messages that fit in one frame, and nothing more: it does not reassemble fragments, and it opens no socket. Chapter 5 puts it on a real connection, between a server and a client in one program.