Masked Frames and the Direction Rule

apply_mask turns a payload into masked bytes and back, but no frame carries a mask yet: encode_frame raises NotImplementedError as soon as it gets a key, and decode_frame does the same when it is asked to read as a server. This lesson replaces both of those branches. Here is the RFC’s masked “Hello” again, field by field:

81 85 37 fa 21 3d 7f 9f 4d 51 58

• 81: byte 0, unchanged by masking. FIN is set and the opcode is 1, text.

• 85: byte 1. 0x85 is 0x80 | 5, the MASK bit on top of the 7-bit length 5.

• 37 fa 21 3d: the masking key, present only because MASK is set.

• 7f 9f 4d 51 58: the payload, “Hello” masked with that key.

Only the payload is masked. The header and the length stay readable, because the receiver needs them to know how many bytes to read and whether a key follows.

MASK sits in byte 1

The MASK bit is the top bit of byte 1, the same byte that holds the 7-bit length. Chapter 3’s length_field(n) returns the length bytes in the shortest form, and its first byte is always that 7-bit field: the length itself up to 125, or the marker 126 or 127. Setting MASK means OR-ing 0x80 into that first byte and leaving every extended length byte after it unchanged. Then come the four key bytes, then the masked payload. With the key 01 02 03 04, these are the headers of a masked 256-byte frame, a masked 65,536-byte frame, and an empty masked text frame:

82 fe 01 00 01 02 03 04
82 ff 00 00 00 00 00 01 00 00 01 02 03 04
81 80 01 02 03 04

In the first, fe is 0x80 | 126, and the 16-bit length 01 00 follows as before. In the second, ff is 0x80 | 127, and eight length bytes follow. The empty frame still carries a key, though there is nothing to mask.

print(hex(0x80 | 126), hex(0x80 | 5))

Where keys come from

RFC 6455 section 5.3 asks the client for a fresh key for every frame, chosen so that no one can predict it. In Chapter 5 your client takes it from os.urandom(4), the standard library’s source of unpredictable bytes. encode_frame does not call it. It takes the key as the argument mask_key, the same way make_client_key(random_bytes=os.urandom) in Chapter 2 took its source of random bytes as an argument. A test can then pass a known key and compare the frame with known bytes, as frames_demo.py does with the RFC’s key. When mask_key is None, the default, encode_frame builds an unmasked frame exactly as before, which is what a server sends.

The direction rule

RFC 6455 section 5.1 states the rule in both directions. A client masks every frame it sends, and a server masks none. A server that receives an unmasked frame must close the connection, and a client that receives a masked frame must do the same. Each side reads frames from the other, so each side knows which kind to expect:

So decode_frame compares the frame’s MASK bit with require_mask and raises ValueError when they differ, in either direction. The default False keeps Chapter 3’s behavior, the client’s view, and a server that forgets require_mask=True refuses every frame its client sends: a loud mistake, found on the first message. The RFC says the receiver closes the connection. Which close code it sends is a question for Lessons 4 and 6; for now, the refusal is a ValueError.

Decoding a masked frame

A masked frame is read in the order its header describes it, one step more than in Chapter 3:

1. read_exact(2), read by parse_header_bytes;

2. the 0, 2 or 8 extended length bytes, read by read_length;

3. when MASK is set, read_exact(4), the key;

4. read_exact(length), the payload, unmasked with apply_mask(payload, key).

The key comes after the extended length, so reading it any earlier would take length bytes for the key. The size check stays where it was, right after read_length: a frame announcing more than max_size bytes is refused before the key and the payload are read.

Your task

In ws_frames.py, replace the branch at the top of each function that raises NotImplementedError:

• encode_frame(opcode, payload, *, fin=True, mask_key=None): with mask_key=None, return the unmasked frame as before. Otherwise raise ValueError unless the key is exactly 4 bytes, and return byte 0, then the length bytes with 0x80 OR-ed into the first of them, then the key, then the payload masked with apply_mask.

• decode_frame(read_exact, *, require_mask=False, max_size=MAX_SIZE): replace Chapter 3’s refusal of every masked frame with the direction rule, raising ValueError when the MASK bit is not what require_mask asks for. When the frame is masked, read the 4-byte key after the length check and return the payload unmasked.

Press Run. The two Lesson 2 sections build the RFC’s masked frame and read it back the way a server does, from a connection that delivers one byte at a time:

Lesson 2: a masked text frame
81 85 37 fa 21 3d 7f 9f 4d 51 58
Lesson 2: the server unmasks it, one byte at a time
{'fin': True, 'opcode': 1, 'payload': b'Hello'}
Lesson 3: the client's pong for the server's ping
  not written yet

Then press Submit. The check builds masked frames with keys and payloads you have not seen, of 0, 7, 256 and 65,536 bytes, and compares them byte by byte, header first. It reads masked frames as a server, delivered whole, one byte and two bytes at a time, and checks the order of the reads; it expects ValueError for a key that is not 4 bytes, for each way of breaking the direction rule, and for a masked frame over max_size, before its key is read. The unmasked frames from Chapter 3 must still work. Every frame so far is data; the next lesson adds the frames that keep a connection in order.