Decode a Frame From a Stream

encode_frame turns a message into bytes. The receiving side needs the opposite: take bytes from the connection and get the message back. The problem is one you solved in Networks with Python Chapter 6. TCP delivers a stream, not packages, so a frame may arrive whole, cut into pieces anywhere, or glued to the start of the next frame. A decoder that assumes one recv holds one frame works on a quiet test machine and fails on a busy network.

Networks’ answer still holds: never ask the connection “what do you have?”, ask it “give me exactly n bytes”. A frame tells you, step by step, how many bytes to ask for next.

read_exact as a parameter

decode_frame(read_exact, *, max_size=MAX_SIZE) gets its bytes only through read_exact, the same kind of function read_length takes. It never sees a socket. That makes it easy to test with a stand-in, and the chapter supplies one in fakes.py. Open it and read its two functions once:

• pieces(data, size) cuts data into a list of chunks of at most size bytes, the way a busy connection might deliver them.

• reader_over(chunks) returns a read_exact(n) function that takes chunks from the list in order, keeps them in a buffer until it has n bytes, returns exactly those, and keeps the rest for the next call. When the chunks run out first, it raises ConnectionError, as a closed connection would.

So fakes.reader_over(fakes.pieces(frame, 1)) is a read_exact over a connection that delivers frame one byte at a time. That is what frames_demo.py hands your decoder in its Lesson 5 section. The one-byte pieces prove only that your decoder asks for exact counts and never assumes more has arrived; collecting the pieces into those counts is the reader’s job, the same receive buffer you wrote in Networks.

In order

A decoder reads a frame in the order its header describes it:

1. read_exact(2), the two bytes every frame has, which parse_header_bytes reads;

2. read_length(...), which reads 0, 2 or 8 more bytes depending on the 7-bit length;

3. read_exact(length), the payload.

Each step needs the one before it to know how much to ask for, so the order is fixed. Asking for more than one step at a time, such as 14 bytes to cover the longest header, would wait forever on a 7-byte frame like the RFC’s “Hello”, or swallow the start of the next frame. Reading exactly one frame also means a second frame right behind it stays in the buffer, intact, for the next call.

Refuse before reading

max_size is the largest payload this decoder accepts, by default MAX_SIZE, one million bytes. Networks with Python Chapter 8 set the same kind of limit on its frames, for the same reason. The 64-bit length lets a peer announce a payload of billions of bytes, and a program that tried to read it would wait for bytes that may never come and hold more memory than it has.

The header already says how big the payload is. So compare the length with max_size as soon as read_length returns, and raise ValueError before asking read_exact for a single payload byte. Checking after the payload has arrived is too late: the waiting and the memory are already spent.

A masked frame

One more refusal. Section 5.7 of the RFC shows “Hello” as a client would send it, masked:

81 85 37 fa 21 3d 7f 9f 4d 51 58

The second byte, 85, has MASK set, and the four bytes after it are the masking key. A server never masks its frames, so a client that receives a masked frame knows something is wrong and refuses it. This chapter’s decoder is the client’s side, and it has no way to unmask yet, so it raises ValueError for any frame whose "masked" field is True. Chapter 4 teaches masking and gives the decoder the server’s side too.

Your task

In ws_frames.py, write decode_frame(read_exact, *, max_size=MAX_SIZE):

1. read 2 bytes and pass them to parse_header_bytes, which raises ValueError for reserved bits and opcodes;

2. raise ValueError if the frame is masked;

3. get the payload length with read_length;

4. raise ValueError if the length is over max_size, before reading any payload;

5. read the payload and return {"fin": ..., "opcode": ..., "payload": ...}, with the payload as bytes.

Do not catch ConnectionError. If the peer closes in the middle of a frame, the caller needs to know.

Press Run. The Lesson 5 section decodes the RFC’s “Hello” fed to your decoder one byte at a time:

Lesson 5: Hello decoded one byte at a time
{'fin': True, 'opcode': 1, 'payload': b'Hello'}

Then press Submit. The check gives your decoder a stand-in read_exact that records every count it is asked for. It decodes the RFC’s frames, a fragment, binary frames of 256 and 65536 bytes with payloads you have not seen, and two frames sent back to back; it checks that an oversized frame is refused before its payload is asked for, that a masked frame, a reserved opcode and a non-minimal length raise ValueError, and that a close in the middle of a frame reaches the caller as ConnectionError.

The learning codec is complete for unmasked frames. The project that ends this chapter writes the tests that hold it to the RFC’s bytes.