Bits Inside a Byte

Once the server has answered 101, the handshake is over and the connection is a plain stream of bytes again. From here on, both sides send frames: small packages of bytes, each with a header that says what kind of message it carries and how long it is. RFC 6455 describes that header in section 5.2. Its first byte, drawn here after the RFC’s diagram, already holds five separate fields:

the first byte of every frame

+-------+-------+-------+-------+-----------------------+
|  FIN  | RSV1  | RSV2  | RSV3  |    opcode (4 bits)    |
+-------+-------+-------+-------+-----------------------+
  1 bit   1 bit   1 bit   1 bit

In words: the leftmost bit is a flag called FIN, the next three bits are three more flags called RSV1, RSV2 and RSV3, and the last four bits together form one small number, the opcode. Eight bits, five fields. Indexing a bytes value gives the whole byte as one number from 0 to 255, and nothing so far takes that number apart. This lesson adds a way to write bits and five operators that work on them.

Writing bits

Python writes one whole number in three ways. 129 is decimal, as usual. 0b10000001 is binary: the 0b prefix says that every digit after it is a bit, worth 128, 64, 32, 16, 8, 4, 2 and 1 from left to right. 0x81 is hexadecimal, or hex: the 0x prefix says that every digit after it is one of sixteen values, written 0 to 9 and then a to f for 10 to 15. Protocols like hex because each hex digit is exactly four bits, so two digits make one byte. In 0x81, the 8 is the bits 1000 and the 1 is the bits 0001, side by side.

Underscores may separate digits for readability: 0b1000_0001 is the same number. int reads binary text when you give it base 2, and hex() writes a number in hex:

The last line shows a method you will meet in every lesson of this chapter: hex(" ") on a bytes value writes each byte as two hex digits, separated by spaces. Protocol documents, RFC 6455 included, write bytes in hex, and this chapter writes every frame that way; its program prints yours the same way.

Test and extract with &

a & b keeps a bit only where both numbers have it set. Used with a fixed number, called a mask, it picks out the bits you want and clears the rest. n & 0x80 keeps only the top bit of a byte, so it is 128 when that bit is set and 0 when it is not. n & 0x0F keeps the low four bits, which is exactly where the opcode sits.

To ask whether a flag is set, compare the result with 0. In Python, & binds tighter than == and !=, so n & 0x80 != 0 means (n & 0x80) != 0.

print(0x85 & 0x80, 0x85 & 0x7F)

Move with << and >>

n >> k shifts the bits of n to the right by k places. The k lowest bits fall off the end, so 0x81 >> 4 drops the low four bits 0001 and leaves 1000, which is 8. n << k shifts to the left and fills in zeros, so 1 << 7 is 128, a single bit in the top position.

Together, a shift and a mask read one bit at any position: (n >> 3) & 1 moves bit 3 to the bottom and keeps only it, giving 0 or 1. Counted this way, from 0 at the right to 7 at the left, position 7 is the top bit.

Combine with | and flip with ^

a | b sets every bit that is set in either number. It combines fields that do not overlap: the flag 0x80 and the opcode 0x01 make 0x80 | 0x01, which is 0x81.

a ^ b is exclusive or. Every bit that is set in b flips in a: a 1 becomes a 0 and a 0 becomes a 1. Flipping the same bits twice gives back what you started with, so x ^ k ^ k is always x. Chapter 4 relies on exactly that.

The second line flips one bit of 0x81, then all eight.

print((0x48 ^ 0x37) ^ 0x37)

Your task

Open ws_frames.py. It is the chapter’s learning codec: a module that will turn messages into frames and frames back into messages, one lesson at a time. Each function’s docstring names the lesson that writes it. The tree also holds ws_handshake.py, the course’s finished version of Chapter 2’s handshake, so this chapter starts from a known file whatever your own Chapter 2 code looked like. Nothing here edits it. Write the four Lesson 1 helpers:

• bit_string(n) returns the 8 bits of n (0 to 255) as text, highest bit first, so bit_string(0x81) is "10000001". Build it inside bit_string itself with a loop over the positions 7 down to 0 (range(7, -1, -1)), taking each bit with >> and &, and adding "0" or "1" to the text. Do not use bin() or a format code: writing the loop is the exercise.

• nibbles(n) returns the high and low four bits as {"high": ..., "low": ...}. Four bits are often called a nibble, half a byte.

• join_nibbles(high, low) does the opposite and returns the byte. Raise ValueError unless both halves are 0 to 15.

• flip_bits(n, mask) returns n with every bit that is set in mask flipped.

Press Run. Run in this chapter starts frames_demo.py, a short program written for the course that prints one section per lesson on the RFC’s own example bytes; a function that is still a stub prints not written yet. Read it: the last two lines call show_lessons() only when the file is started as a program, the if __name__ == "__main__": guard. With your helpers written, the output begins:

Lesson 1: 0x81 in bits
{'bits': '10000001', 'nibbles': {'high': 8, 'low': 1}}
Lesson 2: the first two bytes of Hello
  not written yet

Then press Submit. The check calls each helper on bytes you have not seen and reads the code of bit_string for >> and & and for the shortcuts the task rules out. In the next lesson, these operators read the first two bytes of a real frame.