One Slow Viewer Stalls Everyone

Scoreboard v1 works, and it has a weakness you can read in scoreboard_server.py. apply_score takes the board lock, bumps the version and calls broadcast, which sends the new snapshot to every viewer, one after another, before the lock is released. Only after apply_score returns does scorekeeper_handler send its reply. So every reply to the scorekeeper waits until every viewer has been sent the update. This lesson asks what happens when one of those sends is slow.

When a send waits

You met the answer in Networks with Python: sendall returns once the operating system has taken your bytes into its buffers, not once the other program has read them. The library’s send works the same way underneath. While the peer keeps reading, the buffers drain and nothing waits. When a peer stops reading, its receive buffer fills, then your send buffer fills behind it, and only then does a send have nowhere to put the next bytes. From that moment the send waits until the peer reads again. A receiver that cannot keep up slowing down its sender this way is called backpressure.

Try it with a real client

The project files include pl_ws.py, a program written for this course that plays every client, which you ran in the last chapter. Its stalled-reader scenario starts your server, connects viewer R, a real client that completes its handshake and then never reads again, and has the scorekeeper send ten scores. In the terminal:

python pl_ws.py stalled-reader

A run on the site printed:

scenario stalled-reader: viewer R completes its handshake and then never reads again
server options: the library's defaults with close_timeout=1.0 (open_server is not written yet)
viewer R is registered and reads nothing from now on
the scorekeeper got 10 replies out of 10
the server finished 11 sends to R, which read none of them; R was still registered: True
the kernel's buffers between the server and R took every one of those snapshots
the server shut down

Eleven sends, the join snapshot and ten updates, and not one of them waited. The second line mentions open_server, which Lesson 4 writes; until then the program uses the library’s own settings. On a local computer, a loopback connection held about 550 KB before a send would wait: the library sent 6,325 snapshots to a client that read none of them before its send finally waited. Linux, and the site’s sandbox, will give other numbers, but they are often hundreds of kilobytes. A demonstration with a real stalled reader would have to send thousands of updates before anything slowed down.

A viewer slowed on purpose

On the internet a viewer on a weak phone connection fills those buffers quickly, and its sends then take as long as its network takes. To show that cost on one machine, the slow-viewer scenario connects viewer A and viewer S, a viewer this program slows down on purpose: every send from your server to S waits 0.3 s before it goes out. Then the scorekeeper sends three scores.

python pl_ws.py slow-viewer

The same run on the site printed:

scenario slow-viewer: viewer S is slowed down on purpose; every send from the server to S waits 0.3 s first
server options: the library's defaults with close_timeout=1.0 (open_server is not written yet)
viewer A joined; its first message was a snapshot with version 0
viewer S joined; its first message was a snapshot with version 0
the scorekeeper sent 1 point for Owls; its reply was a snapshot with version 1
the scorekeeper sent 2 points for Foxes; its reply was a snapshot with version 2
the scorekeeper sent 3 points for Herons; its reply was a snapshot with version 3
score 1: the reply came after the server finished sending version 1 to S
score 2: the reply came after the server finished sending version 2 to S
score 3: the reply came after the server finished sending version 3 to S
the server shut down
viewer A received versions [0, 1, 2, 3], then the server closed it with 1001
viewer S received versions [0, 1, 2, 3], then the server closed it with 1001

Every reply came after the slow send to S had finished. The transcript reports order, not time, because the order is what your code decides. The picture draws one score with time running down: the scorekeeper’s thread on the left, a viewer trying to join on the right. Watch how long the board lock stays held, and what both sides wait for.

In words: the scorekeeper’s thread takes the board lock, sends the update to A, then sends it to S, which takes 0.3 s, and only then releases the lock and sends its reply. A viewer that tries to join during that time waits for the lock too, and joins only after the release.

Everyone waits for the slowest

Count who pays for S. The scorekeeper does, because its reply comes after the whole broadcast. Every viewer registered after S does, because the loop reaches it only once S’s send is done. Any new viewer does, because add_viewer needs the lock that the broadcast holds. With a thousand viewers, the slowest one sets the pace for all of them, and a real server cannot choose which clients connect to it.

What could fix it

There are three ways out. The server could drop a viewer that is slow, which is harsh on a viewer that is only briefly behind. It could keep a queue of every update for each viewer and send them when it can, but a viewer that never catches up makes its queue grow without bound. Or it could keep only the newest snapshot for each viewer. A snapshot holds the whole board, so the newest one replaces every older one, and a slow viewer that skips a version misses nothing it needs.

The scoreboard takes the third way. Lesson 2 builds a box for one viewer that holds a single snapshot, the newest, and wakes the thread that will send it.