0%

Chapter 10 · capstone

Capstone: A Live Scoreboard

Capstone project: Write the Scoreboard Contract

Every part of the scoreboard exists: join snapshots, coalescing outboxes, and a token check. But all of it is that pl_ws.py, a program written for this course that plays every client, calls. Nobody could start it from a , nothing written down tells a client what to expect, and nothing says how a quiz ends. This chapter turns it into two programs someone else could run: a server and a client.

The project already holds the plumbing the course has taught you: a room check and two new keywords on open_server, two parsers, three new board entries (last_version, open and finished), a half-written scoreboard_client.py, tests/test_client.py, a README.md, and an outline in SCOREBOARD_PROTOCOL.md. The contract comes first: you write down what both programs promise, then make the server keep the one rule it adds.

What the contract must say

SCOREBOARD_PROTOCOL.md is for someone who will write a client and has never seen your code. Each of its eight headings answers one question that person would ask:

  • Paths: where do I connect, what must a scorekeeper send to be let in, and which answers can turn me away before the upgrade?

  • Messages: what does each of the three message hold, and how many points may a score carry?

  • Versions: who numbers the boards, and what do I do with one that is not newer than the last one I showed?

  • Joining: what arrives first?

  • Closing: which ends my connection, and when?

  • Bounds: how many connections may be open, how many scores make a quiz, and how long does a server run?

  • Exit codes: what do 0, 1 and 2 mean for each program?

  • Not in this version: what must I not expect?

Everything in it is something a client can observe. The server’s locks, outboxes and threads stay out, because no client could tell the difference.

Which of these belongs in SCOREBOARD_PROTOCOL.md?

How a quiz ends

make_board(teams, max_updates=10) stores the length of the quiz as board["last_version"]. The quiz is over once board["version"] reaches it, and then every viewer should receive the last snapshot and a close with 1000.

The obvious way is to shut the whole server down with code 1000 as soon as apply_score makes the last version. But shutdown() closes every open connection from threads it starts itself, not from each viewer’s own thread, so a viewer whose thread has not yet sent the last snapshot can be closed before it gets it. A version written that way ran the quiz night you will run later, in Lesson 5, on a local computer, not on the site, with three updates and a viewer S slowed down on purpose. In all three runs, viewer S received only version 0, as in these lines from the first:

run 1: viewer B received versions [0, 1, 2, 3], then the server closed it with 1000 'the quiz is over'
run 1: viewer S received versions [0], then the server closed it with 1000 'the quiz is over'

Viewer S never saw a score.

Chapter 8’s rule fixes this: each viewer’s own thread is the only thread that sends to it, and a close is a send too. So each viewer’s handler ends the quiz for its own viewer: it sends a snapshot from its outbox; if that snapshot’s version has reached board["last_version"], it calls connection.close(1000, "the quiz is over") and returns; its finally removes it from the registry.

Test the version of the snapshot you just sent, not board["version"]: the board can reach the last version while an older snapshot is still on its way, and closing then would skip the last one. Coalescing makes the rule safe, because nothing can replace the last snapshot. A viewer that joins after the quiz is over gets the last snapshot as its join snapshot and is closed at once.

apply_score keeps the other half: once the version has reached the last one, it raises ValueError("the quiz is over") before changing anything, which Chapter 9’s scorekeeper_handler turns into an error envelope.

In the picture, who sends each close?

The scorekeeper only offers the last version; each viewer's own thread sends it, closes with 1000 and leaves, the last to leave sets finished, and only then does the main thread stop the server, instead of a shutdown that could close the slow viewer before its last send.

In words: the scorekeeper’s thread only offers the last snapshot; each viewer’s thread, the slowed B’s included, sends it, closes with 1000 and leaves; the last to leave sets finished, and only then does the server stop.

Why does each viewer's own handler close its connection at the end of the quiz, instead of the server closing every connection at once?

The finished event

board["finished"] is a threading.Event, like each outbox’s event since Chapter 8. There, an event woke one viewer’s thread; this one tells the program that the quiz is over and no viewer still waits for its last snapshot. Lesson 2’s serve_scoreboard waits on it before it stops the server.

Whoever leaves the registry empty after the last version sets it: usually remove_viewer, when it removes the last viewer; or apply_score, after its broadcast, when no viewer is registered at all, because then nobody will call remove_viewer. Both read the version and the registry while holding board["lock"], so the two facts are true at the same moment. finished must never be set while a viewer is registered: that is the race above again.

A viewer whose network path died leaves only when keepalive notices, about three seconds later (Chapter 8), so the end can come that much later; Lesson 2’s deadline bounds the wait.

The last score is applied while two viewers are registered. When is finished set?

Your task

Fill in SCOREBOARD_PROTOCOL.md in your own words, then change apply_score, remove_viewer and viewer_handler so that the server keeps the rule above. Each change is a few lines around one : a version against board["last_version"].

In the terminal, run the tests:

python -m pytest -q
echo $?

Six assert False tests fail until Lessons 4 and 5, so echo $?, pytest’s exit status, prints 1. Then play a short quiz:

python pl_ws.py last-update

It builds your server for a quiz of two updates, joins viewers A and B (B slowed down on purpose) and sends three scores. A finished version, run on the site, printed:

scenario last-update: a quiz of two updates; viewers A and B join, B slowed down on purpose; the scorekeeper scores three times
server options from your open_server: ping_interval=1.0, ping_timeout=1.0, close_timeout=1.0
your open_server installs a request check, so the scorekeeper sends the token from token.txt
viewer A joined; its first message was a snapshot with version 0
viewer B joined; its first message was a snapshot with version 0
the scorekeeper sent 3 points for Owls; its reply was a snapshot with version 1
the scorekeeper sent 2 points for Foxes; its reply was a snapshot with version 2
the scorekeeper sent 5 points for Herons; its reply was an error with version 2: 'the quiz is over'
viewer A received versions [0, 1, 2], then the server closed it with 1000 'the quiz is over'
viewer B received versions [0, 1, 2], then the server closed it with 1000 'the quiz is over'
finished was set 1 time(s); the registry held 0 viewers the first time
finished was set only after both viewers had left: True
the server shut down

Before your change, the third score made version 3 and finished was never set. Now it gets an error envelope at version 2, both viewers get the last snapshot and then 1000, and finished goes up once, after both have left.

The server now knows how a quiz ends, but it still runs only inside pl_ws.py. Lesson 2 gives it a command line and a deadline.

Task

In SCOREBOARD_PROTOCOL.md, keep every heading and replace every (fill in) with sentences of your own. Under Paths, name /watch and /score and the statuses 401 and 503; under Messages, the snapshot, score and error and the points , 1 to 10; under Closing, the codes 1000, 1001, 1008 and 1011; under Bounds, the options --max-clients, --max-updates and --deadline with their largest values, 5, 50 and 60 seconds; under Exit codes, what 0, 1 and 2 mean.

In scoreboard_server.py:

  1. apply_score: inside the lock, before anything changes, raise ValueError("the quiz is over") once board["version"] has reached board["last_version"]. After the broadcast, if the version has now reached the last one and board["viewers"] is empty, call board["finished"].set().

  2. remove_viewer: after the removal, still reading the board under the lock, set board["finished"] on the same .

  3. viewer_handler: after sending a snapshot whose own version has reached board["last_version"], call connection.close(1000, "the quiz is over") and return. Keep the finally that calls remove_viewer.

Keep every other as it is.

Run python -m pytest -q (six assert False tests fail until Lessons 4 and 5) and python pl_ws.py last-update. Then press Check my work.

Check my work runs your server with stand-in connections and no thread, replacing scoreboard_server.threading with its own locks and events, so keep import threading and make locks only inside functions. It runs a viewer once for every moment at which another thread could run, with one and with three scores arriving there: in a quiz far from over, the viewer must get the join snapshot and then only newer boards, never sent under a lock; in a quiz that ends at those scores, the viewer must be sent the last snapshot, then be closed once with 1000, with nothing sent after the close, and finished must be set exactly when the quiz is over and no viewer is registered. It checks a viewer that joins after the end, a failed, a dead and a crashing viewer, the scorekeeper’s answers (a score after the last version gets an error envelope and changes nothing), and Chapter 9’s token check with a token you have not seen. From SCOREBOARD_PROTOCOL.md it reads only the headings, the absence of (fill in), and the facts named above; the rest of the wording is yours. Any close reason is fine.