0%

When Things Go Wrong · practice

Wait Longer Between Attempts

Lesson 3’s demonstration stopped at the first failure because nothing said how long to wait. Trying again at once is a poor choice: an overloaded service that just failed is still overloaded a millisecond later, and if many programs retry at once they keep it overloaded. A program should wait between attempts, and wait longer each time it fails.

Waits that double

The usual rule is called backoff: wait BASE_WAIT seconds after the first failure, twice that after the second, four times after the third, and so on. With BASE_WAIT = 1 the waits are 1, 2, 4 and 8 seconds.

attempt counts the attempts from 1, so after attempt attempt fails, the wait is BASE_WAIT doubled attempt - 1 times. A loop does the doubling:

BASE_WAIT = 1
for attempt in range(1, 5):
    wait = BASE_WAIT
    for step in range(attempt - 1):
        wait = wait * 2
    print(attempt, wait)

Real systems also add a little randomness to each wait, so that many programs that failed at the same moment do not all come back at the same moment. The course leaves that out, so that tests can check the exact waits.

When the service says how long

A 429 usually tells you how long to wait. Its response carries a Retry-After header, and the scripted model sends Retry-After: 1. When a usable value is there, use it instead of guessing.

Only an error that came with a response has headers. Those are the openai.APIStatusError classes, such as RateLimitError and InternalServerError. Their response attribute is the HTTP response, and error.response.headers.get("retry-after") reads the header, or gives None when there is none. The lookup does not care about upper or lower case, so "Retry-After" works too. A timeout or a connection error has no response at all, so check the type first.

The header is text, and it is not always a number. The HTTP rules also allow a date, such as Wed, 21 Oct 2026 07:28:00 GMT, and a broken server can send anything. float turns "1" into 1.0 and "2.5" into 2.5, and raises ValueError for anything that is not a number. It also raises TypeError for None, which is why None is checked before float is called.

wait_seconds

wait_seconds(error, attempt) follows these steps, in this order:

  1. If error is an openai.APIStatusError, read value = error.response.headers.get("retry-after").

  2. If value is not None, convert it with float inside try. On ValueError the header is a date or nonsense, so ignore it.

  3. If the number is zero or more, return it. A negative number is not a usable wait.

  4. Otherwise, double: start with wait = BASE_WAIT, double it attempt - 1 times in a loop, and return it.

wait_seconds only returns a number. It does not wait. The waiting is done by a sleep function that the caller passes in, the same idea as the injected time in Python in Practice: in a real run the caller passes time.sleep, and a test passes a function that writes the number down and returns at once. The tests for Lessons 7 to 9 run in a fraction of a second although the code they check would wait for seconds.

Run it

The counted scenarios fail a fixed number of times. rate-limit:2 answers the first two requests with a 429, and every request after that with the normal reply. server-error:2 does the same with 500. The scripted model counts per sandbox, so the demonstration calls reset_scripted_model() first, which sets the counts back to zero. Without it, running python chapter9_demo.py a second time would find the counts already used up.

When wait_seconds is written, run python chapter9_demo.py. The demonstration’s retry_demo uses your should_retry and wait_seconds, and waits with time.sleep:

rate-limit:2 with retries:
  Attempt 1: RateLimitError
  Waiting 1.0 s
  Attempt 2: RateLimitError
  Waiting 1.0 s
  Attempt 3: A list is an ordered, changeable collection of values. You write one with square brackets, like [1, 2, 3], and you can add, remove and change items after creating it.
server-error:2 with retries:
  Attempt 1: InternalServerError
  Waiting 1 s
  Attempt 2: InternalServerError
  Waiting 2 s
  Attempt 3: A list is an ordered, changeable collection of values. You write one with square brackets, like [1, 2, 3], and you can add, remove and change items after creating it.
A reply under length:
Run stopped at the next function to build: check_finished is built in Lesson 5.

The 429 waits are 1.0, a float, because they came from the header through float. The 500 carries no Retry-After, so its waits double from BASE_WAIT: 1, then 2. This part of the demonstration takes about five seconds, because the waits are real.

A 503 arrives with the header Retry-After: Wed, 21 Oct 2026 07:28:00 GMT. What does wait_seconds(error, 2) return?

Why does wait_seconds return a number instead of calling time.sleep itself?

Task

In reliable.py, write wait_seconds(error, attempt), where attempt counts from 1. In this order:

  1. If error is an openai.APIStatusError, read value = error.response.headers.get("retry-after").

  2. If value is not None, convert it with float inside try, and ignore it on ValueError.

  3. If that number is zero or more, return it.

  4. Otherwise start with wait = BASE_WAIT, double it attempt - 1 times with a for loop over range(attempt - 1), and return it.

Use BASE_WAIT by name, and do not call time.sleep: this function only returns the number. Run python chapter9_demo.py to see the retries wait, then check your work.