0%

When Things Go Wrong · practice

Retry Only What Is Worth Retrying

Lesson 1 sorted failures into three groups. Only the first, “retry after a wait”, is worth another attempt. This lesson turns that group into a function that a program can ask: given this error, should I try again?

Asking by type

The openai client raises a different class for each kind of failure, and the classes form a family tree. isinstance checks a class and every class below it, so one call can ask about several at once when you give it a tuple of classes:

print(isinstance(3, (int, float)))
print(isinstance("3", (int, float)))

The four classes worth retrying are:

ClassWhat it means
openai.RateLimitError429, too many requests for now
openai.APITimeoutErrorno reply within the client’s timeout
openai.APIConnectionErrorthe connection failed before any reply
openai.InternalServerErrorany status from 500 up: 500, 502, 503 and 504 all arrive as this class

APITimeoutError is already a kind of APIConnectionError, so the tuple would work without it. It is listed anyway, so a reader sees that timeouts were thought about.

Everything else gets False: BadRequestError (400), NotFoundError (404) and PermissionDeniedError (403), and errors that do not come from the client at all, such as a ValueError, a pydantic ValidationError or a StreamBroken.

Not too wide

openai.APIError is the class above every error the client raises over the network, and openai.APIStatusError is the class above every error that came with a status. Both are tempting, because one name covers a lot. Both are too wide: a 400 is an APIStatusError too, and retrying it sends the same broken request again. List the four classes by name.

Two edges

A stream that breaks while you read it also raises APIConnectionError. should_retry says True for it, and that is right for a whole call that failed before any text was shown. A stream that already printed text is handled differently: the supplied stream_once turns that break into StreamBroken, which should_retry rejects, and Lesson 9 explains the rule.

The client’s own retry rule, the one Chapter 5 switched off with max_retries=0, also retries 408 (request timeout) and 409 (conflict). The course leaves those two out; neither comes from llm-api.

Run it

show_failures() returns the errors it met. When should_retry is written, run python chapter9_demo.py. The demonstration asks about each of them:

Worth another attempt?
  RateLimitError: True
  InternalServerError: True
  InternalServerError: True
  APITimeoutError: True
  NotFoundError: False
  BadRequestError: False
rate-limit:2 with retries:
  Attempt 1: RateLimitError
Run stopped at the next function to build: wait_seconds is built in Lesson 4.

The two InternalServerError lines are the 500 and the 504. The cut-off reply is not in the list, because it raised nothing.

The next part of the demonstration is retry_demo in chapter9_demo.py, which tries a request up to three times. It stopped at its first failure, because it needs to know how long to wait. That is Lesson 4.

Why not write should_retry as return isinstance(error, openai.APIStatusError)?

A stream breaks after two sentences are already on the screen. What does should_retry receive, and what does it answer?

Task

In reliable.py, write should_retry(error). Return True when error is an openai.RateLimitError, openai.APITimeoutError, openai.APIConnectionError or openai.InternalServerError, using one isinstance call with a tuple of those four classes. Return False for every other error, including a 400, 404 or 403, a plain ValueError, a pydantic ValidationError and a StreamBroken. Run python chapter9_demo.py to see the verdict for each error from Lesson 1, then check your work.