When Things Go Wrong · practice
Retry Only What Is Worth Retrying
Lesson 1 sorted failures into three groups. Only the first, “retry after a wait”, is worth another attempt. This lesson turns that group into a function that a program can ask: given this error, should I try again?
Asking by type
The openai client raises a different class for each kind of failure, and the classes form a family tree. isinstance checks a class and every class below it, so one call can ask about several at once when you give it a tuple of classes:
print(isinstance(3, (int, float)))
print(isinstance("3", (int, float)))
The four classes worth retrying are:
| Class | What it means |
|---|---|
openai.RateLimitError | 429, too many requests for now |
openai.APITimeoutError | no reply within the client’s timeout |
openai.APIConnectionError | the connection failed before any reply |
openai.InternalServerError | any status from 500 up: 500, 502, 503 and 504 all arrive as this class |
APITimeoutError is already a kind of APIConnectionError, so the tuple would work without it. It is listed anyway, so a reader sees that timeouts were thought about.
Everything else gets False: BadRequestError (400), NotFoundError (404) and PermissionDeniedError (403), and errors that do not come from the client at all, such as a ValueError, a pydantic ValidationError or a StreamBroken.
Not too wide
openai.APIError is the class above every error the client raises over the network, and openai.APIStatusError is the class above every error that came with a status. Both are tempting, because one name covers a lot. Both are too wide: a 400 is an APIStatusError too, and retrying it sends the same broken request again. List the four classes by name.
Two edges
A stream that breaks while you read it also raises APIConnectionError. should_retry says True for it, and that is right for a whole call that failed before any text was shown. A stream that already printed text is handled differently: the supplied stream_once turns that break into StreamBroken, which should_retry rejects, and Lesson 9 explains the rule.
The client’s own retry rule, the one Chapter 5 switched off with max_retries=0, also retries 408 (request timeout) and 409 (conflict). The course leaves those two out; neither comes from llm-api.
Run it
show_failures() returns the errors it met. When should_retry is written, run python chapter9_demo.py. The demonstration asks about each of them:
Worth another attempt?
RateLimitError: True
InternalServerError: True
InternalServerError: True
APITimeoutError: True
NotFoundError: False
BadRequestError: False
rate-limit:2 with retries:
Attempt 1: RateLimitError
Run stopped at the next function to build: wait_seconds is built in Lesson 4.
The two InternalServerError lines are the 500 and the 504. The cut-off reply is not in the list, because it raised nothing.
The next part of the demonstration is retry_demo in chapter9_demo.py, which tries a request up to three times. It stopped at its first failure, because it needs to know how long to wait. That is Lesson 4.
Why not write should_retry as return isinstance(error, openai.APIStatusError)?
A stream breaks after two sentences are already on the screen. What does should_retry receive, and what does it answer?
Task
In reliable.py, write should_retry(error). Return True when error is an openai.RateLimitError, openai.APITimeoutError, openai.APIConnectionError or openai.InternalServerError, using one isinstance call with a tuple of those four classes. Return False for every other error, including a 400, 404 or 403, a plain ValueError, a pydantic ValidationError and a StreamBroken. Run python chapter9_demo.py to see the verdict for each error from Lesson 1, then check your work.