Chapter 13
Choosing the Right Tool
A Loop, Threads, Processes, or Neither
Pulse was the right shape for an event loop. Almost all of its time is spent waiting: for practice-api to answer, for a pong server to reply, for the pause between rounds. Each wait costs the loop nothing, because a waiting coroutine is only a paused call in a list, so one thread can hold hundreds of them. Not every program looks like that. This chapter is about choosing, and this lesson compares the four answers Python gives to “my program has a lot to do”.
Four answers
| Model | Good for | What it costs |
|---|---|---|
| one event loop | many waits at once: hundreds or thousands of idle connections, requests or timers | everything on the loop must be async, or handed to a thread with to_thread; one blocking call stops every coroutine (Chapter 5) |
| threads | a few dozen blocking calls at once, in libraries that have no async version; a thread per wait is fine at that scale, not at thousands | each thread has its own stack, so thousands get heavy; any state two threads share needs a lock around every change, because a thread can be paused between any two steps, not only at an await |
| processes | heavy computation in Python, spread over several processor cores | each process is a separate Python with its own memory, so data has to be copied between them; starting one is slow next to starting a task |
| plain code | one thing at a time, such as one request in a script | nothing runs at the same time; it is the simplest to read, write and test |
The loop’s cost column is what this course has spent most of its time on: async libraries such as httpx, to_thread for the old blocking legacy_check, and the rule that nothing blocks. The threads row has a cost you met in miniature in Chapter 11. A lock was needed there only because an await sat between reading and writing the totals. With threads, every line is such a place, which is why shared state under threads needs care everywhere and why this course kept threads behind to_thread.
The global interpreter lock
In CPython, the Python you have used throughout, threads take turns running Python code: a lock inside the interpreter, the global interpreter lock or GIL, lets only one thread run Python bytecode at a time. Threads still help a program that waits, because a thread that is blocked on the network or the disk lets go of that lock while it waits. They do not speed up pure Python computation, because the computing threads simply take turns. Python 3.13 and 3.14 can also be built without the lock, as a separate free-threaded build; this course names it and does not use it.
Five scenarios
For each one, choose the model and check the reason against the table.
A chat server keeps 2,000 connections open. Most of the time, each one is waiting for its user to type. Which model fits?
A script makes eight calls to a library that has no async version, and each call blocks for a while on the network. Which model fits?
A program counts the prime numbers below ten million with a loop written in pure Python. Which model could make it faster on a computer with several cores?
A nightly script sends one GET request and writes the answer to a file. Which model fits?
A checker like Pulse watches fifty HTTP services with an async client, and must also run one old check that only exists as blocking code. Which model fits?
The pattern behind the answers: count the waits, and ask whether the work is waiting or computing. Many waits point to a loop; a few blocking calls point to threads; computation points to processes; one thing at a time points to plain code. And a program can mix them, as Pulse does.
A mix still keeps one model in charge. Pulse is a loop that borrows a thread for legacy_check. A loop program that also had heavy computation to do would hand that part to a process, for the same reason it hands blocking calls to a thread: the loop’s own thread must stay free to serve the waits. The move can go the other way too. A plain script that one day has fifty addresses to check can gain one async def function and one asyncio.run, and keep the rest of its code plain.
Where threads and processes are taught
This course uses threads in exactly one way, through to_thread, and processes not at all. Writing threaded and multi-process programs well, with threading, concurrent.futures and multiprocessing, is the subject of a planned course on concurrency with threads and processes. The next lesson looks at the line between async and plain code that every one of these programs has to cross somewhere.