Concurrency without chaos
Overlap waiting work while keeping ownership, limits, and cancellation explicit.
BY THE END, YOU CAN
- Distinguish concurrency from parallel execution
- Identify race conditions and bounded concurrency
- Treat cancellation and cleanup as part of the contract
Diagram description: fetch A: 0–4; fetch B: 0.5–5; fetch C: 1–4 waits, 4–7; fetch D: 2–5 waits, 5–8 cancelled; 8: deadline
Overlapping is not the same as simultaneous
Concurrency means multiple tasks can make progress over overlapping periods.
Parallelism means work executes simultaneously. Async I/O can efficiently overlap waits on sockets without making CPU-heavy Python code faster. Threads are useful for some blocking I/O; processes can provide CPU parallelism with isolation and communication costs.
Python's exact threading behavior depends on the interpreter and build, including whether a global interpreter lock is enabled. Choose from the workload and library behavior rather than assuming that adding async or threads always improves throughput.
Shared state creates ordering questions
A race occurs when correctness depends on an uncontrolled ordering of operations.
In async code, a task can yield at an await, letting another task observe or change shared state. A read, wait, then write sequence is not an atomic operation. A lock can protect a critical section within its scope, but a process-local lock does not coordinate independent service replicas.
Often ownership is simpler: one component owns a mutable queue while other components communicate through messages. Keep protected regions small and avoid unnecessary blocking inside them.
Bound the work you start
Starting one task per incoming job can exhaust memory, connections, or a dependency's capacity.
Use a bounded queue and a fixed number of workers, or a semaphore with a bounded submission policy. Cancellation is a request to stop, not evidence that an external effect did not occur. Cleanup should release resources and preserve a recoverable job state.
Structured task lifetimes help ensure spawned work finishes or is cancelled before its parent scope ends. Define how Relay stops accepting work, drains active tasks, and handles unfinished claims.
COMMON FAILURE MODE
A semaphore can limit active calls while an unbounded number of waiting tasks still consumes memory. Bound admission as well.
TRY IT YOURSELF
Sketch a worker pool with four active handlers and a queue capacity of twenty. Define what the twenty-fifth waiting submission sees and how shutdown treats active work.
Make the call in the laboratory.
The two-question checkpoint, your notes, and your saved progress are in the laboratory. Free, with no account.
Open in the laboratory ↗CONNECT IT TO RELAY
Give Relay a bounded worker pool and a documented shutdown sequence. Test cancellation while a job is waiting and while it is running.