Architecture & backpressure
Find bottlenecks, protect dependencies, and keep a burst from becoming a collapse.
BY THE END, YOU CAN
- Identify components by responsibility and failure boundary
- Relate arrival rate, service rate, and queue growth
- Choose an overload policy with explicit consequences
Diagram description: Producers → Queue (30 jobs/s); Queue → Workers; Queue → 429 · retry later (full); 429 · retry later → Producers (slow down)
Draw responsibilities before deployment boxes
Relay has an intake boundary, durable job storage, scheduling logic, workers, and status reporting.
These responsibilities can begin in one deployable application. Splitting them into services introduces network failures, independent deployment, distributed debugging, and operational overhead. Separate a component when independent scaling, ownership, isolation, or a concrete constraint justifies it.
A modular monolith can preserve useful boundaries without requiring every call to cross a network. Your architecture diagram should show data ownership and failure paths, not only boxes with fashionable technology names.
A queue absorbs bursts, not infinite overload
If jobs arrive faster than workers complete them for a sustained period, backlog grows without bound unless admission changes.
Four workers completing two jobs per second each provide roughly eight jobs per second before other bottlenecks. An arrival rate of ten then adds about two waiting jobs per second. In a stable system, Little's Law relates average work in the system to throughput and average time in the system: L = λW.
State the measurement boundary and assumptions before using the relationship for capacity planning.
Push pressure toward the source
Backpressure makes overload visible upstream instead of hiding it in ever-growing buffers.
Options include rejecting new work with a retry hint, limiting tenants, slowing producers, or shedding low-priority jobs. Each policy changes user experience and must match the product's promises. Timeouts need budgets across the full call chain; a downstream request should not continue consuming resources long after its caller has given up.
Retries should be bounded, delayed, and jittered to avoid synchronized traffic spikes. Scaling workers can worsen overload if the real bottleneck is the database.
COMMON FAILURE MODE
Adding a queue can delay visible failure while increasing latency and recovery time. Capacity and admission still need explicit limits.
TRY IT YOURSELF
Jobs arrive at 120 per minute and workers finish 90 per minute. Calculate backlog growth over ten minutes, then propose two overload policies with different user consequences.
Make the call in the laboratory.
The two-question checkpoint, your notes, and your saved progress are in the laboratory. Free, with no account.
Open in the laboratory ↗CONNECT IT TO RELAY
Draw Relay's data and failure boundaries, choose a maximum backlog, and define admission behavior when that limit is reached.