Caching & stale truth
Trade latency for freshness deliberately and examine the races hidden in invalidation.
BY THE END, YOU CAN
- Explain cache-aside and its failure modes
- Distinguish expiry from a strict freshness guarantee
- Control stampedes and cache-key scope
Diagram description: App → Cache (1. read); Cache → Database (2. miss: load); App → Database (write); Database → Cache (then invalidate)
A cache stores a reusable answer
Caching can reduce latency and load when answers are requested repeatedly and may be reused.
In cache-aside, a reader checks the cache, loads a miss from the source of truth, then stores the answer. The design needs a key, value, expiry policy, and ownership of invalidation. Include every input that changes the answer in the key, including tenant or permission scope where relevant.
Accidentally sharing a user-specific result across users is a correctness and security failure, not simply a poor hit ratio.
Invalidation has an ordering problem
Suppose a reader fetches old data, a writer commits new data and invalidates the cache, then the reader fills the cache with its old result.
The cache is stale again despite the invalidation. A time-to-live limits how long that entry remains after it is written, but does not by itself establish a strict bound measured from the original data change under every race. Versioned values, conditional writes, or stronger coordination can help, each with costs.
State the freshness promise your users actually need.
A fast layer can create a slow incident
When a popular key expires, many readers can simultaneously miss and overload the database.
Request coalescing, jittered expiration, or serving an acceptable stale value during refresh can reduce this stampede. Decide how the service behaves when the cache is unavailable: falling back to the database is safe only if the database can absorb the load. Monitor hit ratio alongside latency, evictions, and source load.
For Relay, cached status may be acceptable for a dashboard, while claim ownership requires stronger authoritative checks.
COMMON FAILURE MODE
Using cached job ownership or status to authorize a critical state transition can turn acceptable display staleness into a correctness bug.
TRY IT YOURSELF
Write the three-step stale-fill race in order. Propose a mitigation and identify the extra state or coordination it requires.
Make the call in the laboratory.
The two-question checkpoint, your notes, and your saved progress are in the laboratory. Free, with no account.
Open in the laboratory ↗CONNECT IT TO RELAY
Choose one Relay read that may tolerate stale data, define its freshness expectation, and keep claim transitions outside that cache.