Delivery, leases & duplicate effects
Reason about the gap between doing work and recording that it was done.
BY THE END, YOU CAN
- Compare at-most-once and at-least-once delivery
- Explain lease expiry and stale workers
- Locate the boundary of an exactly-once claim
Diagram description: 1. Queue → Worker: job 9 · lease 30 s; 2. Worker → Payments: charge (key=job-9); 3. Worker → Queue: ack (lost); 4. Queue → Worker: lease expired: job 9 again; 5. Worker → Payments: charge (key=job-9); 6. Payments → Worker: same result, no second charge
The acknowledgement gap
A worker performs an effect and then acknowledges completion.
If it crashes after the effect but before acknowledgement, another worker may repeat the effect. Acknowledging first creates the opposite risk: a crash can lose the work. At-most-once delivery favors avoiding repeats but can lose attempts; at-least-once delivery retries unacknowledged work but permits duplicates.
These are delivery properties, not guarantees about the number of business effects. An email provider, payment system, or database may need its own deduplication contract.
A lease is temporary ownership
A claim lease lets another worker recover a job after a deadline.
But an expired worker may merely be slow and can resume after a new worker has claimed the job. Attach a claim token or monotonically increasing generation and require it for acknowledgement and protected writes. A fencing token only works when the resource receiving the operation checks it; generating a token alone does not stop stale effects.
Size and renew leases using observed durations while planning for process pauses and clock behavior.
Make the effect boundary explicit
An idempotent handler produces an acceptable result when the same intended operation is attempted again.
A database uniqueness key can deduplicate a recorded effect when the effect and deduplication record commit atomically. An external provider needs a compatible idempotency mechanism, or you must accept and mitigate residual duplicates. An outbox records business state and delivery intent in one local transaction, then publishes separately.
It avoids one loss window but the publisher can still send twice. Every 'exactly once' claim should name the system boundary and failure assumptions.
COMMON FAILURE MODE
Marking a deduplication key before an external effect risks lost effects; marking it afterward risks duplicates. The atomic boundary matters.
TRY IT YOURSELF
Draw a timeline with effect, acknowledgement, and a crash between them. Then design a unique operation ID and explain where it must be enforced to prevent duplicate business effects.
Make the call in the laboratory.
The two-question checkpoint, your notes, and your saved progress are in the laboratory. Free, with no account.
Open in the laboratory ↗CONNECT IT TO RELAY
Add claim generations to Relay's design and define an idempotent sample handler. Document one external effect your design cannot make exactly once by itself.