iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To prevent a retry thundering herd, combine exponential backoff with jitter: backoff spaces attempts farther apart, while jitter randomizes their timing so clients that failed together are less likely to retry together. Then limit retries, retry only plausibly temporary failures, and protect operations against duplicate effects.
Why retries can create a thundering herd
When a service fails or slows down, many clients may receive errors at roughly the same time. With plain exponential backoff, each client waits longer before its next attempt, but clients that started together can preserve that timing pattern: their retries may still arrive in clusters. If the service is already struggling, each synchronized wave can add load and delay recovery.
Jitter adds a random element to each wait, spreading attempts across time. It complements rather than replaces backoff: exponential backoff slows repeated attempts; jitter reduces alignment among clients. As Marc Brooker put it in the AWS Architecture Blog, “The solution isn’t to remove backoff. It’s to add jitter.” AWS Architecture Blog: Exponential Backoff and Jitter
How the main jitter strategies differ
Jitter strategies trade off predictable delay against how widely retry attempts spread. There is no single choice that is best for every workload; consider concurrency, acceptable latency, and the amount of retry traffic the service can tolerate.
#1 Best Overall
| Strategy | How the wait is chosen | Practical trade-off |
|---|---|---|
| Full jitter | Choose a random wait from zero up to the current capped exponential-backoff interval. | Spreads attempts broadly and can reduce synchronized work. In the AWS Architecture Blog’s 2015 simulation with 100 contending clients, it produced more than half fewer calls than unjittered exponential backoff; that is a simulation result, not a universal production guarantee. |
| Equal jitter | Retain part of the exponential-backoff interval and randomize the rest. | Leaves some delay deterministic while spreading the remainder. The AWS comparison reports similar client work for full and equal jitter. |
| Decorrelated jitter | Choose the next wait using the previous wait, with a configured bound. | Changes the timing relationship between successive waits. In the AWS comparison, it involved higher client work than full or equal jitter; the best fit still depends on workload and latency constraints. |
The strategy names describe algorithm families, not guaranteed behavior across every library. Check the implementation’s formula and limits rather than relying on a label alone. AWS Architecture Blog: Exponential Backoff and Jitter
Design a retry policy that does not amplify an outage
Classify errors before retrying
Retry failures that might resolve on another attempt, such as throttling, timeouts, or transient service errors. Fail fast on persistent problems such as permission or configuration errors; repeating them adds traffic without making success more likely. If a service supplies retry guidance, such as a Retry-After header, account for it where applicable. AWS Well-Architected Framework: Limit retries
Set limits that fit the caller’s time budget
Use an attempt limit, an elapsed-time budget, and a maximum individual delay. Longer waits may reduce pressure on a recovering service, but they also consume the time available to the caller. A retry policy should stop when another attempt can no longer produce a useful result within that budget. AWS Well-Architected recommends progressively longer intervals, jitter, and a maximum retry count. AWS Well-Architected Framework: Limit retries
Protect side effects from duplicate execution
A timeout tells the client it did not receive a timely response; it does not prove the server failed to perform the operation. Retrying a request that charges an account, creates a record, or triggers another side effect can therefore repeat that effect. Make the operation idempotent, or use an idempotency mechanism that lets the service recognize and safely handle repeat requests. AWS Builders’ Library: Timeouts, retries, and backoff with jitter
Choose one deliberate retry layer
Retries in multiple layers can compound downstream attempts. For example, an application retrying around a client library that also retries can send more requests than either policy suggests by itself. Inspect the actual SDK and library configuration, including whether retries are enabled, rather than assuming a built-in policy is active. AWS Builders’ Library: Timeouts, retries, and backoff with jitter
Observe and exercise the policy
Monitor retry behavior and repeated failures, and test scenarios such as throttling, timeouts, and service disruption. Retries can help with rare transient errors, but under overload they can make resource pressure worse. Check that limits, error classification, and idempotency behave as intended when many callers encounter a failure together. AWS Well-Architected Framework: Limit retries
Rank #4
A concrete AWS SDK reference example
The AWS SDKs and Tools Reference Guide documents standard retry mode as exponential backoff with full jitter. Its documented delay formula is random(0, 1) × min(20,000 ms, base_delay × 2^retry). The guide specifies a 50 ms base delay for transient non-throttling errors, a 1,000 ms base delay for throttling errors, and a 20,000 ms maximum individual delay. It describes three default total attempts: one initial request and two retries. These are AWS reference values, not universal microservice settings; verify the exact SDK and version used by your workload. AWS SDKs and Tools Reference Guide: Retry behavior
Recommended Free Tools
The same reference describes a retry quota: retries can stop when its tokens are depleted. Its stated purpose is to avoid spending time on retries unlikely to succeed and to reduce retry traffic during disruption. For any SDK or library, check its retryable-error classification, jitter algorithm, delay cap, total-attempt counting, retry budget or quota, and whether throttling uses different delays. AWS SDKs and Tools Reference Guide: Retry behavior
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
