Cache Stampede
A cache stampede happens when a popular cache entry expires and many clients request it at once, each triggering an upstream fetch because no valid copy remains. The sudden synchronized miss overwhelms origins and third-party APIs, spiking latency and error rates across the fleet.
Technical Architecture & Deep-Dive
Stampedes are especially painful on globally distributed CDNs where hundreds of edges expire the same key within seconds. Without coalescing, each miss becomes its own billable API call.
Mitigations include stale-while-revalidate, soft TTL with background refresh, request coalescing, single-flight locks in workers, and jittered TTLs so keys do not all expire at the same instant. Proactive cache warming before known events also spreads load.
Key Architectural Benefits
Implement Cache Stampede at the Edge
Deploy lightweight API aggregation, edge caching, and secret injection in minutes without Docker containers or complex servers.
Frequently Asked Questions
Q:Does a longer TTL always prevent stampedes?
Longer TTLs delay the event but make it larger when it happens. Combine TTL with background refresh and coalescing instead of only extending expiry.
Related Architecture Concepts
A thundering herd is a load spike when many clients perform the same action simultaneously, such as retrying after an outage, refreshing at TTL expiry, or waking on a push notification. The synchronized demand can exceed origin capacity even when per-client traffic looks normal.
Request coalescing, also called request collapsing, is a caching technique where many simultaneous requests for the same uncached resource are combined into a single upstream request. The first request fetches from the origin while the others wait and receive the same response.
A soft TTL is an application-defined freshness window that may differ from the Cache-Control header seen by browsers. Edge workers can treat data as logically expired after a soft TTL while still serving the stale copy and refreshing asynchronously, which smooths traffic and reduces cache stampedes without exposing long staleness to clients.
Stale-While-Revalidate (SWR) is an HTTP cache control extension that instructs the edge to serve an existing cached (stale) response instantly to the client, while asynchronously initiating a background fetch to refresh the cache. Users experience zero latency wait time while always receiving fresh data.
Ready-to-Use Edge Recipes
To solve how to avoid third party api rate limits with edge caching, never execute unauthenticated requests or embed private tokens directly in client bundles. RealComposer acts as a sub-10ms edge gateway running on Cloudflare Workers that securely injects credentials from your Secret Vault, normalizes responses, and prevents unauthorized API usage. Route calls through your RealComposer edge URL to guarantee complete privacy, robust origin validation, and low mobile latency.
To cache mailchimp api to stay under rate limits, put a TTL cache on your RealComposer pipeline. Shared edge cache for popular list metadata endpoints. Shared edge cache absorbs repeat traffic before it reaches Mailchimp.