Timeout Budget
A timeout budget is the total time an API endpoint is allowed to spend producing a response, divided among the upstream calls it depends on. Each step receives a maximum duration, and calls exceeding it are aborted so the endpoint can return partial data or a fallback.
Technical Architecture & Deep-Dive
Without explicit timeouts, a single hanging upstream API can hold a request open for 30 seconds or more, frustrating users and consuming resources. A budget makes latency predictable: for example, a 1,500 millisecond endpoint might give its primary call 1,000 milliseconds and optional enrichments 400 milliseconds each in parallel.
Edge pipelines enforce per-step timeouts with AbortController and decide whether a failed step should fail the response or be omitted gracefully.
Key Architectural Benefits
Per-Step Upstream Timeout
const res = await fetch(upstreamUrl, {
signal: AbortSignal.timeout(1000),
}).catch(() => null);
const enrichment = res?.ok ? await res.json() : null;Implement Timeout Budget at the Edge
Deploy lightweight API aggregation, edge caching, and secret injection in minutes without Docker containers or complex servers.
Frequently Asked Questions
Q:How long should an upstream timeout be?
Base it on the upstream p99 latency plus a margin, and keep the total under what users tolerate, typically one to three seconds for interactive screens.
Related Architecture Concepts
P99 latency is the response time below which 99 percent of requests complete, so only the slowest 1 percent take longer. Percentile metrics like p50, p95, and p99 reveal tail latency, the slow outliers that averages hide but users still experience.
Graceful degradation is designing systems to keep providing core functionality when some components fail. For aggregated APIs, it means returning the data that succeeded, with optional sections omitted or replaced by cached values, rather than failing the entire response.
The Circuit Breaker pattern is a software resiliency design that monitors upstream service health and temporarily halts requests to failing endpoints. By failing fast and returning cached fallback data, it prevents cascading system outages and reduces recovery time.