What API rate limiting does
Rate limiting caps how many requests a client can make in a period, such as 100 requests per minute per API key. Requests over the limit are rejected, usually with 429 Too Many Requests, so one client can't use up capacity that others share.
Providers limit for several reasons. A bug or runaway loop in one client shouldn't overload the backend, and capacity should be shared fairly between customers. Login endpoints need to slow password guessing, and expensive operations cost money.
Limits are usually counted per API key, user or access token. Counting by IP address is weaker, because many users can share one address behind NAT and an attacker can spread requests across many. Rate limiting alone doesn't stop a large distributed attack, which needs protection upstream at a CDN or network provider.
How the common algorithms differ
Each algorithm trades precision against memory and how much bursting it allows. Token bucket is a sensible default for most APIs, because it permits the short bursts real clients produce while holding the average rate.
- Fixed window counts requests in set intervals, such as each clock minute. It is cheap but allows bursts at window edges. A client can use a full quota at the end of one window and again at the start of the next. That is up to twice the limit in a few seconds.
- Sliding window log stores a timestamp for every request and counts those in the last N seconds. It is precise but uses memory in proportion to the limit.
- Sliding window counter keeps counts for the current and previous windows and weights the previous one by how much it overlaps the sliding window. It approximates the log with two numbers per client.
- Token bucket holds up to a fixed number of tokens and refills at a steady rate. Each request takes one token, so a client can burst up to the bucket size while the long-run rate stays capped.
- Leaky bucket queues requests and releases them at a constant rate, dropping new ones when the queue is full. It smooths traffic to the backend but adds waiting time.
Enforcing limits across several servers
When an API runs on several instances, they all need to see the same count. A common approach is a shared in-memory store such as Redis, where each request increments a counter key that expires with the window.
Make the check and the increment one atomic operation. If you read the count, compare it and then write it back, concurrent requests can slip past the limit. Atomic increments, or a short script that runs inside the store, close that gap.
Decide in advance what happens when the shared store is unreachable: fail open and allow traffic, or fail closed and reject it.
The 429 response and Retry-After
RFC 6585 defines 429 Too Many Requests for a client that has sent too many requests in a given amount of time. The response can include a Retry-After header (RFC 9110) that says how long to wait, as a number of seconds or an HTTP date.
Many providers also send quota headers such as X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset. These names aren't standardized and their meanings vary, so read each provider's documentation. An IETF draft defines standard RateLimit and RateLimit-Policy fields, but it is still a draft.
Explain the limit in the response body as well, in the same error format as the rest of the API.
HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/problem+json
{
"type": "https://api.example.com/problems/rate-limited",
"title": "Too many requests",
"status": 429,
"detail": "This API key is limited to 100 requests per minute."
}How clients should handle 429 responses
Treat a 429 as an instruction to slow down. If the response has Retry-After, wait at least that long. If it doesn't, back off exponentially: double the delay after each failed attempt, up to a cap, and stop after a fixed number of retries.
Add jitter, a random element in the delay, so clients rejected at the same moment don't all retry at the same moment. A common form called full jitter waits a random time between zero and the current backoff.
A 429 means the server turned the request away, so resending it after the wait is normally safe. Timeouts and 5xx errors are different, because the server may have acted. Retry those automatically only for idempotent methods, or for a POST sent with an idempotency key.
Better still, pace requests on the client so you stay under the limit in the first place.
async function fetchWithBackoff(url, options = {}, maxRetries = 5) {
for (let attempt = 0; ; attempt++) {
const res = await fetch(url, options);
if (res.status !== 429 || attempt === maxRetries) return res;
// Retry-After in seconds; an HTTP-date value falls back to backoff here.
const retryAfter = Number(res.headers.get('Retry-After'));
const backoff = Math.min(30000, 500 * 2 ** attempt);
const delay = retryAfter > 0 ? retryAfter * 1000 : Math.random() * backoff;
await new Promise((resolve) => setTimeout(resolve, delay));
}
}