An API without rate limits is one noisy client away from an outage. But limits set carelessly frustrate honest customers and generate support tickets. This guide explains how rate limiting actually works, which algorithm fits which need, and how to communicate limits so integrators trust your platform instead of fighting it.
Why Rate Limiting Exists
Rate limiting protects shared resources from being exhausted by any single caller, whether the cause is a bug, an aggressive integration, or abuse. It also enforces fairness, so one heavy tenant does not degrade service for everyone else, and it supports your business model when API access is tiered. The nature of the problem is capacity allocation under uncertainty: you cannot know in advance who will call how often, so you set rules that keep the system stable.
The Main Algorithms
Fixed window
Count requests per fixed interval, such as 1,000 per minute, and reset at the boundary. It is simple but has a burst flaw: a client can send the full quota at the end of one window and again at the start of the next, doubling the intended rate for a short spike.
Sliding window
Smooths the fixed-window edge by weighting the previous window as the current one advances. It reduces boundary bursts at the cost of slightly more computation and state.
Token bucket
A bucket refills at a steady rate up to a maximum. Each request spends a token. This allows short, controlled bursts while capping the sustained rate, which matches how real clients behave. It is the most common choice for public APIs.
Leaky bucket
Processes requests at a constant output rate, queuing or dropping overflow. It produces very smooth traffic and suits systems that need steady downstream load.
| Algorithm | Allows bursts | Best for |
| Fixed window | Yes, at edges | Simple internal limits |
| Sliding window | Limited | Fairer general limiting |
| Token bucket | Controlled | Public APIs with bursty clients |
| Leaky bucket | No | Smoothing downstream load |
Communicating Limits to Clients
A limit the client cannot see is a trap. Return standard headers so integrations can self-regulate: the limit ceiling, the remaining count, and the reset time. When a client exceeds the limit, respond with HTTP status 429 Too Many Requests and include a Retry-After header telling them how long to wait. Well-behaved clients will back off automatically if you give them the information to do so.
A Real Scenario
Imagine a billing SaaS whose customer runs a nightly sync that fires thousands of requests in a burst. A pure fixed-window limit either blocks the legitimate sync or is set so high it offers no protection. Switching to a token-bucket limit lets the sync burst within a capped ceiling while still refilling steadily, so normal interactive traffic is unaffected. Pairing that with 429 responses and a Retry-After header lets the customer’s client pause and resume cleanly instead of hammering the API and generating errors. The principle holds broadly: match the algorithm to real traffic shape, and tell clients how to behave.
Common Mistakes and How to Fix Them
- Silent throttling with no headers. Fix: always return limit, remaining, reset, and Retry-After so clients can adapt.
- Using the wrong status code. Fix: use 429 for rate limits, not 403 or a generic 500.
- One global limit for all tiers. Fix: scope limits per API key or tenant so a free user cannot exhaust a paying customer’s capacity.
- Fixed windows on bursty workloads. Fix: prefer token bucket when clients legitimately batch requests.
- No documentation. Fix: publish exact limits and back-off guidance so integrators design for them upfront.
Action Checklist
- Identify what you are protecting: database, downstream service, or fairness across tenants.
- Study real traffic shape before choosing an algorithm.
- Scope limits per key or tenant, not just globally.
- Pick an algorithm that matches burst behavior; token bucket is a safe default for public APIs.
- Return rate-limit headers on every response.
- Respond with 429 and Retry-After when limits are exceeded.
- Document limits and back-off expectations clearly.
- Log limit hits so you can tune thresholds with evidence.
Conclusion and Next Step
Good rate limiting is invisible to honest users and firm against overload. Start by naming the resource you must protect and looking at how your busiest clients actually call the API. That evidence points directly to the right algorithm and thresholds.
FAQ
Should I rate limit by IP address or API key?
Prefer API key or account, because it maps to who is responsible and survives shared IPs and proxies. IP-based limits still help as a coarse defense against anonymous abuse.
What is a reasonable default limit?
There is no universal number; it depends on your capacity and traffic. Set an initial value from load testing, watch how often real clients hit it, and adjust. Start conservative and loosen with evidence.
How should clients handle a 429?
They should honor Retry-After and use exponential back-off with jitter to avoid retry storms where many clients retry at the same instant. Document this so integrators build it in.
Do I need distributed rate limiting?
If your API runs across multiple servers, per-instance counters undercount total usage. A shared store such as Redis lets all instances enforce one consistent limit. For a single instance, in-memory counting is enough.
References
- MDN Web Docs: HTTP 429 Too Many Requests and Retry-After header.
- IETF RFC 6585 (Additional HTTP Status Codes), which defines status 429.