Rate Limiting Is Not a Feature, It Is Your First Line of Defense

Most APIs ship without rate limiting because the happy path works fine when traffic is polite. The trouble starts when traffic stops being polite: a login endpoint becomes a credential-stuffing target, a search endpoint becomes a scraping firehose, and a single buggy client retries in a tight loop until your database falls over. Rate limiting is the control that keeps any one caller from consuming more than its fair share, and it doubles as a blunt but effective security boundary. It is not glamorous, and it rarely makes a roadmap, yet it is one of the highest-leverage protections you can add to a backend. This article covers the algorithms that actually work in production, why per-instance counters lie to you, how to limit by the right dimension, and the headers and status codes that make your limits usable by well-behaved clients.

Choose The Right Algorithm

The two algorithms worth knowing are token bucket and sliding window. Token bucket models a bucket that refills at a steady rate up to a maximum capacity; each request spends a token, and when the bucket is empty requests are rejected until it refills. This naturally allows short bursts while capping the sustained rate, which matches how real clients behave. A fixed window counter, by contrast, simply counts requests per calendar minute and resets at the boundary, which is simple but allows a client to send a full window of requests at the end of one minute and another full window at the start of the next, briefly doubling the intended rate. The sliding window approach fixes this by weighting the previous window's count, giving smoother enforcement. For most APIs, token bucket or sliding window strikes the right balance between fairness, burst tolerance, and implementation simplicity.

Why In-Memory Counters Fail

The first rate limiter most teams write stores counters in a JavaScript object or a local cache inside the Node process. This works perfectly on one server and breaks the moment you scale horizontally. If you run four instances behind a load balancer, each keeps its own counter, so a client limited to one hundred requests per minute can actually make four hundred by spreading traffic across instances. Worse, the limit becomes nondeterministic because it depends on how the balancer happens to route each request. The fix is a shared store that all instances consult, and in practice that means Redis. Redis offers atomic increment operations and key expiry, which are exactly the primitives a counter needs, and a single round trip per request is cheap. Centralizing the counter makes the limit a property of your system rather than an accident of your deployment topology.

Atomic Counts In Redis

Implementing a counter in Redis naively introduces a race condition. If you read the current count, compare it, then write the incremented value as three separate commands, two concurrent requests can both read the same value and both decide they are under the limit, letting more traffic through than intended. The standard fix is to make the read-modify-write atomic. The simplest version uses INCR, which atomically increments and returns the new value, paired with EXPIRE to set the window. For correctness you want the increment and the expiry to happen together, which a small Lua script executed with EVAL guarantees, since Redis runs scripts atomically. Sliding-window implementations often use a sorted set, adding the request timestamp as a member and trimming entries older than the window, then counting what remains. Either way, the rule is that the count and its mutation must be one indivisible operation.

Limit By The Right Key

Deciding what to limit on is as important as the algorithm. Limiting by IP address is the obvious default, but it punishes users behind shared NATs and corporate proxies who share one address, and it is trivially bypassed by attackers rotating through proxy pools. For authenticated traffic, limit by user or API key, because that identity is stable and meaningful. A robust setup applies multiple limits in layers: a coarse per-IP limit to absorb anonymous floods, a per-user limit for authenticated requests, and tighter per-endpoint limits on sensitive routes like login or password reset. Sensitive routes deserve far stricter limits than read-only ones, because a handful of login attempts per minute is normal while hundreds signal an attack. Choosing the key is a policy decision, so make it explicit and document which dimension each limit applies to rather than reaching reflexively for the client IP.

Communicate Limits Clearly

A rate limiter that silently drops requests frustrates legitimate clients and makes integration painful. Well-behaved APIs tell callers where they stand using response headers. The common convention exposes the limit ceiling, the remaining quota in the current window, and the time when the window resets, so clients can pace themselves rather than guess. When a client does exceed the limit, return HTTP status 429, Too Many Requests, not a generic 400 or 500, because 429 is the standardized signal that clients and libraries recognize. Include a Retry-After header telling the caller how many seconds to wait before trying again. This turns rate limiting from an opaque wall into a cooperative protocol: clients that respect the headers naturally back off, and only abusive or buggy clients keep hammering. Clear signaling also reduces support load, since developers integrating your API can diagnose throttling themselves.

Defending Authentication Endpoints

Login, signup, password reset, and token endpoints deserve special treatment because they are where attackers concentrate. Credential stuffing replays leaked username and password pairs against your login at high volume, and without throttling an attacker can test millions of combinations. Apply strict per-IP and per-account limits to these routes, and consider an escalating penalty where repeated failures lengthen the lockout. Pair rate limiting with other signals: track failed attempts per account separately from per IP, so a distributed attack against one account still triggers protection even when each IP stays under its limit. Add a small constant delay or a CAPTCHA challenge after a few failures to raise the cost of automation. Be careful not to leak whether an account exists through different responses or timings. The goal is to make brute force economically pointless while keeping the experience smooth for the genuine user who simply mistyped a password.

Layering With A Gateway

Rate limiting belongs at more than one layer of your stack. An edge layer such as a CDN, a reverse proxy like Nginx, or an API gateway can shed obviously abusive traffic before it ever reaches your application, which protects your servers from spending CPU on requests destined to be rejected. Application-level limiting then enforces business rules that the edge cannot see, like per-user quotas tied to subscription tiers. The two complement each other: the edge handles volume and crude floods cheaply, while the application handles identity-aware and feature-aware policy. Avoid relying on the application alone, because by the time a request reaches your Node process you have already paid the cost of accepting the connection and routing it. A layered defense means the cheapest layer rejects the cheapest-to-detect abuse, and only nuanced decisions travel all the way into your business logic.

Failure Modes And Fairness

Think carefully about what happens when your rate-limiting infrastructure itself has problems. If Redis becomes unreachable and your limiter throws, you must decide whether to fail open, allowing all traffic through, or fail closed, rejecting everything. Failing closed protects your backend but turns a cache outage into a full outage, while failing open keeps the API available but removes your protection exactly when you might be under attack. For most systems, failing open with aggressive alerting is the pragmatic choice for general traffic, while authentication routes should fail closed because an unprotected login is too dangerous. Also watch for fairness: a single noisy tenant should not be able to exhaust a shared limit and starve everyone else, which argues for per-tenant rather than global limits. Finally, monitor your 429 rate as a first-class metric, because a sudden spike often reveals an attack or a misbehaving client before users complain.