Choosing Window Size for API Rate Limits

“1,000 requests per hour” and “17 requests per minute” describe almost the same throughput and behave nothing alike: one lets a client spend everything in the first second of every hour, the other paces them continuously. Window size is therefore a product decision as much as a technical one, and it interacts directly with the boundary behaviour described in fixed window vs sliding window.

The problem in concrete numbers

Take the same budget expressed three ways, and look at the worst-case burst a compliant client can produce:

Every row grants the same sustained throughput. The first row is a denial-of-service vector that you documented and invited. This is the fixed-window boundary problem, and the size of the window is what turns it from a curiosity into an outage.

Decision matrix

The pattern that solves most APIs is a pair: a short window for shape and a long window for budget. “10 requests per second and 5,000 per hour” bounds the instantaneous load at 20 and the sustained draw at 5,000, and both numbers are simple enough to publish.

Step-by-step: choosing the pair

• Start from the capacity question: how many concurrent requests from one client can your backend absorb without degrading? That bounds the short window.
• Set the short window at 1–10 seconds, sized so 2 × limit stays under that capacity figure.
• Start from the commercial question: how much usage does a plan include? That defines the long window.
• Set the long window to the billing period for quotas, or an hour for fair-use ceilings.
• Check the interaction: the long limit divided by its window should be well below the short limit, or the short one never binds.
• Prefer a weighted sliding window for the short limit if boundary doubling would hurt.
• Publish both numbers and the window each applies to; a client cannot pace against an undisclosed window.
• Emit headers for the tighter of the two at any moment, and say which one is being reported.

# Two windows, one decision. Short window shapes; long window budgets.
import math, time

WINDOWS = [
    ("burst", 10, 100),        # name, seconds, limit  -> 10 rps sustained, 20 worst-case
    ("hourly", 3600, 5_000),   # fair-use ceiling
]

LUA = """
-- KEYS: one key per window. ARGV pairs: window_seconds, limit
local now = redis.call('TIME')[1]
local tightest_reset, blocked = 0, 0
for i, key in ipairs(KEYS) do
  local window = tonumber(ARGV[i * 2 - 1])
  local limit  = tonumber(ARGV[i * 2])
  local bucket = math.floor(now / window)
  local k = key .. ':' .. bucket
  local count = redis.call('INCR', k)
  if count == 1 then redis.call('EXPIRE', k, window * 2) end
  if count > limit then
    blocked = 1
    local reset = (bucket + 1) * window - now
    if tightest_reset == 0 or reset < tightest_reset then tightest_reset = reset end
  end
end
return { blocked, tightest_reset }
"""

def check(redis, identity):
    keys = [f"rl:{identity}:{name}" for name, _, _ in WINDOWS]
    args = [v for _, w, l in WINDOWS for v in (w, l)]
    blocked, reset = redis.eval(LUA, len(keys), *keys, *args)
    return {"allowed": blocked == 0, "retry_after": max(1, int(reset)) if blocked else 0}

Note the EXPIRE of window * 2: the key must outlive its own window so a request arriving late in the window still sees the count, and it must not outlive the next window or a stale counter blocks a fresh one.

Gotchas and edge cases

• Publishing only the long window. Clients then assume they may spend it instantly, and the ones that do are behaving exactly as documented.
• Windows that do not divide evenly. A 45-second window means boundaries drift relative to the minute, which makes reasoning about dashboards and support conversations harder for no benefit. Prefer 1, 5, 10, 60, or 3,600 seconds.
• Both windows binding at once. If the long window’s implied rate is above the short limit, the long limit never triggers and you have one limit with extra machinery. Check the arithmetic.
• Aligned boundaries across all clients. Every key resetting at :00 produces a synchronised stampede. Offset the bucket by a per-key hash: floor((now + hash(key) % window) / window).
• Reporting the wrong window in headers. A client told X-RateLimit-Reset: 3400 while the short window rejects it every second will retry uselessly. Report the tighter window and name it.
• TTL equal to the window. A counter that expires exactly at the boundary can vanish while requests are still arriving in that window. Expire at twice the window.
• Changing window size without versioning keys. Existing counters were bucketed by the old window; a change reinterprets them. Include the window in the key name.

Verification and testing

# 1. Confirm the short window binds first and reports itself.
for i in $(seq 1 120); do
  curl -s -o /dev/null -D- -H "X-API-Key: $KEY" https://api.example.com/v1/ping |
    grep -i "^x-ratelimit-window\|^x-ratelimit-reset\|^HTTP/"
done | tail -6
# expect: HTTP 429 with x-ratelimit-window: burst and a reset of a few seconds

# 2. Confirm the long window still binds after a day of paced traffic.
#    Paced at the short limit, the hourly ceiling must be reached in ~50 minutes.
python3 - <<'PY'
import time, requests
sent = ok = 0
start = time.time()
while time.time() - start < 3600:
    r = requests.get("https://api.example.com/v1/ping", headers={"X-API-Key": KEY})
    sent += 1; ok += r.status_code == 200
    time.sleep(0.7)                      # under the short limit, over the hourly pace
print(f"sent={sent} accepted={ok}")      # accepted should plateau at ~5000
PY

# 3. Confirm boundary offsets are per key, not global.
#    Two keys should reset at different instants.
for k in key_a key_b; do
  curl -s -D- -o /dev/null -H "X-API-Key: $k" https://api.example.com/v1/ping |
    grep -i "^x-ratelimit-reset"
done
# the two reset values should differ; identical values mean every client resets together

Test three catches the stampede problem that only appears under load: if every client’s window resets at the same instant, your traffic graph develops a spike at the top of every minute that no amount of capacity planning explains.

Frequently Asked Questions

Related

• Fixed Window vs Sliding Window — the parent guide on window topologies.
• Fixed Window Counter Drift Explained — the boundary behaviour window size amplifies.
• Burst Tolerance Tuning with GCRA — the equivalent sizing question without windows.
• Tiered Access & Quota Enforcement — where the long window becomes a plan boundary.