Burst Tolerance Tuning with GCRA

The emission interval is fixed by the rate you advertise; the tolerance is the only free parameter, and it decides whether your limit feels invisible or infuriating to a legitimate client. Tuning it is a measurement exercise, not a matter of taste, and it extends the parameter reference in the GCRA and virtual scheduling guide with the method for choosing an actual number.

The problem in concrete numbers

A dashboard fires 9 requests when it loads: user, account, two widget queries, four charts, and a feature-flag fetch. Your published limit is 60 requests per minute, so T = 1,000 ms. With τ = 0, the dashboard’s second request — arriving 4 ms after the first — is rejected. The user sees a half-rendered page and reloads, producing another 9 requests. Your limiter has converted a compliant client into a retry loop.

With τ = 9,000 ms (a burst of 9), the whole page loads, and the client then needs 9 seconds of quiet before it has the same slack again — which it has, because a human reads the dashboard for longer than that. The limit is enforced exactly as advertised over any minute-long window, and no user ever notices it.

Decision matrix: tolerance by client type

Note the last two rows: tolerance is also a security parameter. A generous τ on a login endpoint hands an attacker a free burst of credential attempts every idle period, so keep it near zero there even when your read endpoints are generous.

Step-by-step: deriving tau from data

• Instrument arrival timestamps per identity for a week, including weekends and any monthly batch.
• Group arrivals into interaction clusters: consecutive requests separated by less than 500 ms.
• Take the 99th percentile cluster size — that is the burst your legitimate clients actually produce.
• Set burst = ceil(p99_cluster_size × 1.2) for user-facing endpoints, or the batch size for machine clients.
• Compute τ = burst × T and confirm τ is smaller than the window you advertise, or the limit stops meaning anything within one window.
• Simulate the week’s real trace against the candidate parameters before enforcing.
• Roll out in shadow mode and compare the would-be rejections against your support queue.
• Publish both numbers — the rate and the burst — because clients cannot self-pace against a hidden parameter.

# tune_tau.py — derive burst from a real arrival trace.
import math
from collections import defaultdict

GAP_MS = 500          # arrivals closer than this belong to one interaction

def clusters(timestamps):
    """Sizes of consecutive-arrival clusters for one identity."""
    sizes, current, last = [], 0, None
    for t in sorted(timestamps):
        if last is not None and t - last > GAP_MS:
            sizes.append(current); current = 0
        current += 1; last = t
    if current: sizes.append(current)
    return sizes

def percentile(values, p):
    if not values: return 0
    values = sorted(values)
    return values[min(len(values) - 1, int(round(p / 100 * (len(values) - 1))))]

def recommend(trace_by_identity, limit, period_ms):
    all_sizes = []
    for identity, stamps in trace_by_identity.items():
        all_sizes.extend(clusters(stamps))
    p99 = percentile(all_sizes, 99)
    burst = math.ceil(p99 * 1.2)
    T = period_ms / limit
    tau = burst * T
    print(f"p99 interaction size : {p99}")
    print(f"recommended burst    : {burst}")
    print(f"T                    : {T:.0f} ms")
    print(f"tau                  : {tau:.0f} ms  ({tau / period_ms:.0%} of the window)")
    if tau > period_ms:
        print("WARNING: tolerance exceeds the advertised window — the rate is unenforceable "
              "inside one window; reduce burst or lengthen the period")
    return burst, tau

# Example: 60/min published, arrivals collected from access logs.
recommend(load_trace("access.log"), limit=60, period_ms=60_000)

Gotchas and edge cases

• Tolerance larger than the window. With τ ≥ period, a client can spend an entire window’s budget instantly and the advertised rate no longer bounds anything inside one window. Keep τ below about half the period.
• Sizing from the mean. Average request rate says nothing about bunching. A client averaging 0.4 requests per second still needs a burst of 12 for its export job.
• One tolerance for all endpoints. A read-heavy dashboard and a password-reset endpoint have opposite requirements. Set τ per route class, not globally.
• Forgetting the mobile resume storm. Apps returning to the foreground refresh everything at once; a tolerance sized for steady use rejects half of it and users see errors on the screen they just opened.
• Tolerance that hides a broken client. If τ is large enough that a runaway loop never trips the limit, you have lost your early warning. Watch the near-limit rate as well as the rejection rate.
• Changing tau without versioning the key. Existing schedules were computed under the old tolerance; the transition period behaves as neither setting. Version the key prefix on material changes.

Verification and testing

# Replay a real trace against candidate parameters before enforcing anything.
def simulate(trace, limit, period_ms, burst):
    T = period_ms / limit
    tau = burst * T
    tat, admitted, denied, worst_wait = 0.0, 0, 0, 0.0
    for now in trace:                       # ms timestamps, ascending
        if now < tat - tau:
            denied += 1
            worst_wait = max(worst_wait, (tat - tau) - now)
        else:
            tat = max(now, tat) + T
            admitted += 1
    return {
        "admitted": admitted, "denied": denied,
        "reject_rate": round(denied / (admitted + denied), 4),
        "worst_wait_s": round(worst_wait / 1000, 1),
    }

for burst in (0, 4, 9, 20):
    print(burst, simulate(real_trace, limit=60, period_ms=60_000, burst=burst))
# burst 0  -> reject_rate 0.41  worst_wait_s 8.0   (rejects normal page loads)
# burst 4  -> reject_rate 0.12  worst_wait_s 5.0   (partial pages)
# burst 9  -> reject_rate 0.004 worst_wait_s 2.0   (only the export job waits)
# burst 20 -> reject_rate 0.000 worst_wait_s 0.0   (no longer constrains anything)

Pick the smallest burst whose rejection rate against real traffic is close to zero. Anything larger stops constraining abuse; anything smaller starts rejecting the clients you built the API for.

Frequently Asked Questions

Related

• GCRA & Virtual Scheduling — the parent guide defining the emission interval and tolerance.
• Implementing GCRA in Redis Lua — where these numbers are actually applied.
• Choosing Window Size for API Rate Limits — the equivalent sizing question for window algorithms.
• SDK & Client-Side Throttling — how a client uses the published burst to avoid rejection entirely.