Protocol-Specific Throttling: GraphQL, WebSocket, gRPC & Webhooks

Counting requests works because in REST one request costs roughly one unit of work — an assumption that collapses the moment a single GraphQL query can fan out into 4,000 database reads, a WebSocket connection can push 10,000 messages without another HTTP request, or your own service becomes the client hammering someone else’s webhook endpoint. This guide extends the enforcement patterns from the Backend Middleware & Distributed Tracking reference to protocols where “one request” is the wrong unit, and shows what to count instead: query cost, messages per connection, streamed frames, and outbound delivery attempts.

The unifying idea is that every limiter needs a cost function. In REST the cost function is the constant 1. In GraphQL it is a static analysis of the query document. On a WebSocket it is a per-message charge against a budget that lives as long as the connection. For outbound webhooks it is a per-destination pacing schedule. Once you accept a variable cost, the underlying algorithm barely changes: the token bucket still works, you just deduct cost tokens instead of one.

Mechanism: cost functions and where they are evaluated

A protocol-aware limiter runs in three phases. Identify resolves the caller — an API key header for GraphQL, the authenticated session captured at the WebSocket handshake, the gRPC metadata entry, the subscriber id for an outbound delivery. Price computes the cost of the operation before executing it. Consume performs one atomic deduction and either proceeds or rejects.

The pricing phase is where each protocol differs, and it is subject to a hard constraint: pricing must be cheaper than the work it protects. A GraphQL cost analysis that walks the query AST is O(nodes) — microseconds for a 200-node document, and worth it when the alternative is a 4-second resolver storm. A cost function that itself queries the database has defeated its own purpose.

Two structural details separate a working implementation from a leaky one. First, connection-scoped state must also roll up to an account, or a client opens 200 WebSocket connections and multiplies its budget exactly the way per-instance proxy zones multiply an edge limit. Second, streaming limits need a mid-stream rejection path; a limiter that can only reject at connect time cannot stop a client that behaves for the first minute and then floods.

Configuration reference: a cost-aware bucket

The parameters below generalise across the four protocols; only the units change.

max_single_cost deserves attention: without it, a query priced at 12,000 against a 10,000 bucket can never succeed, but it will still empty the bucket on every attempt if you deduct before checking. Reject over-budget operations with a distinct, explanatory error — “query cost 12,000 exceeds the maximum of 5,000; reduce nesting or page size” — rather than a generic rate-limit message that invites a retry loop.

Implementation walkthrough: one primitive, four call sites

Start with a cost-aware consume built on the same atomic script pattern used across this site, then call it from each protocol’s hook.

-- consume.lua — deduct `cost` units atomically; returns {allowed, remaining, retry_after}
-- KEYS[1] = bucket key   ARGV = capacity, refill_per_sec, now_ms, cost
local cap  = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now  = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

local st     = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(st[1]) or cap
local ts     = tonumber(st[2]) or now
tokens = math.min(cap, tokens + (now - ts) / 1000 * rate)

local allowed = 0
if tokens >= cost then
  tokens  = tokens - cost
  allowed = 1
end
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', KEYS[1], math.ceil(cap / rate * 1000) + 1000)

local retry = 0
if allowed == 0 then retry = math.max(1, math.ceil((cost - tokens) / rate)) end
return { allowed, math.floor(tokens), retry }

The GraphQL call site prices the document during validation, so nothing executes before the charge lands:

// GraphQL: price the operation, then consume that many units.
import { parse, validate, specifiedRules, TypeInfo, visit, visitWithTypeInfo } from "graphql";

const DEFAULT_LIST_SIZE = 50;
const MAX_SINGLE_COST = 5_000;

// Static cost: each field costs 1, multiplied by the page size of every enclosing list.
export function priceQuery(schema, document): number {
  const typeInfo = new TypeInfo(schema);
  let cost = 0;
  const multipliers: number[] = [1];

  visit(document, visitWithTypeInfo(typeInfo, {
    Field: {
      enter(node) {
        const first = node.arguments?.find((a) => a.name.value === "first" || a.name.value === "limit");
        const size = first && first.value.kind === "IntValue"
          ? Number(first.value.value)
          : DEFAULT_LIST_SIZE;
        const isList = String(typeInfo.getType() ?? "").includes("[");
        const parent = multipliers[multipliers.length - 1];
        const here = isList ? parent * size : parent;
        multipliers.push(here);
        cost += here;
      },
      leave() { multipliers.pop(); },
    },
  }));
  return cost;
}

export async function graphqlRateLimit(ctx, document) {
  const cost = priceQuery(ctx.schema, document);
  if (cost > MAX_SINGLE_COST) {
    throw new GraphQLError(
      `Query cost ${cost} exceeds the per-operation maximum of ${MAX_SINGLE_COST}. ` +
      `Reduce nesting or request a smaller page.`,
      { extensions: { code: "QUERY_TOO_EXPENSIVE", cost, max: MAX_SINGLE_COST } },
    );
  }
  const [allowed, remaining, retryAfter] = await ctx.consume(`gql:${ctx.apiKey}`, cost);
  ctx.response.http.headers.set("X-RateLimit-Remaining", String(remaining));
  if (!allowed) {
    ctx.response.http.headers.set("Retry-After", String(retryAfter));
    throw new GraphQLError("Rate limit exceeded", {
      extensions: { code: "RATE_LIMITED", cost, retryAfter },
    });
  }
}

The WebSocket call site charges per inbound message and keeps a connection-scoped counter alongside the account bucket:

// WebSocket: per-connection budget plus an account-wide bucket.
const PER_CONN_PER_SEC = 20;

wss.on("connection", (socket, req) => {
  const accountId = authenticate(req);          // resolved once, at handshake
  let allowance = PER_CONN_PER_SEC;
  let last = Date.now();

  socket.on("message", async (raw) => {
    // Local per-connection bucket: cheap, no network hop, stops single-socket floods.
    const now = Date.now();
    allowance = Math.min(PER_CONN_PER_SEC, allowance + ((now - last) / 1000) * PER_CONN_PER_SEC);
    last = now;
    if (allowance < 1) {
      socket.send(JSON.stringify({ type: "error", code: "rate_limited", retryAfter: 1 }));
      return;                                    // drop the message, keep the socket
    }
    allowance -= 1;

    // Account-wide bucket: catches a client that opens many sockets.
    const [allowed] = await consume(`ws:${accountId}`, 1);
    if (!allowed) {
      socket.send(JSON.stringify({ type: "error", code: "account_rate_limited" }));
      socket.close(1008, "rate limit exceeded");  // policy violation close code
      return;
    }
    handle(JSON.parse(raw.toString()));
  });
});

Note the deliberate asymmetry: a single noisy message is dropped with an error frame and the socket survives, but a client breaching its account budget across sockets is disconnected with close code 1008. Killing a connection for one fast message turns a minor client bug into a reconnect storm, which costs you more than the messages would have.

Distributed and scaling considerations

Connection-oriented protocols pin state to a node. A WebSocket lives on one server for its lifetime, so per-connection counters are naturally local and free — but per-account counters are not, and reading them from the shared store on every message multiplies your Redis traffic by the message rate. The standard mitigation is a two-level scheme: enforce the strict per-connection budget locally on every message, and sample the account-wide check — every N messages or every second — accepting a small overshoot in exchange for a fraction of the round trips.

Outbound webhooks invert the problem: you are the client, and the limit belongs to somebody else’s server. Pacing must be per destination endpoint, because one slow subscriber must not consume the delivery workers that other subscribers depend on. Give every endpoint its own queue partition and its own bucket, honour the Retry-After the subscriber sends back, and cap concurrency per endpoint so a subscriber that responds in 30 seconds cannot pin your worker pool. When a destination has been failing for hours, back off to a long interval and mark the subscription unhealthy rather than retrying at full rate forever.

Failure modes and mitigations

• Cost function drifts from reality. A resolver gets slower, a list default changes, and prices no longer reflect work. Re-derive the cost model from measured resolver latency each quarter, and alert when the correlation between query cost and execution time drops.
• Introspection and mutations priced like reads. A mutation that triggers three downstream writes costs far more than its node count suggests. Apply explicit per-field weights for expensive mutations rather than relying on structural cost alone.
• Budget multiplication by connection count. Without max_conns_per_account, per-connection budgets are trivially bypassed. Enforce a connection cap at the handshake, and count connections as a limited resource in their own right.
• Mid-stream limits with no exit. If your gRPC interceptor only checks at stream start, a well-behaved opener can flood later. Charge per message inside the stream handler and end the stream with RESOURCE_EXHAUSTED when the budget is gone.
• 429 semantics lost in the protocol. GraphQL returns 200 with an errors array by default, so clients that only inspect HTTP status never notice they were limited. Put a machine-readable code in extensions and document it; where the transport allows, also send the header triplet.
• Webhook retry amplification. Retrying a failed delivery from three workers because the job was not locked triples the load on a subscriber that is already struggling. Use an idempotency key per delivery and a single-flight lock per endpoint.

Response contract per protocol

The client-facing contract must be discoverable in whatever channel the protocol offers. For GraphQL, that means a stable extensions.code of RATE_LIMITED plus cost, remaining, and retryAfter fields, mirrored into X-RateLimit-* headers on the HTTP transport so proxies and dashboards see them too. For WebSockets, define a message envelope ({ "type": "error", "code": "rate_limited", "retryAfter": 2 }) and reserve close code 1008 for account-level breaches. For gRPC, return RESOURCE_EXHAUSTED with retry-after-ms in the trailing metadata — never UNAVAILABLE, which instructs well-built clients to retry immediately. For outbound webhooks, the contract is yours to honour, not to emit: respect the subscriber’s Retry-After and treat a 429 from them as a signal to slow that endpoint’s queue.

Migrating a REST limiter to cost-based counting

Teams rarely start with a cost function; they arrive at one after a GraphQL endpoint or a batch API makes request counting meaningless. The migration is mechanical if you take it in order.

• Instrument first: record execution time and downstream call count per operation for two weeks, keyed by operation name.
• Fit a cost model to that data — usually nodes × page size for GraphQL, or an explicit weight table for batch endpoints — and check the correlation between predicted cost and measured time.
• Deploy the cost function in shadow mode: compute and log the price, keep enforcing the old request count.
• Compare distributions. If the 99th-percentile priced cost is 40× the median, your capacity must be sized for the tail, not the median.
• Choose capacity so a typical client’s normal working set fits comfortably, and max_single_cost so no single operation can drain the bucket.
• Publish the cost model before enforcing it: clients cannot optimise a query against a price list they cannot see.
• Enforce for new API keys first, then existing ones, with a documented grace period and a per-key override for anyone the model prices badly.

The shadow-mode step is what makes the switch survivable. A cost model that looks reasonable in a design document routinely prices a customer’s main dashboard query at three times their whole budget, and shadow logging surfaces that before it becomes an incident.

A second, subtler decision: charge before or after execution? Charging before is safe — the client cannot exceed the budget — but the price is an estimate, and a query that turns out cheap has still been billed for its worst case. Charging after is accurate but arrives too late to prevent the work. The production compromise is to charge the estimate up front and refund the difference when the operation finishes measurably cheaper, capped so that refunds cannot exceed the original charge. A refund is a second atomic call, so only do it where the estimate error is large enough to matter.

Documenting costs so clients can adapt

A cost-based limit is only fair if clients can predict it. Three things must be published, and all three belong in the API reference rather than a blog post: the price list (how each field, method, or message type is weighted), the budget (capacity and refill rate per plan), and the runtime signal (where the client can read the price it was charged and the balance remaining).

The runtime signal is the one most implementations forget. For GraphQL, return the charge in the response extensions:

{
  "data": { "organisation": { "repositories": { "nodes": [] } } },
  "extensions": {
    "rateLimit": { "cost": 1051, "remaining": 3949, "resetIn": 47, "maxSingleCost": 5000 }
  }
}

With that block present, a client library can log the expensive queries in its own telemetry, a developer can iterate on a query until its price drops, and a support conversation about “your API is throttling us” starts from a number instead of an argument. The equivalent for WebSockets is a periodic {"type":"budget", ...} frame; for gRPC, the trailing metadata; for outbound webhooks, your own delivery logs shared with the subscriber.

In this section

• GraphQL Query Cost Rate Limiting — pricing documents before execution and returning a usable error.
• WebSocket Message Rate Limiting — per-connection budgets, account roll-up, and close codes.
• gRPC Interceptor Rate Limiting — unary and streaming interceptors with RESOURCE_EXHAUSTED.
• Webhook Delivery Throttling and Retries — per-endpoint pacing, honouring subscriber limits, and safe retries.

One last principle applies across all four protocols: whatever unit you charge in, publish it. A client that knows a query costs fifty units and its budget is fifty thousand can plan its work; a client that only discovers the price by being rejected will retry, complain, or both.

The four protocol guides below each apply that principle to one transport, with the cost function, the enforcement point, and the rejection signal spelled out concretely for that protocol’s tooling.

Start with whichever protocol is already causing you trouble; the cost function is the transferable idea, and once one protocol is priced correctly the others follow the same three phases of identify, price, and consume.

Related

• Backend Middleware & Distributed Tracking — the parent reference for enforcement placement and state.
• Token Bucket Implementation — the primitive every cost function feeds.
• Redis Counter Architecture — the shared store behind cost-aware buckets.
• Edge & Gateway Enforcement — the tier that shields these protocols from volumetric abuse.
• Retry Queue Implementation — the client-side counterpart to outbound delivery pacing.