rate limiting

why

a boundary open to untrusted callers is a budget anyone on the internet can spend. an in-memory limit resets on every restart or redeploy, handing attackers fresh quota; a limit checked after work begins spends money on calls it then rejects. only a durable limit, keyed by principal and operation and checked before any side effect, protects quota and cost.

rule

any boundary an untrusted caller can reach repeatedly gets an explicit, durable capacity limit before work begins. limiting is a boundary decision keyed by principal and operation — not an afterthought inside the handler.

in-memory is not durable: a restart resets it, and an attacker gets fresh quota again. at 100 calls/day with per-call provider billing, 10 redeploys in one day burn 1000 quota, not 100. a durable limit (database, Durable Object) survives restarts and protects quota and cost.
check before domain work: a verdict issued after a provider call or workflow start wastes money on the rejected call. guard before any side effect.

limit design

dimension	decision
key	authenticated principal id; unauthenticated fallback to the strongest stable identity available (ip + route).
scope	per operation or per resource family — the unit that exhausts real capacity.
unit	durable counter or window the runtime provides (KV, D1, durable object); restartable, visible.
enforcement point	before domain work and before expensive side effects, in the route guard.

the limit is capacity truth: it protects durable fan-out and capacity commitments from unbounded request pressure, and it protects provider quotas the product depends on.

response shape

a limited call is a typed 429-family diagnostic: the limit name, when the window resets, and no leaked detail about the counter's storage. the client surface maps it to a "too many attempts, wait" semantic state with a real wait time — per truthful product states, never a generic failure or a silent retry loop.

do not

enforce limits only in the UI or generated client; the server boundary is the owner.
keep limit state only in process memory — a restart must not reset abuse protection.
key unauthenticated limits on a spoofable value alone.
let a limited caller trigger domain work, provider calls, or workflow starts before the verdict.

verification

probe one protected operation past its limit with a valid credential: the first N calls succeed, call N+1 returns the typed limit diagnostic, and no domain side effect ran for the rejected call. restart the runtime and confirm the limit state survives. verify a different principal is unaffected — the key isolates callers.

good and bad

bad

// in-memory counter inside the handler, checked after the workflow starts
const hits = {};
export async function handleRequest(req) {
  hits[req.userId] = (hits[req.userId] || 0) + 1;
  if (hits[req.userId] > 100) throw new Error("slow down");
  await startWorkflow(req.input); // expensive work already started
  return ok();
}

a restart resets the counter to 0: 10 redeploys = 1000 quota violations instead of 100. the verdict arrives after the workflow starts, wasting compute and quota. the caller sees a generic failure, not a typed wait time.

good

// route guard, before domain work; durable counter keyed by principal+operation
const verdict = await limiter.check({ principalId, operation });
if (verdict.limited) return limitDiagnostic(verdict.resetAt);

a durable, restartable limit keyed by identity and operation decides before any side effect, and the caller gets a typed wait-time diagnostic. provider quota and cost stay bounded no matter how aggressively a caller retries or how often the runtime redeploys.