API Rate Limit Policy Simulator

--- name: api-rate-limit-policy-simulator description: Simulates API rate limits before you ship them - token bucket, fixed window, sliding log and sliding counter, per client, endpoint, tier or global - against generated or recorded traffic with retrying clients. Shows who gets 429s and for how long, fixed-window boundary bursts, retry herds, noisy neighbours, abusive clients that still get through and peaks over backend capacity. Includes an offline stdlib Python simulator with --self-test. --- # API Rate Limit Policy Simulator You help backend and platform engineers choose rate limits with evidence instead of round numbers. A limit that looks safe on paper ("300 per minute per key") can still let 1.8x that through at a window boundary, throttle a mobile app whose thousands of users share one API key, release a herd of retries at the reset second, and do nothing against a scraper that stays just under it. You describe the policies and the traffic in a small JSON file, replay it, and read the result client by client and policy by policy. ## Files in this skill - `scripts/rate_sim.py` - simulates token bucket, fixed window, sliding log and sliding counter limiters keyed by client, endpoint, client+endpoint or global, scoped to endpoints and client tiers; generates steady, Poisson or burst traffic (or replays a CSV trace), lets clients retry immediately, with backoff or by honouring Retry-After, and reports 429s, delays, failures, worst-case and observed window counts, peaks vs capacity, findings RL01-RL10 and a sample 429 response (Python 3.8+ standard library only) - `references/algorithms.md` - how each algorithm behaves, its worst case, memory cost and when to pick it - `references/policy-design.md` - choosing keys, tiers, numbers, headers and client retry rules; what the finding codes mean - `templates/rate-limit-review.md` - the review to hand back - `examples/orders-api.json` - a public orders API with a per-key fixed window, a login limit and a global fixed window; five clients including a shared mobile key, a bulk partner, a scraper and a credential stuffer - `examples/orders-api-report.md` - the real simulator output on that file - `examples/orders-api-tuned.json` - the same API after the fixes: per-tier token buckets, a global token bucket sized to capacity ## Workflow ### 1. Collect the inputs Ask for: the limits as configured today or as proposed (algorithm, key, numbers, which endpoints and which clients), where they are enforced (gateway, CDN, app, Redis), the backend capacity in requests per second it can sustain, and the traffic: the biggest legitimate clients and their shape (steady, bursty batch jobs, many users behind one key or one NAT IP), how each one retries, and any abuse seen in logs. Real access logs beat guesses: export `t,client,endpoint` for a busy 5-15 minutes and pass it with `--trace`. ### 2. Run the simulator ``` python3 scripts/rate_sim.py examples/orders-api.json python3 scripts/rate_sim.py examples/orders-api-tuned.json python3 scripts/rate_sim.py policies.json --trace access-sample.csv python3 scripts/rate_sim.py policies.json --json python3 scripts/rate_sim.py --self-test ``` Exit 0 = no high findings, 1 = high findings (legitimate clients throttled or admitted traffic over capacity), 2 = bad input. Without a shell, reason with `references/algorithms.md`: per key, compare the client's peak per window with the limit, the worst case (2x limit for fixed windows, burst + rate for buckets) with capacity, and think through what each client does after a 429. ### 3. Read the result - Legit clients first (RL01): any failed or delayed requests, and how long they waited. One key shared by an app's users needs its own tier or a per-user key. - Capacity (RL03): the peak admitted second vs what the backend sustains. A global limit is the only thing that protects capacity; per-key limits multiply by the number of keys. - Shape (RL04, RL10): fixed windows let bursts through at boundaries and hand out a whole window at once; retries without jitter all come back in the same second. - Abuse (RL02): a scraper under the per-key limit is not stopped by limits; it needs detection, auth or a per-endpoint limit. - Retries (RL05): attempts per logical request above 1.5 means the clients amplify load when throttled. ### 4. Tune and re-run Change one thing at a time and re-run with the same traffic: switch fixed windows to token buckets or sliding counters, add tiers with `clients` / `exclude_clients`, size the global bucket to capacity (rate = sustainable rps, burst = what the backend absorbs for a second), add endpoint limits for login and expensive calls, and agree pacing with batch partners. Use `references/policy-design.md` for the numbers. ### 5. Deliver Fill `templates/rate-limit-review.md`: verdict, a before/after table per client, the policy changes as config, the 429 response and headers, client guidance (Retry-After, backoff with jitter, max retries) and what to monitor after rollout. ## Example result On `examples/orders-api.json` the shared `mobile-app` key hits the 300/minute per-key window: 1,042 of its 1,839 requests succeed only after waiting up to 60 s (RL01, high). Honouring Retry-After without jitter releases 289 retries in second 240, so admitted traffic peaks at 313 req/s against a capacity of 80 (RL03, RL10). The per-key fixed window lets 544 requests through in one rolling minute (1.8x, RL04), and the scraper still gets 45% of its calls through while its immediate retries push it to 2.67 attempts per request. The login limit stops the credential stuffer (40 of 480). In `examples/orders-api-tuned.json` (tier token buckets, global bucket 50 req/s + 30 burst, partner pacing at 2 req/s) legit clients see no 429s, the peak is 27 req/s and only the scraper findings remain. ## Rules - Simulate with the traffic you actually have (or a trace), not an average; bursts and retries are where limits fail. - Never call a limit safe for capacity unless a global limit bounds the total. - Treat delayed requests of legitimate clients as harm, not as success. - Always return 429 with Retry-After, add jitter, and tell clients how to back off. - Rate limits are not abuse protection on their own; say when authentication, bot detection or WAF rules are needed. - Results are a model: confirm with gateway metrics after rollout and keep the limits in config, not code.