A 429 means you exceeded a limit. What is less obvious is that there are several limits and you can hit any of them.
Three kinds of limit
Requests per minute. How often you may call. Usually hit by people parallelising small tasks.
Input tokens per minute. How much text you send. Hit by anyone pushing large context: agents, repository work.
Output tokens per minute. How much the model generates. Hit by long answers or many concurrent generations.
The first is rarely the binding one. On agentic workloads you hit the second long before it.
Backing off correctly
A naive immediate retry makes things worse: you add load exactly when there is already too much. You want growing delay plus a random component.
import random, time
def call_with_retry(fn, attempts=5, base=1.0):
for i in range(attempts):
try:
return fn()
except RateLimitError:
if i == attempts - 1:
raise
delay = base * (2 ** i) + random.uniform(0, 0.5)
time.sleep(delay)
The random part stops ten of your workers retrying in lockstep. Without it you get waves of failures instead of steady throughput.
Read the response headers
The API tells you how long until reset. That beats guessing: if the server says wait twenty seconds, wait twenty, not the thirty-two your formula produced.
Not hitting limits at all
Do not send a whole file when you need a method. The input limit is spent on context, and context compounds on agentic runs.
Use named keys for accounting. Separate keys show background and live usage separately; they do not create separate quotas. A client queue and rate limiter control background and live load.
Queue work and bound request rate. Sequential requests can still exceed a provider quota. Use a bounded queue, a client rate limiter, and provider retry guidance when available.
Cache repeated context. If every request drags the same large instruction along, cache it provider-side.
On separate keys
Name keys for production, background work, and experiments to keep usage accounting separate. Keys do not isolate quotas; control each workload with a client queue and rate limiter.
Claudexia shows usage statistics per key. Name keys by workload, choose Premium or Cheap, and optionally set expiry or Premium fallback for Cheap. The dashboard key API does not expose per-key RPM, TPM, concurrency, or budget settings.
In short
Requests, tokens, and concurrency can all affect a workload. Use bounded retries with jitter, respect retry headers when available, and control background and live traffic with a client queue and rate limiter. Named keys provide separate usage accounting, not independent quotas.