TTS 502/503 cold starts: retry strategy that works — Audra

You sent a perfectly valid request and got a 502. Nine times out of ten on a batch TTS API, that is a render container cold start: the gateway is up, the request is right, and the compute warming up behind the request answered before it was ready. It is transient by nature, which makes it the one 5xx you should retry automatically rather than page anyone about. This post is the retry policy that works.

Why it happens

Audra runs renders on containers that scale to zero and warm on demand — the cost model that keeps per-character pricing flat also means the first request after an idle period can land on a cold container. The gateway reports warm status per model tier, and a request routed to a tier that is not warm fails fast with 5xx rather than hanging. Two facts follow: it self-heals (the container warms), and it correlates with idle time, not with your request size.

The retry policy, in order

• Retry only 502/503 — and only a few times (3 is right). A 401 is a key problem, 402 is credits, 413 is size, 422 is validation, 429 is rate limit with an explicit Retry-After header in seconds. None of those are transient.
• Back off exponentially with the first wait short: 2s, 8s, 32s. Cold starts resolve in seconds-to-tens-of-seconds; the first retry at 2 seconds often succeeds outright. A flat 60-second sleep before retry one wastes more time than the failure did.
• Send the same Idempotency-Key on every attempt (UUID per logical render). POST /v2/speech replays the same audio when the key matches — retries cannot double-bill, and a retry that lands after a slow success returns the original result.
• Honor Retry-After when present: the gateway sets it on 429s and on operational 503s. It is in seconds; parse the integer, sleep that long, retry once more.
• Instrument the outcome: log status code, attempt count, and the X-QA-Passed header on success. A pattern of 502s concentrated after idle periods is cold starts; a pattern independent of timing is an incident — check /status next.

Retry code that fits the policy

import requests, time, uuid

key = str(uuid.uuid4())   # one key per logical render

for attempt in range(3):
    r = requests.post(
        "https://audratalks.com/v2/speech",
        headers={"Authorization": f"Bearer {AUDRA_KEY}",
                 "Idempotency-Key": key},
        json={"model": "audra-essential", "voice": "june",
              "text": script, "format": "mp3"},
        timeout=3600,
    )
    if r.status_code < 400:
        break
    if r.status_code in (502, 503) and attempt < 2:
        time.sleep(2 ** (2 * attempt + 1))   # 2s, 8s
        continue
    r.raise_for_status()

open("out.mp3", "wb").write(r.content)

Before escalating any retry-resistant 5xx, check /status and /blog/status-page-uptime-tts — the triage tree there distinguishes cold starts from incidents in one look. And remember the time budget: a ~500-character render legitimately takes 1–3 minutes; make sure what looks like a 'failure' is not just a client timeout shorter than the render.