Why a TTS outage hits users harder than an STT or LLM one
An STT or LLM failure in a callbot loop can hide behind a second or two of silence, but a TTS failure breaks the exact audio the caller is listening to right now. On a phone call with no screen, one broken or dropped word is often enough for the caller to decide the line is dead and hang up, leaving far less room to recover than an STT or LLM stall. In the STT→LLM→TTS pipeline, TTS is the one stage a caller cannot be shielded from.
Only between utterances: the one constraint unique to TTS failover
LiveKit Agents' Fallback Adapter switches to a backup provider as soon as the primary errors out, marks the failed provider unhealthy, and periodically probes it so traffic can move back once it recovers. For TTS specifically, the adapter will not switch mid-utterance — it lets whatever audio is already playing finish, then applies the backup provider starting with the next sentence. A mid-turn switch on the LLM or STT stage is easy for a caller to miss, but a voice change inside the same sentence is heard immediately, which is why the switch point is pinned to the utterance boundary.
From boundary detection to recovery cooldown: running a TTS failover chain
At design time, pin numeric targets to the TTS stage rather than leaving them implicit: detect a primary error or timeout within 2 seconds, complete the switch before the next sentence starts (within roughly 1 second), and keep the whole TTS failover under a 3-second turn budget. A reasonable baseline is 99.9% monthly availability for the TTS stage, and the number of weeks that fall under that line becomes the health signal for the chain design itself.
One failure pattern is wiring the switch to fire the instant an error is detected instead of at the next utterance boundary. As noted above, LiveKit's TTS adapter only switches after the currently playing audio finishes; ignoring that boundary and splicing in the backup provider mid-sentence is what produces an audible voice change the caller notices right away.
A second pattern is leaving the backup voice to the provider's default instead of a pinned choice. A voice ID is a configuration value, not something a fallback auto-matches — LiveKit's own example pins Cartesia as the primary and Inworld's "Ashley" voice as the backup, a pairing chosen in advance for similar tone, gender, and accent. Skip that audition step and the caller can feel like they are suddenly talking to someone else.
The operations checklist needs circuit-breaker settings. VideoSDK's Fallback Adapter gives the TTS stage two tiers: a temporary disable window, and a permanent disable once failures hit a configured count in a row. Without the temporary window, a provider that is only intermittently failing gets retried on every single turn and the delay compounds; without the permanent tier, a provider that is fully dead keeps sitting at the top of the chain indefinitely.
Before shipping, force the primary TTS provider into a timeout or 5xx state and run a scenario test to confirm the switch actually lands inside the utterance boundary. Logs need provider ID, the reason for the switch, how long the switch took, and whether the switch point lined up with an utterance boundary — these fields are what make the pattern traceable after the fact. Because call audio and transcripts sit in the same logs, PII masking rules apply identically whether or not a fallback fired.
Weekly, tally switch frequency by provider and by reason to track the top causes. A provider that switches often should drop in chain priority, and a chain that almost never switches is itself a signal worth re-checking — the temporary-disable window may be too wide and quietly absorbing quality degradation the chain should have caught.
What to check right now
TTS failover in a callbot loop starts with switching only at utterance boundaries and pinning a backup voice's tone and accent ahead of time. Add a circuit breaker that separates temporary and permanent disables, log the switch reason and timing, and adjust chain order weekly — and callers can move to the next voice before they ever notice the provider underneath has gone down.
References
Fallback strategies — LiveKit Docs
Fallback Adapter — VideoSDK Docs
Ask AI about this article
The assistant has read this article. Ask anything — it answers from the text and says so when something isn't in it.
Loading the chat…