One Name Is Enough to Flip the Language

Realtime voice models can switch languages mid-utterance on their own. Reports on the OpenAI developer community describe GPT Realtime 2.1 exhibiting reproducible language drift even when the system prompt is written entirely in Spanish or German — and switching to Russian when a caller mentions "Anastasia," or to Arabic for "Amir."

In a callbot loop this breaks the conversation outright. A session running in Korean that jumps to English or Russian over a single agent name leaves the caller repeating themselves without knowing what went wrong. Language switching isn't an edge case to special-case later — it's a gate that sits on the normal input path of every turn.

The Detection Signal Is a Probability, Not a Fact

OpenAI's realtime transcription guide states that setting the language parameter in ISO-639-1 format improves both accuracy and latency. Even so, speech in other languages can still get recognized, and when the model can't produce a reliable prediction the detected-language field comes back as an empty array. "Detected language" is a probabilistic signal with fluctuating confidence, not a value you can hand routing decisions to on its own.

The dedicated translation endpoint sidesteps this uncertainty structurally: it's designed around one session per output language, removing any room for language to drift mid-session. A callbot loop should borrow the same idea — own the language decision explicitly at the application layer instead of leaving it to the model's implicit inference.

From Design to Operations: A Language-Switch Gate Checklist

Planning starts by fixing switch-related target numbers. A reasonable baseline: false-switch rate under 2% of turns, recovery to the prior language within 1 turn of a misdetection, and a confirmation-question response rate above 90%. Without these numbers, you only learn about drift from complaints filed after the fact.

Failure patterns fall into four groups: a name or proper noun triggering an unwanted switch; drift recurring in long conversations even with the language pinned in the system prompt; initial misdetection from background noise or accent when the language parameter is left unset; and an empty detected-language array that gets mishandled — either forcing the previous language through unchanged or resetting blindly to a default.

Recovery branches need to run automatically, without waiting on a human. When confidence is low or the field comes back empty, hold the prior turn's language and insert one short confirmation question — "Should I continue in Korean?" If switch attempts exceed two in the same session, lock automatic switching and hand off to a human agent. Send the confirmation question itself in the candidate language so the caller doesn't misread its intent.

The operations checklist starts with the log schema: detected language code, confidence or empty-array status, the utterance span that triggered the switch, whether a confirmation question fired, and the final locked-in language — recorded per session so a false-switch incident can be replayed later. Pre-deployment scenario tests need dedicated tracks for utterances containing proper nouns, heavy accents, and code-switching sentences that mix two languages. PII masking rules also need to be split by language, since phone-number and ID-format regexes differ across locales.

The improvement loop starts with a weekly review of false-switch logs to build a list of proper nouns that keep triggering the bug. Feeding that list back into the system prompt as a context hint cuts recurrence from the same names. Every time a new language is added, extend the regression suite with a code-switching scenario for it — coverage should grow with the language list, not lag behind it.

What Callbot Teams Should Check Today

Because the detected-language field is a probability rather than a fact, a callbot loop needs an explicit detect-confirm-recover gate, with a false-switch rate under 2% and recovery within 1 turn as its target numbers. Real reports already document a single name flipping the response language — leave it unhandled, without confirmation questions or replayable logs, and callers abandon the session without ever knowing why.

References

Realtime transcription guide — OpenAI

GPT Realtime 2.1 exhibits language drift — OpenAI Developer Community