86,400 Calls Could Be Sitting in a Queue: Designing a Capacity and Backpressure Gate for the Callbot Loop

Two Separate Capacity Ceilings: Telephony and the LLM

Teams scaling a callbot loop often track "how many concurrent calls can we handle" as a single number. In practice, each layer enforces its own, separate ceiling. On the telephony side, a carrier such as Twilio caps concurrent active calls and new call creations per second (CPS) independently. Twilio's own error documentation states that exceeding the account's concurrent-call allowance fails immediately (error 10004), while call-creation requests that exceed the CPS limit aren't rejected — they're queued for up to 24 hours and processed in order. At a nominal 1 CPS, that single queue can hold up to 86,400 calls.

The reasoning layer imposes a different ceiling. Anthropic's official rate-limit documentation caps requests per minute (RPM), input tokens per minute (ITPM), and output tokens per minute (OTPM) using a token-bucket mechanism, where capacity refills continuously rather than resetting once a minute — so a "60 requests per minute" allowance can, in practice, only tolerate bursts around 1 request per second. When call volume spikes, it's common for the LLM layer to throttle first even while the telephony layer still has headroom.

Two Failure Signals an Average-Load Dashboard Never Shows

These two ceilings fail differently. Exceeding the telephony concurrency limit fails immediately and loudly — the call-creation API returns an error you can retry or alert on right away. Exceeding CPS fails quietly instead. A queued call looks like it's "in progress" while it may not actually dial for hours, and it gets auto-cancelled once it passes 24 hours in the queue. If a dashboard only shows average concurrent calls, that delay stays invisible until a campaign wraps and someone asks why the call count fell short of target.

From Design to Operations: A Callbot Loop Capacity and Backpressure Checklist

Start capacity design by multiplying the contracted CPS and concurrency ceilings by the actual average call duration to define a single "concurrent-call ceiling" number. Hard-code 80% of that ceiling as the alert threshold and 95% as the line where new call creation stops. Apply the same principle to the LLM layer: carve out roughly 70% of the contracted RPM as a budget dedicated to the callbot loop, kept separate from any text-chatbot or internal batch traffic sharing the same account.

When the telephony layer returns error 10004 (concurrency exceeded), log that attempt as an immediate failure rather than a queued one, then retry shortly after or route it to a lower-priority campaign. For calls sitting in the CPS queue, cancel and reschedule any that wait past a set threshold (say, 10 minutes) — that's safer than letting a call connect 24 hours later with the original context gone stale. When the LLM layer rejects a request, retry only once or twice with exponential backoff, then fall back to a pre-scripted response rather than leaving the call in silence.

Before deploying, load-test at 1.5x expected peak concurrency to see which layer saturates first. Log queue wait time, LLM turn-generation failure rate, and fallback-response count as dedicated fields on call records, and mask PII — dialed numbers and recordings — before storing them. Set the deploy bar at an LLM turn-generation failure rate under 0.5%, and a final failure rate under 0.1% after retries and backoff.

When queue wait and LLM retries spike at the same time, you need both metrics on one dashboard, overlaid. Show telephony-layer headroom and LLM-layer headroom as separate percentages, and give operators a single switch — usable manually too — that pauses new outbound call creation the moment either one crosses its alert line.

Declare the recovery condition in code as well. Resume call creation automatically once both layers' headroom stays above the alert line for 10 minutes, and ramp back up at half the normal rate right after resuming, so you don't immediately swing past the ceiling again.

Aggregate the top queuing, rate-limit, and concurrency-exceeded events every week to see which hours or campaigns keep hitting the bottleneck. If a pattern repeats at the same hours, adjust the dialer's pacing algorithm (the interval between call creations) or decide to raise the LLM tier — and back that decision with the data. Raising a contracted capacity limit is a cost decision that's hard to reverse, so base it on at least two weeks of headroom trend, not a single bad day.

Takeaways

A callbot loop's capacity ceiling sits separately in the telephony layer and the LLM layer, and the two fail differently — one loudly and immediately, the other quietly through queuing. Hard-code an 80%/95% alert line on the concurrent-call ceiling and a 0.5% LLM failure bar, then watch both layers' headroom on one dashboard, and you catch a shortfall before a campaign ends rather than after.

References

Rate limits — Claude Platform Docs, Anthropic

Error 30001: Queue overflow — Twilio