hotato GitHubGet started

The voice agent failure atlas

48 failure modes82 linked reports20 stacksdata as of 2026-08-27

Named failure modes in voice agents. Each one collects every public report of the same bug, links back to the builder who hit it, and carries a status re-checked daily.

Failure modes

ModeStatusReportsReported onLast reported
Barge-in teardown raceWhen the caller interrupts the agent mid-speech, the system's interrupt handling fails to cleanly tear down the in-progress turn, corrupting the conversation state.acknowledged11assemblyaiazure-speechdeepgramelevenlabslivekitopenai-realtimepipecatsilerotwilio2026-08-20
Endpoint misdetectionEnd-of-turn detection fires at the wrong moment, so the agent either answers before the caller has finished or waits after they have.acknowledged10azure-speechcartesiadeepgramelevenlabskokorolivekitnemopipecatsilerotwiliowhisper2026-08-06
Self-echo VAD false triggerThe agent's own synthesized speech bleeds into the audio input and voice-activity detection mistakes it for the caller starting to talk, triggering a false interruption.open2livekitsilero2026-08-23
Silent TTS failover deadlockThe agent's speech synthesis silently produces no audio even though it logs the turn as spoken, and the safeguard meant to detect and recover from this never triggers, leaving the caller in permanent silence.acknowledged4cartesiaelevenlabshome-assistantpipecatpiper2026-08-10
Interruption fallback overstates delivered textWhen the caller interrupts mid-response and the system lacks word-level delivery timestamps, it falls back to recording the full intended text as spoken, so the conversation history includes words the caller never actually heard.acknowledged2deepgrampipecattelnyx2026-08-25
VAD retrigger cancels committed turn deliveryDuring the short delay before an already-finished user turn is delivered, a fresh VAD speech-start event unconditionally cancels that delivery, so the caller's completed utterance is silently dropped and never reaches the agent.open1livekitsilero2026-08-27
Mid call reconnect drops buffered contentWhen the live connection to the speech/language service needs to reconnect mid-call, buffered audio, text, and tool results that were in flight are silently discarded instead of resent or recovered.open1gemini-livepipecat2026-08-27
AMD dual stream desync stallA call-answered classifier depends on two independently arriving signal streams, and when those streams fall out of sync a caller who answers and greets quickly is never classified in time, so the call stalls until the detection timeout.open1cartesialivekitwhisper2026-08-26
Audio gate cooldown blocks endpoint finalizationA local audio-level gate stops forwarding the caller's audio to the speech-to-text service after a brief silence cooldown, so the remote signal that would mark the turn finished never arrives and the call stalls.acknowledged2deepgramlivekitopenai-realtimepipecatsilero2026-08-20
VAD fires but STT yields no transcriptVoice-activity detection registers that the caller is speaking, but the speech-to-text stage fails to produce any transcript even though the same audio is clearly audible in a separate recording of the call.fixed2deepgramelevenlabslivekitpipecatwhisper2026-08-07
Muted greeting overlap strands turnWhen the caller speaks while their microphone is deliberately muted during the agent's opening greeting, the overlap corrupts turn-tracking so the caller is never heard again for the rest of the call.open1deepgrampipecattelnyx2026-08-25
Stream error recovery false successWhen the model's response stream errors out mid-turn, the system's retry logic logs a successful restart but never actually resumes speaking, cutting the agent off mid-sentence and leaving the caller silent for the rest of the call.open2cartesiaelevenlabslivekitpipecat2026-08-12
Generation timeout not cancelled on early abortWhen speech generation fails almost instantly, the agent still waits out its entire fixed timeout before reacting, leaving the caller in silence for seconds longer than the actual failure took.acknowledged2gemini-livelivekitopenai-realtime2026-08-12
VAD endpoint misdetectionThe system fails to detect the caller's speech because voice-activity detection misjudges frame windowing and retained internal state, so the caller appears silent when they are talking.open2livekit2026-07-24
Tool call cancel race stuck in progressWhen the caller interrupts a tool call before the system has finished marking it as started, the call gets permanently stuck flagged as in-progress, stalling the conversation.acknowledged2pipecat2026-08-09
Tool call mid-utterance audio truncationWhen the agent calls a tool while still speaking, the system marks buffered audio as fully played once generation finishes, cutting the agent's speech off mid-word and sometimes stalling the call.open2gemini-livelivekittwilio2026-08-07
Stale partial transcript phantom interruptionA stale partial speech-to-text result gets re-sent after the final transcript and is mistaken for new caller speech, cutting the agent off right as it starts answering.acknowledged2elevenlabsgemini-livepipecat2026-08-03
Teardown cancels trailing transcript commitWhen a call ends, the system cancels the still-in-progress task finalizing the caller's last speech-to-text result, silently discarding whatever the caller said right before the call closed.fixed1deepgramlivekit2026-08-18
Orphaned interrupted tail flushed post goodbyeWhen a response is interrupted before it can be properly closed off, the unspoken leftover text stays buffered and later gets synthesized anyway, so the caller hears a stray fragment after the conversation should have ended.open1pipecat2026-08-16
Voicemail detected end call frame droppedAfter correctly detecting a voicemail greeting, the agent's command to hang up the call is silently discarded by internal message handling, so the call never actually ends.open1pipecat2026-08-14
Post interruption context timing bleedAfter the caller interrupts the agent, word-timing data from the interrupted turn keeps being applied because generation timing isn't scoped per turn, corrupting the record of which words the caller actually heard.open1pipecat2026-08-14
Unanswered tool call blocks mic inputWhen a tool call is dropped without a response, the system permanently gates future microphone input on that unanswered call, silently discarding everything the caller says afterward.open1gemini-livelivekitopenai-realtime2026-08-09
Endpointing latency short replyAfter switching to neural voice-activity detection, the system takes an unusually long time to respond when the caller gives a short one-word reply.open2livekitultravox2026-04-27
Hold state audio leak exposes consult toolsWhen a caller is placed on hold during a transfer, their audio keeps reaching speech recognition, turn detection, and the model instead of being blocked, exposing tools meant only for the behind-the-scenes consult call.fixed1livekit2026-08-05
Interruption audio event id collision dropAn interruption event fired at nearly the same moment as the agent's reply causes the reply's audio to share the interruption's event id, so a staleness filter wrongly discards it and the caller hears nothing.open1elevenlabs2026-07-31
Turn completion filter dead-air timer orphanA turn-completion check accepts a degenerate, textless model output as a valid finish, ending the turn with no reply while also disabling the timer meant to recover from silence, leaving the caller in permanent dead air.open1pipecat2026-07-29
Idle reprompt turn latch response dropAfter the system re-prompts a silent caller, a turn-completion gate that only resets when the caller speaks treats the caller's reply as already answered and silently discards the agent's response.fixed1cartesiapipecat2026-07-28
Async interruption response misattributionA delayed decision about whether an earlier overlap counted as an interruption gets wrongly applied to a later, unrelated moment, cutting the agent off for speech that was never actually judged an interruption.open1livekit2026-07-26
Segmented turn interruption disarmWhen the agent's turn is split into multiple speech segments, such as around a tool call, a later segment resets interruption tracking so the caller's still-ongoing interruption is silently ignored.open1livekit2026-07-26
Agent interrupts callerThe agent starts speaking while the caller still holds the floor, cutting the caller off mid-utterance.open1livekitopenai-realtime2026-07-25
TTS word drop context raceA queue-ordering race in assembling the agent's own spoken output into conversation history drops the turn's final word from the stored record, even on a normal turn with no interruption, though the caller still hears the full sentence.fixed1cartesiaelevenlabspipecat2026-07-24
Dead-air after callerThe caller finishes and the line goes silent for seconds before the agent responds, with no audible acknowledgement in between.open1pipecat2026-07-24
Late STT final race after turn flushA speech-to-text result that finalizes just after the system has already ended and flushed the caller's turn is still processed, racing against the turn boundary and risking corrupted turn state.open1cartesiadeepgramlivekit2026-07-21
Tool fire after cancelA tool call the agent decided on before being interrupted still executes afterward, so an action like hanging up can fire even though the caller is still on the line.acknowledged1pipecat2026-07-10
Post hangup pipeline wedge blocks future callsAfter a call is torn down mid-pipeline, the system's call-handling state fails to reset, so later calls are answered but produce complete dead air with no agent activity until the service is restarted.open1asteriskhome-assistant2026-07-05
Tool call not cancelled on pipeline teardownWhen the call itself is torn down or cancelled, tool calls still in flight are never cancelled and keep executing in the background after the caller has left the conversation.fixed1pipecat2026-07-03
Tool call timeout zombie handlerWhen a tool call times out, the system stops waiting but never cancels the underlying handler, which keeps executing in the background and can trigger the same real-world action twice.fixed1pipecat2026-07-02
AMD premature timeout before answerOn outbound calls, the answering-machine-detection timer starts before the callee has actually answered, so ring or early media triggers a false timeout that the system wrongly treats as a voicemail greeting.fixed1livekittwilio2026-06-22
Tool call audio output silenceAfter the agent completes one or more tool calls within a turn, its spoken response never gets synthesized, leaving the caller in silence.acknowledged2deepgramlivekitopenai-agentsopenai-realtimesilero2026-02-06
Agent dispatch latency before connectThe agent process is dispatched to an inbound call so many seconds after it arrives that the caller has already hung up before the agent joins.open1livekittwilio2026-06-09
VAD background noise mistaken for caller speechVoice-activity detection mistakes background noise or nearby conversations for the caller talking, so silence-based inactivity timeouts never trigger even though the caller isn't actually speaking.open1cartesiadeepgramelevenlabslivekit2026-06-09
Forced close drain hang orphans connectionWhen a call is forcibly ended, the shutdown sequence that interrupts the agent and drains pending audio/state can hang indefinitely, so the session's speech-recognition connection is never released and the call never fully tears down.open1livekit2026-06-08
Half-duplex barge-in ignoredWhile the agent is speaking, the system never listens for the caller's speech, so any attempt to interrupt or clarify goes completely unheard until the agent finishes.stale1asterisk2026-05-18
Silence timeout teardown during processing gapWhen the agent's response takes too long to generate, the telephony layer reads the resulting silence as call inactivity and tears down the call itself, then misreports that the caller hung up.open1elevenlabslivekitopenai-agentstwilio2026-05-11
VAD false triggerVoice-activity detection treats non-speech or a backchannel as a floor claim, so the agent yields to something that was not a turn.open1livekit2026-05-10
Agent replies to own outputThe system mistakes the agent's own generated response for new caller input, causing the agent to reply to itself instead of to anything the caller said.acknowledged1assemblyaicartesiadeepgramelevenlabslivekitsilero2026-01-28
AMD tone detection premature teardownThe system stops listening for the voicemail beep tone as soon as it detects silence after the greeting, so a beep arriving just after is missed and the tone-triggered action never fires.open1twilio2017-05-30
AMD result arrives after call answeredThe system's answering-machine-vs-human classification only comes back after the call has already been answered and engaged, so the agent cannot act on whether it reached a person or a machine in time.open1twilio2014-09-22
How an entry gets here: collected, distilled, mapped, re-checked daily
  1. Collected from the source's own API or a robots-permitted public page, with the response cached and hashed.
  2. Distilled into a failure mode only when it names something that recurs across stacks. A vendor outage, a billing question or a feature request is declined and queued for a person.
  3. Mapped to a hotato command only when that command's subcommand exists in the shipped CLI.
  4. Re-checked daily. The status moves when the re-check stores the evidence link, quote and date behind the move.

Check your own call: hotato autopsy ./call.wav prints each barge-in, talk-over, dead air and latency incident in a recording you already have, with a timestamp and an id.The modes that belong are the ones inherent to running a spoken conversation, so they turn up on every stack: what the caller heard, on what, and how often.Submit a failure mode you have hitDiscuss an entry under the entry itself: what fixed it, what did not, and what the record is missing.For an agent: feed.json, feed.atom, feed.schema.json, atlas.llms.txt.Every report names the handle as it was posted and links the post; the removal path takes a quote down within 24 hours.atlas 0.1.0, data as of 2026-08-27T18:10:49Z