Launch is tomorrow, and a colleague asks what the voice agent does if a customer says something odd. AI voice agent failure modes repeat across sectors. Ten examples follow, each with sample dialogue, cause and fix.
Ten failures at a glance
Each row matches a numbered section below. Use the last column as your pre-launch test list.
| # | Failure | Cause | Fix | Test |
|---|---|---|---|---|
| 1 | Hides it is AI | No honest reply | Disclose up front | Ask "Are you a robot?" |
| 2 | Invents answers | Thin knowledge brief | Add "I don't know" rule | Ask 20 off-topic questions |
| 3 | Voicemail as person | No machine detection | Voicemail script | Call a voicemail phone |
| 4 | Talks over customer | Latency | Tune turn-taking | Call over a real line |
| 5 | Mishears numbers | Accent errors | Read back values | Compare transcript to record |
| 6 | Sounds fake | Stiff script | Native-speaker review | Locals rate calls |
| 7 | No human exit | No trigger | Staff the handover | Ask for an agent three times |
| 8 | Calls too often | No retry cap | Set caps | Check retry log |
| 9 | Over-promises | No never-say list | Fail call on one breach | Bait it with fee requests |
| 10 | Silent line | No carrier audio | Watch silent answers | Listen to daily sample |
1. The agent hides that it is AI
Customer: "Am I talking to a robot?"
Agent: "I'm calling about your payment. Do you have a minute?"
The root cause is usually an optional disclosure setting. The EU has required AI phone systems to say so since 2 Aug 2026. In the US, AI-voice calls already need consent under the TCPA (FCC 2024 ruling). Colorado follows on 1 Jan 2027.
Fix it with an upfront disclosure and one truthful reply. Ask the question in every test call.
Buyers weigh legal risk against hostile reactions from some customers. We advise disclosing.
2. The agent invents an answer
Customer: "Is there a late fee if I pay on Friday?"
Agent: "No, there's no fee if you pay this week."
The fee was never in the brief. The root cause is a thin knowledge brief, because the model fills gaps with a confident guess. Regulated buyers fear this most, since banks say the main blocker is an agent that "goes rogue".
Keep the brief to checked facts. Add a rule: "I don't know, let me get someone." Then ask 20 out-of-scope questions in testing.
3. Voicemail gets treated as a person
Agent: "Hi Sam, about your loan balance of…"
(The greeting beep sounds. The agent keeps pitching to a machine.)
The root cause is no answering-machine detection and no voicemail script, so the agent reads the full pitch, balance included, to whoever hears the message later. Write a separate voicemail script with a minimal disclosure and no account details. Then count only calls where the customer spoke as reached. Call a phone set to voicemail before launch.
4. The agent talks over the customer
Customer: "Yes, I can pay on the…"
Agent: "…so the balance due is four hundred. Would you like to continue?"
Customer: "Hello?"
(Three seconds of silence follow.)
The root cause is turn-taking. Each reply passes through speech to text, the model, then voice synthesis, and every step adds delay. Weak barge-in settings stop the agent yielding when the customer speaks. Test on a real phone line, not only in the browser. Add background noise, strong accents and people who pause mid-sentence.
5. It mishears numbers and names
Agent: "So you'll pay fifteen thousand on the thirteenth?"
Customer: "No, fifty. On the thirtieth."
Agent: "Great, thank you. Goodbye."
The cause is a transcription slip on digits, names or a local accent, so the wrong value goes into the record. "Fifteen" and "fifty" sound alike on a phone line. Read back every amount, date and ID, and wait for a clear yes. Then compare each transcript with the fields it filled.
6. The voice sounds fake or foreign
Customer: "Why are you talking like that?"
Agent: "I'm sorry, how can I help with your account today?"
(The customer hangs up.)
The root cause is a generic accent, the wrong language, or stiff script wording. Buyers fear AI that sounds scripted. Local voice matters too, because people tune out foreign accents.
Pick the language before the voice. Rewrite the script for speech, with short sentences. Insight7 voice campaigns run in Yoruba and Igbo, with Pidgin in beta. Have native speakers rate test calls.
7. No way out to a human
Customer: "Agent, please."
Agent: "I can help with that. What is your account number?"
Customer: "Agent. Please. I want a person."
Agent: "I can help with that. What is your account number?"
The root cause is no escalation trigger, or no one staffed to take the handover. Buyers ask whether the handover is polite, what happens with a disputed amount, and whether anyone chases an unanswered handover. Gartner found that 87% of 3,566 customers say a company using GenAI for service must offer a way to reach a human.
Set triggers for anger, disputes, sensitive topics and any request for a person. Send callbacks to a queue someone owns.
8. It calls too often, too late
Customer: "You called me twice this morning, and I said stop."
Agent: "Sorry about that. Do you have a minute to talk about your balance?"
The root cause is missing guardrails: no calling hours, retry cap, do-not-contact list or consent flag. Set each one before the first call goes out. Check the retry log for any contact reached more often than the cap allows.
9. It promises what it cannot deliver
Customer: "Can you take off that late fee?"
Agent: "Of course, I'll waive that for you. Your loan is approved too."
The root cause is no hard line between what the agent may say and what only staff may decide. A friendly model drifts toward yes. Write a never-say list covering waivers, approvals, rates and legal advice. Then bait the agent with those requests in testing. Score one such promise as a failed call. QA leaders say one critical error should fail the whole call. A procedure score can hide the mistake that worries a bank's risk team.
10. The line goes silent and nobody notices
(The call connects. The customer hears nothing and says nothing.)
Agent: "Hello, I'm calling about your repayment. Can you hear me?"
Agent: "Hello?"
The cause is a carrier or gateway that connects the call but delivers no audio. The dashboard still shows "answered". Track the share of answered calls with no customer speech. Listen to a sample daily in week one. Pause the campaign when that share spikes.
Run these tests before any launch
Turn each failure above into one test, and run them in three stages. First, talk to the agent in the browser to catch script and knowledge gaps. Second, ring your own phone to expose latency, voicemail and audio problems. Third, run a small live pilot, then read every transcript and score each call. Banks want this short, low-risk kind of pilot.
Insight7's voice campaigns are in beta. You can test an agent in the browser with no phone number. A campaign also has a test call to your own phone, consent marking, calling hours, retry rules, a do-not-call list and a scorecard. A copied campaign leaves out anyone added to the do-not-contact list since the original. Calls where only the AI spoke are not counted as reached, and a carrier's temporary refusal does not use up a retry. A dashboard lets a reviewer read each transcript at call end.
Adapt the list to your sector
Weight the ten failures by what your campaign does. Collections and reminder campaigns lean on failures 8 and 9, because they call often and discuss fees. Surveys and service check-ins weigh more on 3 and 6, since many people prefer a message to a call. Inbound lines weigh most on 7. Write each sample dialogue from your own past calls. Log every new failure as an eleventh row.
Related reading: AI Agent Regression Testing Checklist for Every Script or Prompt Change, How to Build a Go-Live Approval Gate for AI Agents in Regulated Industries and AI Voice Agent Test Case Template With 10 Scenarios Every Campaign Should Pass.
Questions people also ask
How do you test voice AI agents before launch?
Turn each of the ten failures into a scripted test call. Use a tester who did not write the script. Record every call, and rerun the whole set after each script or voice change.
What are the most common reasons AI voice agents fail?
Most failures trace to gaps in setup: a thin knowledge brief, no escalation rule, no calling limits and an untested phone line. Each is cheap to close before launch.
Do AI voice agents have to say they are AI?
In a growing list of regions, yes. EU AI Act Article 50 has applied since 2 Aug 2026 and covers automated phone systems. Colorado adds a similar duty from 1 Jan 2027.
What are the consent requirements for AI outbound calling?
Rules vary by region. In the US, AI-voice calls need consent under the TCPA, following the FCC's 2024 ruling. Record consent per contact, with its source and date.
How should an AI voice agent hand off to a human?
Hand off with context. Pass the reason and a short summary so the customer does not repeat themselves. Insight7's inbound voice lines are in beta, and callback requests go to a queue.
How many test calls are enough before going live?
Cover all ten failures at least once on a real phone, then repeat with different voices, accents and background noise. You are ready when a full round finds no new type of failure.
What numbers show that a voice agent is failing in production?
Watch the share of answered calls with no customer speech, early hang-ups, repeated requests for a person, and transcript and field mismatches. Measurement matters, since 42% of 432 AI customer-service use cases analysed by Gartner could not be measured.
Listen before you launch
Listening finds most voice agent failures, because a dashboard can still say "answered" while the customer hears silence. Pick your three riskiest failures, write a test call for each, and run them on your own phone before launch.
