TL;DR
- Every AI voice agent fallback should be one of six planned moves.
- Pick the fallback by risk, from a simple clarify to a warm handoff.
- Always tell the caller what happens next.
1. Ask once to clarify, then rephrase
Ask one clarifying question, rephrase once if needed, then move to another fallback. Many "can't answer" moments are not knowledge gaps. A noisy line, a clipped word or a transcription error can turn "my card was declined" into something the agent can't match. A clean second try fixes most of these.
Here is the pattern:
- Say what you heard, so the caller can correct it.
- Rephrase the question once, in simpler words.
- Cap the loop at two failed turns. A third "Sorry, can you repeat that?" is where callers hang up.
Sample dialogue:
Agent: I want to check I have this right. Are you asking about a late fee on your March invoice?
Caller: No, the fee on the, uh, the transfer.
Agent: Thanks. Is that a fee for sending money to another bank?
Caller: Mm, I can't hear you well.
Agent: The line is breaking up. Let me connect you with a teammate who can help. I'll stay on while I do.
The last line is the exit to the next fallback.
2. Answer what you know, park the rest
Answer the part of the question you can, say plainly what you can't, and log the open piece. Callers fear a bot that goes quiet or recites a script when they step off it. A partial answer shows the agent is listening.
Ground every answer in an approved source. A knowledge base built from your own docs, product guides and playbooks gives the agent one source of truth, so it has no reason to guess. You can read more on the AI Knowledge Base page. If the answer is not in the source, the agent says so.
The pattern:
- Answer the in-scope part first.
- Name the gap in one plain sentence.
- Say who will close it and by when.
- Log the open question with the call.
Sample dialogue:
Caller: What's the fee for an international transfer, and can I raise my daily limit?
Agent: The transfer fee is two dollars for amounts under one thousand. I can't change limits on this line, and I won't guess at the rules. I've noted your request, and a specialist will call you tomorrow morning. Is this number the best one?
Caller: Yes, that works.
Repeated gaps in the log tell you which documents to add.
3. Offer a callback and capture details
Offer a callback when no human is free or the office is closed. Then collect four things before the call ends. A callback only works if the caller trusts it will happen, so the agent must capture enough to make it happen.
Capture these:
- Name, as the caller says it.
- Reason, in one sentence.
- Best time, with time zone.
- Number, read back and confirmed.
Sample dialogue:
Agent: I can't answer that one, and our billing team has left for the day. May I arrange a callback?
Caller: Sure.
Agent: What's the best time tomorrow?
Caller: After ten in the morning.
Agent: Thanks. So that's a call after ten about a double charge, on the number ending in 4417. Is that right?
Caller: Yes.
Agent: Done. A billing specialist will call you tomorrow after ten.
Give each queued callback an owner and a deadline. A request nobody owns is a silent failure.
4. Hand off warm, with context attached
Transfer the call with a summary attached, so the caller never repeats themselves. A cold transfer drops the caller into a queue with no history. A warm transfer gives the human the story first.
Pass three things with the call:
- The transcript so far.
- The caller ID and any account the agent matched.
- The reason for the transfer, in one line.
Send them through your SIP transfer and write them into the CRM record, so the human sees them as the call rings in. Buyers often ask whether the handover is polite and whether anyone chases an unanswered call. Give every transfer an owner and a timeout. If no one answers, fall back to a callback.
Callers expect this. In a Gartner survey of 3,566 customers, 87% said a company using GenAI for service must offer a way to reach a human.
Sample dialogue:
Caller: Are you a real person?
Agent: No, I'm an AI assistant. I can bring in a person now. I'll tell her what we discussed.
Agent: Connecting you to Maria in billing. She knows you're asking about a double charge on your March statement.
Disclosure rules differ by region:
- EU: Article 50 of the AI Act requires automated phone systems to say they are AI from the first interaction.
- US: the TCPA covers AI-generated voices in outbound calls.
- Colorado, US: the Chatbot Safety Act, effective 1 January 2027, requires public-facing conversational AI to disclose it is AI.
5. Escalate sensitive topics at once
Write a list of trigger topics in advance. When one comes up, send the caller to a human straight away. Regulated buyers, banks first, say their main worry is an AI agent that goes off script and makes a public mistake. Buyers also ask what happens to a disputed amount or a sensitive issue.
Common triggers:
- A billing dispute or a disputed amount.
- A complaint.
- A caller who sounds vulnerable or distressed.
- Legal, regulated or fraud topics.
- An explicit request for a person.
- A sharp drop in caller sentiment.
Treat a mishandled trigger as a failed call. QA leaders say one critical error should fail the whole call, even when the procedure score looks perfect.
Complaint handling is also regulated in many markets, such as banking regulators in Africa, Europe and Asia. In one example, the CBN fined institutions N430M in 2025 for slow complaint handling.
Sample dialogue:
Caller: This charge is wrong, and I want to file a complaint.
Agent: I'm sorry about that. A complaint needs a person, so I'm passing you to a specialist now. I'll send her the charge and what you told me.
Caller: Fine.
Agent: You don't need to explain it again. Please stay on the line.
6. Fail safe: exit politely, route onward
Apologise once, give the caller a next step by text, email or phone line, and log the call. Use this when nothing else works: no human is free, a tool is down, the line is silent, or the loop cap is hit. The caller should hang up knowing what happens next.
The pattern:
- Apologise one time, in one sentence.
- Name the route: an SMS link, an email or the support number.
- Send the message before the call ends.
- Tag the call with its reason: silence, loop or system failure.
Sample dialogue:
Agent: I'm sorry, I can't reach your account right now, and I don't want to keep you waiting. I'll text you a help link and our support number. Is this mobile the right one?
Caller: Yes.
Agent: Sent. A teammate will also see this call and email you today. Goodbye.
Fallback calls show where your knowledge gaps are. Score them, read the transcripts, and fix the missing documents. Insight7's AI call scoring scores every call, so you can review fallbacks without sampling. In the voice beta, a dashboard lets a reviewer read each transcript when the call ends.
Measure the result. Gartner looked at 432 AI customer service use cases. Only a quarter showed positive ROI, and 42% could not be measured. So measure fallback outcomes yourself, or you cannot show the design pays off. Track fallback rate by reason weekly.
Related reading: 11 Reasons Customers Hang Up on AI Voice Calls and How to Fix Each One and Outbound Dialers vs AI Voice Agents Compared for High-Volume Customer Campaigns.
Questions people also ask
What should an AI voice agent say when it doesn't know the answer?
It should say so plainly, then name the next step. For example: "I can't answer that, and I won't guess. I'll have a specialist call you tomorrow." Admit the limit, offer a route, and confirm the details.
How many times should a voice agent retry before handing off to a human?
Allow one clarifying question and one rephrase, so two failed turns at most. A turn fails when the agent finds no match or the caller repeats themselves. After that, hand off or book a callback.
What is a warm transfer in an AI voice agent?
A warm transfer connects the caller to a person after the agent passes along a summary, the transcript and the reason for the call. The caller doesn't repeat themselves. A cold transfer sends the caller into a queue with no history.
When should an AI voice agent escalate to a human?
Escalate at once for billing disputes, complaints, distressed callers, legal or fraud topics, and any request for a person. Escalate when sentiment drops sharply, too. Don't wait for the retry cap.
Does an AI voice agent have to tell callers it is AI?
In many places, yes. EU rules apply to automated phone systems from the first interaction. Always answer truthfully when a caller asks.
How do you test voice agent fallbacks before going live?
Write one hard test call for each of your six fallback triggers. Talk to the agent in the browser, with no phone number needed. Try noise, silence, off-topic questions and anger. Check that every call ends with a named next step.
How do you measure whether your fallback design is working?
Track fallback rate by reason each week. Add the share of fallbacks that end in a resolved case, and how often the same caller calls back. Falling repeat gaps mean the knowledge base is improving.
Never leave a caller stuck
Every fallback should name the next step and who owns it. Clarify once, answer what you can, and hand off before the caller asks twice. Rule of thumb: never leave a caller without a named next step.

