A caller wants their balance now. Your agent must not read it to a stranger, and a voice that sounds right proves nothing. This is why AI voice agents verify caller identity in steps. Each step unlocks only some account data.
What identity verification means for a voice agent
Identity verification is checking that the caller is the real account holder before the agent shares anything private. It has three parts. Identification is who the caller claims to be. Authentication is proof of that claim. Authorisation is what that person may see or change.
NIST's Digital Identity Guidelines use the same split. They cover identity proofing, authentication and federation. They also say a service uses assurance levels to make authorisation decisions. So a phone flow can unlock a little data after a weak check and more after a strong one.
The agent cannot lean on voice or caller ID alone. The FBI warns that criminals use AI voice cloning to pose as trusted people. Caller ID is easy to spoof as well. A voice that sounds right is a hint, never proof.
Your QA manager should treat voice as one signal among several, and never as the key.
The verification flow, step by step
A good flow runs six steps, and each one adds proof before the agent says more.
- Match the calling number. The agent compares it with the number on the account.
- Disclose and ask consent. The agent says it is AI and asks to continue.
- Ask for an identifier. This could be an account number, a date of birth or a policy ID.
- Add a second factor. The agent asks knowledge questions, or sends a one-time code to the registered phone or app.
- Check the answers against the record. The lookup goes to the CRM or core system.
- Set the access level. The result decides what the agent may share next.
Every factor is weak alone. Caller ID can be spoofed. Knowledge answers leak in data breaches or can be guessed. A text code can be intercepted, or a fraudster can talk the real owner into reading it out. Layering factors helps because an attacker must beat several different weaknesses in one call.
The hard part is speech. Speech-to-text slips on digits, names and dates, especially on a noisy line. So the agent should read each captured value back for confirmation and allow a few retries.
The match itself belongs in code. The system compares the caller's answer with the stored record and returns pass or fail. The language model never judges that, because a model can be talked round and a string comparison cannot.
A lender told us its agents confirm the registered number, ask security questions, then open the CRM and share the balance. A bank asked how to verify a caller through the CRM before any sensitive detail is released.
Insight7 has inbound voice lines in beta. People ring a number and the AI agent answers, and it can look the caller up.
What the agent can share at each level
The riskier the action, the stronger the proof the agent should ask for. A leaked due date does little harm, but a changed phone number can hand the account to a thief. Treat the table as a design pattern, not a legal standard. Your regulator and risk team set the real lines.
| Level | Proof so far | What the agent may share or do |
|---|---|---|
| 0, unverified | None | Product details, branch hours, FAQ answers |
| 1, identified | Number matched plus one identifier | Status only, such as "payment received" or "case open" |
| 2, authenticated | A second factor passed | Balance, last transactions, due date, masked details |
| 3, step-up | Fresh code, or a human check | Address, phone or bank detail changes, payouts, full account numbers, disputes |
Two rules hold at every level. First, mask card and account numbers, for example "ending in 4821". Never read a full number aloud. Second, never give a security answer back to the caller, even to confirm it. A fraudster who is guessing learns from every echo.
This follows data minimisation. GDPR Article 5(1)(c) says personal data should be limited to what is necessary. So the agent shares the least that answers the question. NIST's assurance levels match proof strength to risk in the same way.
Enforce the levels in code. Each data tool the agent can call should check the session's level before it returns anything. Then a clever caller cannot talk the model into skipping a step, because the lookup refuses to answer.
How failed verification gets handled
Design the failure path before the happy path, because a fraudster will test it first.
- Cap the retries. A wrong answer gets a retry, then a limit of two or three attempts.
- Drop to Level 0 after the cap. The agent offers a human or a callback to the number on file.
- Lock and log. Lock the account for a cooling period. Log every attempt with the time, the number and the check that failed.
The agent must not hint at which answer was wrong. "That did not match" is enough. "The date of birth was off" tells a guesser where to aim.
Plan for the edge cases too:
- New phone number. The number match fails, so ask for a stronger factor or call back on the registered contact.
- Noisy line or mispronounced name. Ask for a spelled name or keypad digits before counting a failed attempt.
- Caller is not the account holder. A spouse or an assistant gets Level 0 only. Offer to have the holder call, or to add them as an authorised user.
- Suspected cloning or pressure. The answers are perfect, but the caller rushes and pushes to change a phone number. Send the call to a human.
Handover is where most designs fall short. The agent should pass the human a case summary and a list of what is already verified, so the caller does not start again. Buyers raise this on their own. They ask whether the handover is polite, what happens with a sensitive issue or a disputed amount, and whether anyone chases a handover nobody picked up.
Callers expect this exit. Gartner's 2026 survey of 3,566 customers found that 87% say a company using GenAI for service must offer a way to reach a human.
Regulated buyers, banks first, name an AI that "goes rogue" as their main blocker, and they want proof from real deployments. Write the failure rules down and test them before launch.
Then review failed verification transcripts every week. Scoring every call, as AI Call Scoring does, lets you check each call against a verification scorecard you build.
Limits and what to ask a vendor
A CRM with limited API access may return only a name and a status, which keeps the agent at Level 1.
Rules differ by region:
- EU: Article 50 of the AI Act says systems that talk to people must tell them it is AI, unless that is obvious. It has applied since 2 August 2026.
- US: The FCC ruled that AI-generated voices are "artificial" under the TCPA, so these calls need consent.
- Nigeria: Nigeria's central bank told banks and payment firms to keep payment transaction data on local servers by 1 January 2027.
Buyers ask about data location early. Ask every vendor:
- Where do recordings live?
- Which models touch them, and who owns the data?
- What does the agent do when the CRM is down?
- How is the audit trail kept?
Related reading: Knowledge Base Readiness Checklist for AI Customer Service Agents, Caller Authentication Methods for AI Phone Agents Compared by Security and Customer Effort and 15 Inbound Call Types AI Voice Agents Can Resolve Without a Human.
Questions people also ask
How do AI voice agents verify who is calling?
They check in layers. The agent matches the calling number to the account and asks for an identifier. It then adds a knowledge question or a one-time code. The answers are compared with the CRM record, which sets the access level.
Can an AI voice agent verify identity by voice recognition alone?
No. A cloned voice can pass a voice-only check. A clone also fails on a code sent to the owner's phone, because the fraudster never receives it. Pair voice with a code or a knowledge check.
What information can an AI voice agent share before verification?
Only public information: product details, branch hours and FAQ answers. Account data waits until the caller is identified. Even then, unlock status only first, such as "payment received".
What happens if a caller fails identity verification on an AI voice call?
The caller loses access to account data and is offered a human or a callback. The lock should end only after the cooling period, or after a human re-verifies the holder through a registered contact.
Is a one-time passcode safe for verifying callers over the phone?
Not alone. A text code can be intercepted, or a fraudster can talk the real owner into reading it out. Send it to a registered device and add another factor.
Can voice cloning fool an AI voice agent?
Yes, if voice is the only check. A clone cannot read a code sent to the owner's phone or know account details it never saw. Layered factors and human escalation cover the gap.
Does an AI voice agent have to tell callers it is AI?
In many regions, yes. The EU AI Act requires systems that talk to people to say they are AI, unless that is obvious. In the US, AI-generated voices need consent under the TCPA. Check your own regions.
Access rises with proof
Access should rise only as the proof rises, and every failed check needs a defined exit. Take one question into your next meeting: if verification fails twice, who picks up the caller, and with what context?
