On Monday morning, a contact centre lead opens the campaign dashboard. The AI caller reached 600 people, but nobody can say why 90 hung up in the first ten seconds, or why a dozen who asked for a human got stuck in a loop. You improve AI voice agent performance by reading real conversations in a fixed loop: group the issues, find patterns, draft fixes, approve, test, measure.

Set one baseline before you change anything

Write down today's numbers before you touch a script. Without them, you cannot tell a real gain from noise.

Track five outcomes:

  • Answered-by-human rate
  • Completed-goal rate
  • Hand-off rate
  • Drop-off in the first ten seconds
  • Complaint rate

Fix the definitions first. A call counts as reached only when a person spoke. In Insight7's beta voice campaigns, calls where only the AI spoke are not counted as reached. Voicemail greetings then stop inflating your reach.

Retries can also distort the baseline. In the same beta, a carrier's temporary refusal does not use up one of the contact's retry attempts. Count those refusals as carrier faults, not as customers who ignored you.

Record the campaign type, calling hours and retry rules next to the numbers. Insight7's beta campaigns carry a type, consent marking, calling hours, retry rules and a do-not-call list.

Step 1: Group failed calls by issue

Sort each failed call into one bucket before you debate any fix. Six buckets cover most voice agent trouble:

  • Wrong or missing answer
  • Robotic or scripted tone
  • Bad hand-off
  • Wrong timing or consent
  • Silence or carrier fault
  • Compliance miss

Tag twice. A call-level tag says what went wrong overall. A moment-level tag marks the exact second it happened, such as the opening line or the point where the customer asked for a person. Moment tags are what let you find a pattern in Step 2.

Buyers fear an agent that sounds scripted, and they keep asking what happens when the customer goes off script. So give tone its own bucket and tag the moment the caller drifted from the script. Buyers also ask whether a hand-off is polite, how a disputed amount is treated, and whether anyone chases a hand-off nobody picked up. Each of those questions is a tag.

Let one critical error fail the whole call. A checklist score can hide what matters, and a call can pass on procedure while the customer ends up angry. A missed consent step or a missing disclosure should zero the call, whatever the rest scored.

Do not sample. Scoring all conversations replaces the 2% manual sampling and its rater bias. Start from a starter rubric on the Criteria screen, then edit it to match your six buckets.

Step 2: Find the patterns behind the groups

Count each group across every cut your data allows. Split your Step 1 tags by campaign, script version, time of day, customer segment, language and voice. A pattern shows up when one cut holds far more failures than the others. For example, most hang-ups might sit in evening calls, or in one voice.

Then find the moment. Your moment tags show where drop-off clusters: the opening line, the identity question, the ask to do something in an app, or the hand-off. A cluster at one moment points to one fix.

Next, separate script faults from data and carrier faults. A script problem repeats across hours and routes. A carrier problem follows the route, so a pile of silent calls belongs in the carrier bucket.

Read five to ten transcripts for each pattern. The numbers tell you where the problem is. The transcripts tell you why.

Last, check your scorer against a human. An automated score can disagree with a human auditor's judgement of the same call. Have an auditor score a small set of the same calls. Trust a trend only when the two agree.

Insight7 scores every call against a weighted scorecard your team owns, and the scores roll up to the person, the project and the organisation. For voice campaigns, a dashboard in beta shows calls in progress and lets a reviewer read each transcript when the call ends.

Step 3: Draft one fix for each pattern

Write one fix per pattern and no more. If you change the opening line and the retry rules together, you cannot tell which one helped.

Match the fix to the pattern:

  • Hang-ups in the first seconds: rewrite the opening line. Say who is calling and why in one short sentence.
  • Wrong or missing answers: add the product facts the agent lacked. Take them from transcripts where the caller asked and the agent stalled.
  • Drop-off at an ask: cut the ask to one action.
  • Bad hand-off: write a rule for when to pass the call, who receives it, and who follows up if nobody picks up.
  • Robotic tone or the wrong accent: change the voice or language. Buyers ask for local accents, Pidgin and native languages. Voice campaigns can run in Yoruba and Igbo in beta, and the manager picks the language before the voice. Nigerian Pidgin is a newer beta campaign language.
  • Wrong timing: change calling hours or retry rules.

Now write each fix as a claim you can test: "A shorter opening cuts hang-ups in the first ten seconds." Name the metric from your baseline and the size of gain you expect.

Keep personal details out of every script. Use placeholders for names, amounts and account numbers, and never paste lines from a real customer call into a prompt.

Step 4: Approve changes with a named owner

Give every change one accountable approver before it reaches a live call. For most scripts that is your QA manager. Add compliance as a second sign-off when the script is regulated. Regulated buyers, banks first, say their main blocker is an agent that "goes rogue" or makes a public mistake. A named owner is your answer to that fear.

Approve each change against a short checklist:

  • Consent is marked for every contact.
  • The opening line says the caller is an AI.
  • The hand-off path to a person works.
  • Calling hours fit the contact's region.
  • The do-not-call list is applied.

The disclosure item is law in a growing number of places. In the EU, Article 50 of the AI Act says providers must inform users when they are talking directly to an AI system. It has applied since 2 August 2026. In the US, the FCC's 2024 ruling means calls using AI voices need the called party's prior express consent.

Keep a change log with the version, the reason, the date and the approver.

Step 5: Test on a small sample

Test each fix in private before any customer hears it. In Insight7's beta, a campaign's agent can be tested by talking to it in the browser, with no phone number needed. Say the opening line aloud and ask the awkward questions your Step 2 transcripts showed. Then place a test call to your own phone, so you hear audio quality and how the voice sounds on a handset.

Next, run a small pilot with a control group. In the beta, any campaign can be duplicated with its contacts, and people added to the do-not-contact list since the original are left out of the copy. Change one thing in the copy and leave the original on the old script. Split the contact list at random, and use the same calling hours for both arms.

Keep the pilot small. Regulated buyers, banks especially, will not pay to find out whether a change works. They want a short, low-risk pilot on a small sample, with payment only after results show a measured lift. Hundreds of numbers per arm is a sensible start for a lender's collections line. Raise the number if your goal is rare, such as a funded account.

Write down the pass mark before you launch.

Step 6: Measure the lift, then repeat

Compare each pilot arm with your baseline, using the same definitions. Give each arm enough calls, and keep the hours and customer segment the same. If the gap is small, run the pilot longer.

Watch for side effects. A shorter opening may cut hang-ups but raise hand-offs or complaints, so read those two numbers next to your goal metric.

Then put money on the result. Champions need a financial case they can carry upstairs, so set your cost per call against what it costs you to acquire or serve a customer. Buyers want proof that behaviour changed.

Measuring is the real work. Gartner's analysis of 432 AI customer-service use cases found a quarter with positive ROI and a quarter with negative ROI. Another 42% were unclear, because leaders could not say what value they got. Customer-service AI spending rose 38% while overall service budgets rose 2%, so leaders want proof.

Ship the winner and retire the loser. Log the result beside the change from Step 4. Then start the next review, weekly for a live campaign or at the end of each campaign.

Related reading: Human-in-the-Loop Improvement Workflow for AI Agents With Approval Steps for Script Changes, How to Run Quality Assurance on AI Voice Agents Using the Same Scorecard as Human Agents and AI Voice Agent QA Scorecard Template With Criteria for Accuracy, Compliance and Customer Experience.

Questions people also ask

How do you improve AI voice agent performance after launch?

Run a fixed loop: group failures, find patterns, draft one fix per pattern, approve, test, then measure. Give the loop one owner, usually your QA manager, and a set cadence, such as a weekly review for a live campaign. Change only one thing per cycle, so every gain or loss has one clear cause.

What metrics should you track for an AI voice agent?

Track answered-by-human rate, completed-goal rate, hand-off rate, early drop-off and complaint rate. Add cost per completed goal, so a finance reader can follow the result. Split every metric by route and calling hour too. A carrier fault can look like a script fault until you separate the routes.

How do you test an AI voice agent before going live?

Use awkward cases on purpose: interrupt it, stay silent, give a wrong number, and ask for a person. Run these in a browser or sandbox first, then on a real handset. After that, run a small pilot with a control group. Pass each case or fix it before any customer hears the change.

How many calls do you need to review to find real patterns?

Read five to ten transcripts per pattern, but count across all calls. Counting needs volume, because small gaps in rates are noisy. As a rough guide, a gap of five points on a 10% rate needs several hundred calls per arm to trust. Smaller gaps need more. Keep hours and segments the same in both arms.

How often should you update an AI voice agent's script?

Review weekly for a live campaign, and ship a change only when a pilot has passed its pre-set mark. Never edit mid-test. Retry and timing rules are the easiest changes to try. One consumer lender found that mornings and evenings answered better than midday, so it moved its start time to 8pm.

How do you stop an AI voice agent from sounding robotic?

Write the opening line the way people speak: short sentences, one idea, no jargon. Then tag the moment where callers drift off script, because that is where a scripted voice shows. Buyers often ask for an agent that sounds like their own reps rather than a generic voice. Switch voice or language if the accent feels foreign.

When should an AI voice agent hand a call over to a human?

Hand over when the caller asks, when the agent fails to answer the same question twice, and when the issue is sensitive or disputed. Buyers ask how to do this without seeming rude. Write a polite line for the switch. Decide who receives the call and who follows up if nobody picks up.

Do you have to tell customers they are talking to an AI?

In many places, yes, and it is wise everywhere. The EU's Article 50 requires that people are told they are interacting with an AI system, unless that is obvious. The FCC ruling is about consent for AI-generated voices. Say it in the first sentence, and ask your counsel about each market you call.

How do you know if an AI voice agent change actually worked?

Compare the changed arm with a control arm that kept the old script, using the same definitions of reached and completed. Write the pass mark before launch. Then check side effects, such as more hand-offs or complaints. Before you roll out, repeat the pilot once on a fresh contact list. A gain that appears twice is far more likely to be real.

Should a human approve every change to an AI voice agent?

Approve every change a customer can hear. Wording, hand-off rules, disclosure and consent steps all need a named approver. Some changes can skip sign-off if you pre-agree narrow limits, such as a retry gap within a set range or a calling hour inside approved windows. That keeps the owner from becoming a bottleneck. Log every change either way.

Take one tested change into next week

Your agent improves as fast as your team turns real calls into one tested change. Insight7's built-in calibration lets managers review, override and fine-tune AI ratings, so your scorer stays honest while the scripts change.

Which single pattern in last week's calls would we fix first, and who signs off on the change?

Book a demo of Insight7