TL;DR
- A human-in-the-loop improvement workflow lets the agent surface problems while people approve every script change.
- Roles, risk-tiered approval rules and a change log keep the loop safe.
- Test each change before release, and log who approved what.
Map the loop and name the roles
Draw the loop on one page, then put a named person on every step. The agent surfaces problems, and people decide what changes.
- Call: the AI agent handles the conversation.
- Score: every call is scored against the team's scorecard.
- Flag: a low score, critical error or failed handover raises a flag.
- Propose: the QA analyst writes a change request.
- Review: the script owner and a risk reviewer check it.
- Approve: the approver for that risk tier signs off.
- Test: the change runs in a test call.
- Release: a small share of calls gets it first.
- Monitor: scores are watched, then the loop returns to step 1.
| Role | May do | May not do |
|---|---|---|
| QA analyst | Flag issues, attach evidence, propose changes | Edit the live script |
| Script owner | Write and edit wording, run tests | Approve their own change |
| Compliance or risk approver | Approve or reject high-risk changes | Author the change they approve |
| Operations lead | Schedule releases, order a rollback | Skip a required approval |
| Auditor | Read the log, sample decisions each quarter | Change tickets or scripts |
No one approves their own work. That one rule makes the table a control.
People stay in the loop because customers expect it. In a Gartner survey, 87% of 3,566 customers said a company using GenAI for service must offer a way to reach a human. Regulated buyers, banks first, name an AI that "goes rogue" as their main worry. They also ask who chases a handover nobody picked up.
Collect evidence from every call
Base every script change on all calls. A 2% manual sample can miss a wording problem that shows up in only a few calls, and it carries rater bias. Scoring every call lets a change request cite a pattern with call links, using a weighted scorecard the team owns.
Five signals should open a change request:
- A low score on a script-driven criterion.
- A critical-error fail.
- A keyword alert, such as a customer asking "Am I talking to a robot?"
- A failed handover.
- Customer drop-off at the same point in the script.
Alerts can watch keywords and low scores, send a weekly digest of the weakest calls, and notify by email or Slack.
Let one critical error fail the whole call. QA leaders warn that a checklist score can hide what matters, because a call can be perfect on procedure and still leave the customer angry.
Calibrate before you trust the numbers. Have human reviewers score a sample of the same calls and compare. Built-in calibration lets managers review, override and fine-tune AI ratings. Fix scorecard criteria first, then script wording.
An insurer told us its agents kept disputing QA outcomes. It wanted a structured, supervisor-led way to contest a score. Contested scores are useful evidence too, because they show where criteria are unclear.
Write each script change as a ticket
Put every script change in a ticket, and make the ticket the first record in your audit trail. A ticket needs these fields:
- Ticket ID and requester: who asked, and when.
- Evidence: links to the calls that triggered it.
- Old wording and new wording: the exact text, side by side.
- Reason: which failure the change should fix.
- Baseline and expected shift: for example, handover failures today and the drop you expect.
- Risk tier: low, medium or high. This decides who approves.
One ticket carries one change. If a script owner edits the greeting and the consent line together, nobody can tell which edit moved the score. Small tickets also get approved faster, and a bad one is easy to roll back.
The baseline makes the ticket useful later. Gartner analysed 432 AI customer-service use cases. Only a quarter showed positive ROI, and 42% could not be measured. Without a baseline, your change joins that 42%.
Aim each change at a gap your QA data shows, not at a hunch. Training buyers want proof that behaviour changed, not that a session happened. Buyers also need numbers they can carry upstairs.
Set approval rules by risk tier
Sort every ticket into a risk tier, and let the tier set who signs and what testing is needed.
| Tier | Change types | Approvers | Test before release | Rollback rule |
|---|---|---|---|---|
| Low | Tone, greeting, pacing | Script owner's peer, 1 approver | Browser test | Operations lead reverts if scores drop for two review periods |
| Medium | Offers, pricing, qualification questions | Script owner's manager plus operations lead, 2 approvers | Browser test, then test call | Revert if the target metric worsens against baseline |
| High | Disclosures, consent, AI-identity statements, escalation and handover rules, sensitive data, collections language | Compliance or risk approver plus operations lead, 2 approvers | Test call, then a small-sample release | Revert at the first compliance miss or failed handover |
Three rules sit on top of the table:
- Separation of duties. The author cannot approve. A high-risk change always needs compliance sign-off.
- Emergency pause. Any approver or the operations lead can pause a script at once. A named reviewer must examine the pause within a set time, such as one working day.
- Tier disputes. When in doubt, pick the higher tier.
Disclosure and consent wording belongs in the high tier because the law is moving. Examples by region:
- EU: Article 50 of the AI Act has applied since 2 August 2026, and any AI that talks to people, including phone systems, must say it is AI from the first interaction.
- US: AI-voice calls need consent under the TCPA, as the FCC confirmed in February 2024.
- Colorado: from 1 January 2027, public-facing conversational AI must disclose that it is AI.
- Nigeria: the central bank's March 2026 guidelines accept AI in compliance but demand transparency, governance and yearly independent model validation.
A compliance lead at a bank will ask who approved an edit to the AI line, so name that person on the ticket. Handover rules are high risk too, because buyers ask how sensitive issues and disputed amounts are handled. A two-signature rule is the answer you can show buyers who fear a rogue agent.
Test, release and log every decision
Test every approved change before a customer hears it. Run low-risk edits in a browser test, and run medium and high-risk edits as a test call to a reviewer's own phone. Then release to a small share of calls, such as one in ten, and set a watch period with a rollback trigger written on the ticket.
Log each step as it happens. Each ticket should hold:
- Ticket ID and evidence links
- Script version before and after
- Requester, reviewer and approver
- Timestamps for each step
- Test result and release date
- Outcome review: did the metric move as expected?
Make the log read-only. Nobody edits an entry after it is written, and corrections are added as new entries. Keep it as long as your regulator or contracts require. The auditor samples tickets each quarter and checks that every high-risk change has a compliance signature. Keep this change-ticket log in your own ticketing system.
Insight7 supports the scoring side of this loop. It scores every call against a weighted scorecard your team owns. Alerts can watch for low scores and notify by email or Slack. Compliance teams can use calibration workflows to review and override AI scores. The audit trail Insight7 keeps covers call evaluations only, not script-change approvals. Separately, in beta, a Voice campaign's agent can be tested in the browser with no phone number needed. A reviewer can also read each call's transcript when it ends.
Related reading: How to Run Quality Assurance on AI Voice Agents Using the Same Scorecard as Human Agents, AI Voice Agent QA Scorecard Template With Criteria for Accuracy, Compliance and Customer Experience and How to Improve AI Voice Agent Performance From Real Customer Conversations.
Questions people also ask
What is a human-in-the-loop workflow for AI agents?
It is a repeatable process in which the AI agent handles calls and surfaces problems. Named people review, approve and log every change to its behaviour. In this workflow, the agent never rewrites its own script.
Who should approve changes to an AI agent's script?
Approval depends on the risk tier. A peer or manager can sign off low and medium changes. A compliance or risk approver signs off high-risk ones. The author never approves.
How often should you review and update an AI agent's script?
Review scores weekly. A weekly digest of the weakest calls gives you a natural review point. Batch low-risk edits, and run a full script review each quarter.
What should an audit trail for AI agent changes include?
It should let a stranger rebuild any decision a year later. That means who asked, who approved, which exact text changed, and what the metrics did afterwards. Store entries read-only.
Which AI agent script changes need compliance approval?
Any wording a regulator or customer could hold you to. That covers AI-identity statements, consent, disclosures, escalation and handover rules, sensitive data and collections language. Buyers raise handover design on their own, so treat it as high risk.
How do you test an AI agent script change before it goes live?
Talk to the agent in a browser first, since some tools allow this without a phone number. Then replay the triggering scenario in a test call. Finally, release to a small share of calls.
Do customers have to be able to reach a human when an AI agent handles the call?
The disclosure laws above centre on saying it is AI, rather than offering a human. Customers expect a human route anyway: 87% of surveyed customers said companies must offer one.
Let only signed changes reach customers
An AI agent should improve only through changes that a named person approved, tested and logged, with a baseline to prove the result. So take one question into your next meeting: who can approve a disclosure change today, and where is that decision recorded?

