A common mistake is giving the whole reactivation list a discount. Another is comparing this month's offer calls with last month's plain ones. Both pay for returns that would have happened anyway. Test incentive vs no-incentive offers with one random split.
Write one hypothesis and one success metric
Start with one sentence you can prove wrong. For example: "Offering 10% off on the reactivation call raises net value per contact compared with the same call with no offer."
Pick one primary metric that counts the full path. Count every contact on the list, then check whether each was reached, whether they reactivated within a fixed window (say 14 days), and what they were worth after the incentive cost. Call this net value per contact. "Answered" and "interested" alone mislead. A free offer makes people say "sounds good" and then never come back.
Set the decision rule before launch. Here is an example with assumed numbers, not a benchmark. A reactivated customer is worth $60 in margin, and the incentive costs $15 per redemption. Suppose the plain call reactivates 4% of contacts, which is $2.40 per contact. The offer arm must reach about 5.3% to match that ($2.40 divided by $45). Anything under a 1.3 point lift loses money. Buyers often need this kind of cost-against-value case to carry upstairs.
Add three guardrails: opt-outs, complaints and call length. A rising opt-out rate in one arm cancels a small win.
Split one list at random into groups
Pull one lapsed-customer list and split it at random. Do not split by region, signup date or segment. Those splits mix the offer's effect with differences between the people. Random assignment spreads those differences evenly across the arms.
Clean the list first. Remove anyone on your do-not-contact list before you split.
Build three groups:
- Offer arm: gets the call with the incentive line.
- Control arm: gets the same call with no offer.
- Holdout (optional): gets no call at all. It shows whether the calls lift reactivation, or whether customers would have returned anyway.
Stratify by lapse length. Sort the list into bands, such as 3 to 6 months lapsed and 6 to 12 months lapsed. Then randomise inside each band. A customer lapsed for three months is far likelier to return than one lapsed for a year.
Set the sample size before launch. Evan Miller's sample size calculator asks for your baseline conversion rate and the smallest effect you want to detect. It returns a sample size per variation. Use your break-even lift from step one as that smallest effect.
Take the assumed numbers again: a 4% baseline and a 1.3 point lift to 5.3%. The standard two-proportion formula is n = (1.96 + 0.84)² × [p1(1−p1) + p2(1−p2)] ÷ (p2 − p1)², for a 5% significance level and 80% power. Here that gives about 4,100 contacts per arm. This is an illustrative calculation, and calculators use slightly different formulas, so confirm your own figure in one. If your list is smaller, test a bigger offer or accept that only large lifts will show up.
Change only the offer in the call script
Build two campaigns that match in every setting except one line. Use the same voice, script, calling hours and retry rules, and launch both in the same window. Only the offer sentence differs.
Test both versions before launch. Listen to a test call and check that the offer line sounds natural and that the control never hints at a deal.
Count "reached" the same way in both arms. Calls where only the AI spoke should not count as reached. A carrier's temporary refusal should not use up a retry. A bad carrier hour then does not shrink one arm's attempts.
An incentive offer is marketing, so consent rules apply. Check each region you call:
- EU: Article 50 of the AI Act kept its 2 August 2026 start date. Automated phone systems must say they are AI from the first interaction.
- US: the FCC's February 2024 ruling treats AI-generated voices like other artificial or prerecorded voice calls under the TCPA. They are legal, but restricted.
- Colorado: its Chatbot Safety Act was signed on 29 May 2026 and takes effect on 1 January 2027. It requires public-facing conversational AI to disclose that it is AI.
Use the same disclosure line in both arms, so it cannot explain a gap between them.
Insight7's beta voice campaigns list win-back as a campaign type, with consent marking, calling hours, retry rules and a test call to your own phone. Contacts load by CSV upload or typed entry. You make the random split in your list before upload. Duplicating a campaign copies its contacts and leaves out anyone added to the do-not-contact list since the original, which gives the second arm identical settings. You can also talk to a campaign's agent in the browser with no phone number. In the beta, calls where only the AI spoke are not counted as reached, and a carrier's temporary refusal does not use up a retry. A dashboard shows calls in progress, and call outcomes are pulled from the transcript. The beta spec also lists a campaign scorecard when a campaign ends. Separately, Insight7 can score every call, not a sample, against a weighted scorecard your team owns, so both arms are judged by the same criteria.
Run it long enough, then stop peeking
Write the end date and the sample size on the plan before launch. If you stop on a good-looking day, you break the test. Evan Miller explains that deciding to run until you see a significant difference makes the reported significance levels meaningless. Stop only on the planned end date.
Run both arms at the same time, at the same hours of day. Answer rates swing with the time of day and the day of the week. If one arm calls at lunch and the other at night, you are testing the clock. Run at least one full weekly cycle.
Use your live view to look for breakage only. Look for silent calls, a script that repeats, or an offer line spoken in the wrong arm. Fix the fault and note the date.
Read results by net value, not response rate
Compare net value per contact between the arms, then check whether the gap is bigger than chance. Run a two-proportion test on reactivation rates, or put a confidence interval around the difference. If the interval includes zero, the result is "not significant". In that case keep the cheaper no-incentive call, or rerun with a bigger list.
Next, price each return. Cost per reactivated customer is the incentive spend plus call cost, divided by reactivations. Use the earlier example numbers. The offer arm reactivates 5.5% of 4,000 contacts, which is 220 customers. Each is worth $60 minus a $15 incentive, so net value is $9,900, or about $2.48 per contact. The plain arm earns $2.40. The rate rose, but the profit barely moved.
Check whether customers still buy 30 to 60 days later. A discount can buy a one-off return that never repeats.
Then read transcripts for the why:
- Which objections did the offer solve?
- Did customers say they would have come back anyway?
Cut results by lapse length and past spend. Treat those cuts as new hypotheses, because each slice is a smaller sample.
Check the guardrails last. Compare opt-outs, complaints and requests for a human between arms.
Our advice: buyers in regulated sectors prefer a small, low-risk pilot, paid for only after a measured lift. A two-arm test on one list fits that well.
Decision rule: ship the arm with the higher net value per contact, then test the next variable: offer size, timing or wording.
Related reading: Average Outbound Calls Per Hour in a Call Center Compared With AI Voice Agent Capacity, How to A/B Test AI Voice Agent Scripts, Voices and Offers on the Same Customer Segment and AI Voice Agent Pilot Plan Template for a Two-Week Proof of Concept.
Questions people also ask
How big a sample do I need for an A/B test?
It depends on your baseline rate and the smallest lift you care about. As a general rule, halving that lift roughly quadruples the sample you need. Enter your break-even lift into a sample size calculator before launch. If your list is too small, test a bigger offer.
How long should an A/B test run?
Divide the total sample you need by the contacts you can call per day. Round up to whole weeks, with a minimum of one. Add time for retries and the reactivation window, so late returns still count.
What is a good reactivation rate for win-back calls?
No single figure fits every business, because rates depend on lapse length, product and list quality. If you ran a plain-call win-back campaign before, use its rate as a starting guess for the sample size. Your control arm then gives the true baseline.
Should I offer a discount to win back lapsed customers?
Offer one only if it beats the plain call on net value per contact. If the lift is too small to cover the incentive cost, keep the cheaper call and test a different lever, such as timing.
What is a control group in an A/B test?
It is the group that gets the standard experience. Here, the control gets the same call with no offer. An optional holdout gets no call, which isolates the effect of calling.
How many times should I redial a customer before giving up?
Set a fixed retry cap and use the same one in both arms. Otherwise the arms differ in more than the offer, and one arm gets more chances to reach people.
Do AI calls need to say they are AI?
Plan to disclose in every arm, whatever your region's rules say today. A different disclosure line in one arm could change who stays on the call. That would blur the offer's effect.
Close
An incentive earns its cost only if it beats the plain call on net value per contact. Insight7's beta supports win-back campaigns, and you can duplicate one for the second arm. This week, split one lapsed list at random into two groups and write the one offer line that differs.

