Running SMS, email and voice on one lapsed list and crediting the most clicks sends budget to the wrong channel. This AI voice calls vs SMS vs email for customer reactivation comparison and A/B test plan shows which channel wins customers back.
How the three channels compare at a glance
Each channel does a different job on a lapsed list. This table shows the trade-offs.
| AI voice call | SMS | ||
|---|---|---|---|
| Reach and answer pattern | Only answered calls count. Retries help. | Delivered to the phone, often read fast | Large reach, but easy to ignore |
| Speed of reply | Immediate, live | Minutes | Hours to days |
| Depth of conversation | Highest. Handles objections and reasons | Short, one or two turns | One-way, long form |
| Cost per attempt (relative) | Highest | Low | Lowest |
| Consent and disclosure | Strictest. Consent and AI disclosure | Own opt-in rules | Own opt-in rules |
| What it reports | Answers, transcripts, outcomes | Delivery, clicks, replies | Opens, clicks |
| Best reactivation use | High-value or unclear lapses | Short nudge, link action | Detail, proof, big lists |
Take your baseline from your last two campaigns, not from a published open rate.
Look at the "what it reports" row. Opens and clicks are activity. Only a purchase, a funded account or a loan application proves a customer came back.
Rules differ by region, and you should check each channel before launch. In the US, AI voice calls already need consent under the TCPA, as set out in the FCC's 2024 ruling. In the EU, AI Act Article 50 sets transparency duties for AI systems that talk to people, and automated phone systems must say they are AI. Email and SMS carry their own opt-in rules.
Insight7's voice campaigns, in beta, cover the voice arm. A campaign has a type, and win-back is one of them. It also has consent marking, calling hours, retry rules and a do-not-call list. Contacts go in by CSV or typed entry. A dashboard shows each transcript, with outcomes pulled from it. Calls where only the AI spoke are not counted as reached. Separately, Insight7 call scoring evaluates 100% of customer conversations instead of a 2% manual sample.
Voice wins when lapse reasons are unclear
Use a voice call when you do not know why the customer left. A text or email can only offer a discount. On a call, the customer can say the delivery was late or a rival was cheaper. The AI can answer that objection and make the offer in the same conversation.
The reason is also recorded. A reviewer can read each transcript when the call ends, and yes/no answers and call outcomes are pulled from it. Review those transcripts and rank the reasons customers give for leaving. Your SMS and email copy can then use that list.
A commercial bank we spoke with wanted to compare voice and SMS on one customer set, so it could show management which channel converted. This article's test plan builds on that idea.
Voice has limits:
- Robotic sound. Buyers fear AI that sounds scripted. Their first question is what happens when a customer goes off script. Keep the greeting and replies short, and test the agent before launch.
- Channel preference. Some customers prefer app or WhatsApp messages to calls. Lapsed customers may also not recognise the number. Expect a lower answer rate than on an active list.
- Answered is not reached. A call where only the AI spoke is not a contact. Use that rule in your test, or voice will look better than it is.
SMS and email win on cost and speed
Start with a written channel when the list is big and the offer is simple. A text or email costs far less per attempt than a call. You can reach every lapsed customer and keep voice for the ones worth a conversation.
- SMS suits a short nudge and any action that needs a link, such as an app download or a reorder page.
- Email suits longer offers, price details and proof of what has improved since the customer left.
Voice followed by SMS is a sequence, not a rival. Our advice is to test a text with a link after a call. The customer taps once and does not have to search an app store. Give it its own arm in your test, as the plan below does.
Run this A/B test plan for reactivation
Write the plan down before the first message goes out. Gartner found that only a quarter of 432 AI customer-service use cases showed positive ROI, and 42% could not be measured at all. These six steps keep your campaign out of that last group.
- Name one primary metric. A reactivated customer is a completed purchase, funded account or loan application inside a set window, such as 30 days. Clicks and answered calls are side metrics only.
- Segment, then assign at random. Group lapsed customers by lapse length and value. Inside each group, randomly assign people to four arms: voice only, SMS only, email only, and voice then SMS.
- Hold out a control. Contact nobody in this group. Its return rate shows who would have come back anyway. Real lift is an arm's rate minus the control's rate.
- Keep everything else equal. Use the same offer, send day and calling hours in every arm. Count a voice contact as reached only when the customer spoke.
- Size the test before launch. Put your baseline rate, the smallest lift worth detecting, a confidence level (95% is common) and power (80% is common) into a free sample-size calculator. Every arm and the holdout need that sample, so the total is five groups. A smaller lift needs a much bigger sample. Run the full length and do not stop early because one arm looks ahead.
- Attribute by arm, not by channel report. Give each customer a unique ID or CRM tag at assignment. Referral codes fail on voice, because a caller cannot note a code mid-call. Random assignment with tags scales well, and the same ID lets you credit the voice-then-SMS arm separately from SMS only.
Guardrails
- Set stop rules first, such as pausing an arm if opt-outs or complaints pass a limit you chose in advance.
- Sync one do-not-contact list across all arms, so nobody who opted out is contacted in a copy of the campaign.
- Mark consent and set the same retry rules in each arm. A carrier's temporary refusal should not use up a retry.
Read-out
Compare cost per reactivated customer, then 30-day retention of those customers. A cheap win that churns again is not a win. Add the reasons customers gave in voice transcripts. Your champion needs a number to carry upstairs, and cost per reactivated customer set against your cost to acquire a new one is that number.
How to read results and pick a channel
Pick the channel by who lapsed and how well you understand why. The test numbers then tell you whether that guess was right.
- Voice fits high-value or high-uncertainty lapsed customers.
- SMS fits quick nudges and link actions.
- Email fits large, low-cost lists and offers that need detail.
- Voice then SMS fits most real campaigns.
Work out the winner with one line: cost per reactivated customer equals total arm spend divided by reactivated customers, with the holdout's return rate taken out first. Rerun the test each quarter, because lists and offers go stale.
Related reading: AI Voice Agent ROI Calculator With Worked Examples for Customer Service and Collections and AI Voice Agent Pilot Plan Template for a Two-Week Proof of Concept.
Questions people also ask
Which is better for customer reactivation, SMS or email?
Neither wins on every list. SMS suits a short nudge or a link action. Email suits detail and big, low-cost lists. Let a holdout-controlled test pick the winner on reactivated customers, not opens or clicks.
Do AI voice calls convert better than SMS?
Sometimes, mostly where the reason for lapsing is unclear. Only your own split test can tell you. Count a voice contact as reached only when the customer spoke, or voice will look better than it is.
How do I run an A/B test for a reactivation campaign?
Name one primary metric, such as a purchase within 30 days. Randomly assign lapsed customers to voice, SMS, email and voice then SMS. Add a no-contact holdout. Keep offer and timing equal, and size the test before launch.
What is a good response rate for a win-back campaign?
No single figure applies, because lists, offers and lapse lengths differ. Take your baseline from your last two campaigns. Judge each arm by reactivation rate above the holdout, not replies.
How many customers do I need in each test group?
It depends on your baseline rate, the smallest lift worth detecting, confidence and power. A smaller lift needs a bigger sample. Size the holdout too, because every lift is measured against it.
Do I need consent to send AI voice calls to lapsed customers?
Check the rules for every region you call into, since consent and AI disclosure duties vary. Keep a consent record per contact, and mark it in your campaign tool. SMS and email have their own opt-in rules.
Should I follow an AI voice call with an SMS?
Test it as its own arm, and treat it as a hypothesis until your numbers back it. Attribute that arm separately from SMS-only. A text with a direct link also lets you see who tapped after the call.
Let the holdout pick the winner
The busiest channel often differs from the one that wins customers back. Only a split test with a no-contact holdout shows which one does. This week, split your lapsed list into four arms plus a holdout, and write down one metric before launch.
