A discount that always appears in the same situation, such as an abandoned cart discount, a first sign-up offer or a cancellation offer, stops rewarding a behaviour and starts teaching it. Customers learn the rule and use it, and the shop pays a discount on sales it would have made anyway. The only way to know what that costs is to keep a share of customers without the discount and compare the purchases and margin of the two groups.
Customers have learned the system
Iterable, a company that sells an email and SMS platform, published a 2026 survey of 1,084 consumers in the US and the UK. The figures are self-reported and come from a vendor, but the pattern is clear:
- 37% admit to abandoning a cart in the past year to get a better offer;
- 67% delay purchases waiting for a bigger discount;
- about one in five uses another email address to get the first-purchase discount again;
- 18% unsubscribe from the email list right after receiving the offer.
Byron Sharp reached the same conclusion by another route, with decades of purchase data. Price promotions mostly reach people who already buy the brand, give a short spike and teach buyers to wait for the next one.
Why the flow seems to work
The abandoned-cart flow is judged by the sales it “recovers”: someone got the email with the code, clicked and bought, and the sale counts for the flow. What that number does not say is how many of those people would have come back on their own, with or without the code.
It is the classic case of a system that does exactly what we ask. We asked the flow for conversions, and it delivers conversions. Part of them is margin handed to people who did not need the incentive.
How to set up the test
1. List the flows with a discount
First, a simple inventory: which flows give a discount automatically, how much, and to whom. The most common are welcome, abandoned cart, win-back for inactive customers and retention at subscription cancellation.
2. Keep a group without the discount
In each flow, a share of customers, chosen at random, stops receiving the discount. Between 10% and 20% is usually enough. The group can get a plain reminder without a code, or nothing, depending on what you want to compare.
3. Write the metric before starting
The main metric is margin per customer in the group. A flow with a discount almost always converts more. The question is whether the difference pays for the discount given to everyone who would have bought without it. Alongside it, track the 30-day purchase rate and average order value.
4. Let it run for four to six weeks
Any less, and one abnormal week decides the result. And never during Black Friday or Christmas, when everyone’s behaviour changes. A test prepared now starts in January, with normal traffic.
5. Read the result
There are three possible outcomes. The discount brings extra margin, and it stays. The discount brings sales but not margin, and it moves to a lower value or to an alternative that does not touch price. Or the discount changes almost nothing, and it goes.
Two checks you can run now
Even without a test, two numbers can be pulled from data the shop already has:
- How many first purchases with the welcome code come from emails or addresses that have bought before. It is the direct measure of people using another email to repeat the discount.
- What share of the month’s sales had any discount, and what the average discount was. If it rises every month, customers are learning to wait.
What to test instead of the discount
The holdout group also works for testing alternatives that leave price alone. They include a plain reminder with no code; free shipping above a threshold; a clear returns guarantee; a bundle priced better than the separate products. These often weigh as much as the discount or more, as we explain in bundles, free shipping and email: what matters more than the discount.
There is a limit to bear in mind in Europe. Giving different offers to different people touches consumer protection and price transparency rules. A holdout test is different from personalising prices per person, but it is worth having a lawyer check the design before running it.
The holdout group is the simplest way to measure incrementality, the central idea of incrementality: how to know what your marketing actually causes.
Frequently asked questions
Does the abandoned-cart discount work?
It works in the sense that it brings sales. The question is how many of those sales would have happened without the discount. Only a holdout group answers that.
What share of customers should go without the discount in the test?
Between 10% and 20% is usually enough, chosen at random, for four to six weeks.
What is the right metric for judging a discount?
Margin per customer in the group, compared with the group without the discount. Conversion rate alone misleads, because the discount almost always converts more.
Can I run the test during Black Friday?
It is better to wait. Behaviour in those weeks is abnormal and the result does not apply to the rest of the year. The test should start in January.
Want to know what your automatic discounts cost you?
Book a call and we will run the inventory of your flows and design the first test with you.
