Marketing incrementality is the share of sales that would not have happened without a campaign, a channel or a discount. It differs from what ad platforms report, because they credit whoever clicked, even if that person was going to buy anyway. The way to measure it is always the same: compare a group that received the action with a similar group that did not. Without that comparison, marketing ends up spending part of the budget talking to people who were already going to buy.
What the platforms measure
When someone clicks an ad and buys, the platform counts the sale as its own. The ad may have convinced them, or it may have been someone who already knew the brand, searched for its name and clicked the first result, which happened to be paid.
The second case is very common in campaigns that talk to people already close to buying: retargeting, brand search, customer lists, abandoned-cart emails. These campaigns show excellent attribution numbers and bring little growth, because they harvest demand that other channels created.
There is also influence that leaves no trace. According to Search Engine Land, close to 97% of visits that follow a mention in an AI assistant carry no identified source. That influence shows up as direct traffic and brand searches, and no platform claims it.
The question that decides the budget
Before increasing or cutting a channel, the useful question is simple: if we switched this channel off for a month, how much would sales actually fall?
The answer rarely matches what the platform says. With one of our clients in 2025, the reported cost per acquisition jumped overnight while sales stayed flat. The problem was in the tracking, and that is where it was fixed. When reported cost spikes and sales do not move, it pays to distrust the signal first and the channel second.
The four tests that measure incrementality
In each one, a group receives the action, a similar group does not, and the results are compared. What changes is what gets split and what it costs.
| Test | What gets split | What it is for |
|---|---|---|
| Holdout group | Customers or contacts, at random | Emails, automatic discounts, repeat-purchase flows |
| Geo test | Regions | Ad channels that cannot be split person by person |
| On/off test | Time periods | Small accounts, when there are not enough regions |
| Platform study | Users, split by the platform itself | Large accounts, such as Google’s Conversion Lift |
The holdout group is the simplest and works for almost every email and discount flow, as we explain in automatic discounts: how much they pay for sales that would have happened anyway. Region and time-based tests are the way to measure ads on modest budgets, described in incrementality tests for ads.
For companies with enough budget and history, a marketing mix model combines data from several channels and months and estimates the effect of each. There are open-source tools for this. The model becomes far more reliable when it is calibrated with test results.
The rules for any test
- Write down beforehand what you will measure and what counts as a result. For example, margin per customer over 30 days.
- Choose similar groups. The group without the action has to resemble the group that gets it: the same kind of customer, the same season, regions with comparable sales.
- Give it enough time. Four to six weeks for most tests. Less than that, and the noise of one good or bad week decides the result.
- Do not test in abnormal weeks. Black Friday, Christmas and large campaigns change everyone’s behaviour. That is the time to measure the campaign itself, as we explain in how to measure Black Friday.
- Measure on margin. A discount that brings more sales and less margin can be a bad deal. The rule also applies to anyone paid on results, as we explain in paying for results.
The signals nobody claims
Tests answer specific questions. Between tests, three numbers show whether marketing is building anything, and they are worth tracking every month:
- the trend in searches for the brand name and in direct traffic;
- the share of sales made with a discount, and the average discount;
- the share of sales that comes from new buyers.
Byron Sharp, in How Brands Grow, shows that brands grow mainly by winning new buyers, many of them occasional, and much less by making existing buyers more loyal. An ad account that only “works” on existing customers shows good numbers and leaves the business where it was.
Where to start
In a mid-sized company with an online shop, the order that usually gives the most information for the least effort is this:
- Add a holdout group to email flows with automatic discounts.
- Measure what share of the ad budget reaches new buyers.
- Run an on/off test on brand search.
- Only then, with the results in hand, consider a marketing mix model.
None of these steps needs new tools. They need someone to define the test before it starts and let it run to the end. The media plan these tests fit into is in paid media strategy for scaling brands. The way to measure the real cost of acquiring a customer in how to measure marketing ROI and your real CAC, and alternatives to discounting in bundles, free shipping and email.
Frequently asked questions
What is incrementality in marketing?
The share of sales that would not have happened without a marketing action. It is measured by comparing a group that received the action with a similar group that did not.
Why are Meta and Google numbers not enough?
Because they credit the campaign the person clicked on, even if they were going to buy anyway. Campaigns that talk to people already close to buying look better than they are.
Do I need a large budget to measure incrementality?
No. A holdout group in an email flow costs nothing to set up. Region and on/off tests work on modest budgets. Marketing mix models need more data and more history.
How long does an incrementality test take?
Four to six weeks in most cases, outside abnormal periods such as Black Friday or Christmas.
Want to know how much of your budget talks to people who were already going to buy?
Book a call and we will design the first test with you, with the holdout group and the metric written down before it starts.
