What incrementality is and how to measure it (before peak season arrives)

If you’re going to present an 8x ROAS at November’s leadership meeting, it’s worth knowing first how much of that 8x the campaign earned and how much would have happened anyway. That difference is called incrementality, and it is the only metric that answers the question that really matters: which sales would not exist without this investment?

What incrementality is

Incrementality measures the causal effect of a marketing action: the sales, conversions or revenue that happen because of the campaign and would not have happened without it. It doesn’t count sales the campaign touched; it counts sales the campaign caused.

The distinction sounds subtle and it isn’t. A customer who buys every month receives a 15% coupon and buys. The attribution model credits the sale to the coupon. Incrementality asks whether they would have bought without it. If the answer is yes, the coupon didn’t generate a sale: it generated a discount.

Why ROAS misleads you

ROAS, CPA and every attribution metric (last touch, linear, data-driven) share out sales that have already happened among the channels that touched them. None of them can tell you how many of those sales were going to happen regardless. And in three specific situations the gap explodes:

• Warm audiences. Retargeting abandoned carts and repeat customers shows sky-high ROAS because it reaches people who were already going to buy. It’s the classic case of a 15x ROAS with incrementality close to zero.
• Peak season. Around Black Friday and El Buen Fin demand rises on its own. Any campaign running those days looks like it works, because the tide rises for every channel at once.
• Brand search. Bidding on your own name captures clicks that would have arrived organically anyway. It is usually the highest ROAS in the account and one of the least incremental.

The practical result: budget flows to the channels that attribute best, not to the ones that sell most. And the figure you present to leadership is inflated without anyone having lied.

How to measure it: the control group

The only way to know what would have happened without the campaign is to have a group of comparable people who don’t receive it. It’s the same principle as a clinical trial, and it’s called an incrementality test or holdout test.

1. Define the population. The segment you are going to target: for example, customers with more than two purchases in the last twelve months (an RFM analysis gives you these cuts in minutes).
2. Split at random. A treatment group receives the campaign; a control group (usually between 5% and 20%) receives nothing. Assignment has to be random; if you pick by hand, you are no longer measuring the campaign, you are measuring your judgement.
3. Launch and wait. The measurement period covers the campaign and the product’s usual conversion window.
4. Compare. Sales per person in treatment minus sales per person in control. That difference, multiplied by the size of the targeted group, is incremental revenue.
5. Divide by cost. Incremental revenue divided by spend: that is incremental ROAS, the only one you can defend.

An example with round numbers. You target 100,000 customers; 10,000 are held out as control. In treatment, 6.0% buy; in control, 5.2%. The lift is 0.8 points: 720 purchases the campaign caused. With an average order value of €60, incremental revenue is €43,200. With a campaign cost of €12,000, incremental ROAS is 3.6x.

Lift: 6.0% − 5.2% = 0.8 pts
Incremental purchases: 0.8% × 90,000 = 720
Incremental revenue: 720 × €60 = €43,200
Incremental ROAS: €43,200 / €12,000 = 3.6x
Attributed ROAS: 6% × 90,000 × €60 / €12,000 = 27x

Both figures are true; only one tells you whether to run the campaign again.

The four most common mistakes

• A control group that is too small. With 500 people in control, a 0.8-point difference is noise. Work out the minimum size before, not after.
• A contaminated control. If the control group receives another campaign, or is reached through another channel, it is no longer a control.
• Measuring only immediate conversion. A campaign can lift this week’s sales and depress next month’s (purchases pulled forward). Measure the full window.
• One test and move on. A channel’s incrementality changes with seasonality and with the audience. Holdouts have to be a habit, not an experiment.

How we do it at FLYDE

The problem with incrementality tests has never been the concept, but the logistics: splitting the control at random in every channel, making sure nobody reaches it another way and then matching real sales to both groups. When customer data, audiences and sales live in different systems, each test is a weeks-long project that happens once a year.

In FLYDE the control group is held out when you build the audience, before it is activated in Meta, Google, email or whichever channel applies. The holdout is the same across every channel, because it comes from the unified profile and not from each platform, so a person in control cannot receive the campaign another way. Sales are matched to both groups as they arrive from ecommerce or point of sale, and the Incrementality module returns the lift, incremental revenue and incremental ROAS of every activation, with the confidence interval so you know whether the result is significant or noise.

Illustrative example using the figures in this article; not taken from any customer.

What changes isn’t the formula. It’s that measuring a campaign’s incrementality costs the same as launching it, so you measure it on all of them.

What to do with this before Black Friday and El Buen Fin

It’s too late to redesign November’s media plan. It isn’t too late to measure it properly:

• Hold out a control group in your two or three highest-budget audiences (repeat customers, abandoned carts, brand search). 10% is enough.
• Don’t touch that group in any channel during the campaign or the week after.
• When the season closes, calculate the incremental ROAS of each one and put it next to the attributed ROAS.

If you sell in Mexico, how to prepare El Buen Fin 2026 with the data you already have covers the four lists to start with and how to hold out a control group even by hand.

The gap between those two columns is the most useful figure you’ll have for planning your 2027 budget. If you want to know how much revenue you’re leaving in inflated attribution, the dormant revenue calculator gives you a first figure in two minutes, and the customer data self-assessment tells you whether your data is ready to measure this way.