Incrementality in Marketing to Measure the True Impact of Your Campaigns

Here’s something nobody talks about enough:

Your best-performing campaign might not be doing anything.

Not because the creative is bad. Not because the targeting is off. But because the users it’s “converting” were going to convert anyway. They’d already decided. Your ad just happened to be there when they finally clicked through.

That’s the marketing incrementality problem. And if you’re running mobile campaigns purely on last-click attribution, you’re almost certainly paying for conversions you didn’t cause.

What Actually Is Incrementality in Marketing?

Incremental marketing is the measure of genuine, additional impact your marketing creates.

Not reported conversions. Not attributed installs. Actual conversions that happened because of your campaign and wouldn’t have happened without it.

Think about it this way.

Say your campaign reports 1,000 installs this month. Looks great. But 400 of those users found your app through word of mouth, searched for it by name, and were going to install regardless of whether your ad existed. Your campaign didn’t cause those installs. It just showed up at the end.

The 600 remaining installs? Those are incremental. Those are the ones your budget actually produced.

That difference, 600 vs 1,000, changes everything. The cost per acquisition, the real ROAS, the decision about whether to scale that campaign or redirect the budget somewhere else entirely.

Incrementality in marketing gives you the denominator that actually matters.

Why Last-Click Gets This So Wrong

Last-click attribution gives 100% of the credit for a conversion to the final touchpoint before it happened.

Sounds simple. Easy to report. Completely misleading.

Here’s a realistic scenario.

A user discovers your app through an Instagram post. Sees your ad a few days later while browsing. Searches for the app name on Google a week after that. On the way to the App Store, they click a retargeting ad that was already following them across every surface.

Under last-click: retargeting campaign gets full credit.

Under incrementality measurement: retargeting campaign gets credit for precisely what it contributed, which in this case may be close to nothing, because the user had already decided.

This isn’t a minor rounding error. It’s a systematic bias that pushes budget toward channels that are good at showing up last, rather than channels that are good at creating genuine demand.

Retargeting campaigns are structurally brilliant at winning last-click. They target users who have already shown intent. Of course those users convert at high rates. They were already interested. The retargeting ad didn’t make them interested. It just collected the credit.

Meanwhile, the upper-funnel brand campaign that actually created that awareness and intent in the first place? It loses the last click almost every time. So last-click data says brand is underperforming. Budget shifts to retargeting. The pool of genuinely interested organic users slowly depletes. Retargeting performance starts to decline. The team wonders what went wrong.

What went wrong was the measurement.

Incrementality Testing: How You Actually Measure This

Incrementality testing is the method for separating real campaign impact from baseline conversions that would have happened without you.

The logic is simple even if the execution isn’t.

Split your audience into two groups. One sees your campaign (test group). One doesn’t (holdout group). After a defined measurement window, compare conversion rates between the two groups. The gap between them is your incremental lift.

Incremental lift = Test group conversion rate minus Holdout group conversion rate

If your test group converts at 8% and your holdout converts at 5%, the campaign generated 3 percentage points of genuine incremental lift. The 5% would have happened anyway. Your campaign added the 3 points on top.

Sounds straightforward. Three things make it genuinely hard.

First, the holdout needs to be clean. If users in the holdout group see your ads on a platform you didn’t exclude them from, the control is contaminated. Your lift estimate becomes unreliable.

Second, the test needs enough time. A 7-day test for a campaign targeting users with a 21-day decision cycle will over-represent early adopters and under-represent the organic baseline. The incrementality result looks better than reality.

Third, the groups need to be genuinely comparable. Random assignment sounds obvious but is frequently botched. Assigning holdouts by geography or device type instead of truly randomising creates selection bias that invalidates the whole test.

Get these three things right and incrementality testing tells you something genuinely useful. Get them wrong and you’ve spent weeks running a test that produces a number you can’t trust.

The Three Main Marketing Incrementality Testing Methods

Holdout Groups

The most widely used approach. Withhold your campaign from a randomly selected portion of your eligible audience. Measure conversion rates in both groups over the test period.

Works well when you have enough scale in both groups to produce statistically significant results, when the holdout period is long enough to capture your typical conversion cycle, and when you can genuinely prevent the holdout group from seeing the campaign across all relevant placements.

That last part is harder than it sounds in mobile. If your campaign runs across Meta, Google, TikTok, and programmatic networks simultaneously, you need holdout suppression across all of them or the test is meaningless.

Ghost Ads

A more rigorous version of holdout testing. The holdout group sees a fake ad, same format and placement as the real ad, carrying no actual message. They experience the act of being shown an ad without receiving any information from it.

This controls for the attention effect of an ad appearing versus not appearing, and produces a cleaner experimental result.

Technically demanding. Requires ad network cooperation. Not always feasible. But when you can run it properly, it’s the closest thing to a true controlled experiment in advertising.

Synthetic Control

When live holdout testing isn’t possible, statistical modeling reconstructs what would have happened in the absence of the campaign using historical data.

Useful for campaigns where withholding ads from a portion of the audience isn’t operationally viable or commercially acceptable. Less clean than a live test. Still considerably more useful than assuming last-click attribution is telling you the truth.

How to Measure Incrementality in Mobile Specifically

Mobile attribution adds complications that don’t exist in web measurement.

ATT consent gaps on iOS. Since iOS 14.5, users who declined ATT consent can’t be individually identified. You can’t assign them cleanly to holdout groups based on user-level signals. Privacy-preserving incrementality approaches use aggregated data and SKAdNetwork postbacks to estimate lift across non-consented users. The signal is noisier. It’s still better than nothing.

Attribution window misalignment. Your MMP might measure on a 7-day click window. Your test might run for 14 days. The mismatch can make campaigns look more incremental than they are because late conversions within the test period aren’t attributed but are still counted.

Fraud contamination. If fraudulent events are inflating your test group’s reported conversions, your lift estimate is wrong. The holdout group doesn’t receive fraudulent activity the same way the test group does, so the difference between groups includes fraud noise alongside genuine campaign impact. Fraud needs to be cleaned before the incrementality analysis runs, not after.

Short decision cycles versus long ones. A gaming app where users decide to install within minutes needs a shorter test period than a fintech app where users research for weeks before converting. Matching test duration to your actual conversion cycle is one of the most commonly missed details in mobile incrementality testing.

What Changes When You Start Measuring Incrementality

The most consistent finding when teams run their first proper incrementality test: channels that looked great on last-click look considerably less impressive on incremental ROAS.

Retargeting almost always takes the biggest hit. The campaigns that produced the best last-click ROAS frequently produce the lowest incremental lift, because they’re primarily converting users who were already in the process of converting organically.

Brand and upper-funnel campaigns often show the opposite. They rarely win the last click. Incrementality testing reveals they were generating real demand the whole time, demand that last-click was quietly attributing to whoever showed up at the end of the funnel.

The budget implication is significant.

Teams running purely on last-click data tend to:

• Over-invest in retargeting
• Under-invest in brand and awareness
• Scale channels past the point where they’re generating genuine lift
• Miss the organic baseline collapse that happens when retargeted audiences are exhausted

Teams running incrementality measurement alongside attribution tend to:

• Allocate toward channels that generate real demand
• Understand when a channel’s incremental efficiency is declining before it collapses
• Make scaling decisions based on causal evidence rather than correlation

That’s a different kind of confidence when you’re presenting budget recommendations to a leadership team.

What Incremental ROAS Actually Looks Like

Reported ROAS and incremental ROAS are different numbers. Sometimes very different.

A campaign reporting $4 ROAS on last-click might look like this when you run the incrementality numbers:

Total reported conversions: 2,000
Holdout group organic conversion rate: 4%
Test group conversion rate: 6%
Incremental lift: 2 percentage points
Incremental conversions: approximately 800 out of 2,000 reported

Incremental ROAS on that campaign is calculated against 800 conversions, not 2,000. If the reported $4 ROAS assumed 2,000 conversions, the actual incremental ROAS might be closer to $1.60.

That changes whether you scale the campaign. Whether you continue it. Whether you rebalance the budget toward something showing higher genuine lift.

Last-click was hiding this. Incrementality revealed it.

Incrementality and Fraud: Why They Have to Work Together

Incrementality testing exposes mobile attribution fraud in a way that standard attribution doesn’t.

Here’s why. Fraudulent activity, click injection, SDK spoofing, incentivised installs with no genuine intent, inflates the test group’s reported conversion count. The holdout group doesn’t receive that same fraudulent inflation because they’re not being served the campaign.

When you calculate lift, the inflated test group conversion rate produces an artificially high incrementality number. Your campaign looks highly incremental. The reality is that a portion of the test group’s apparent conversions are fake.

If fraud isn’t cleaned before the incrementality analysis runs, you’re building budget decisions on corrupted data. You’re scaling campaigns based on fraudulent lift.

This is why fraud prevention and incrementality measurement need to operate in the same data layer. Clean the conversion data first. Then calculate lift. Any other sequence produces unreliable results that look like genuine insight.

How Apptrove Connects Attribution and Incrementality

Most measurement setups treat incrementality as a separate thing. A different report. A different tool. A different team running the analysis months after the campaigns ran.

That separation is the problem.

When incrementality data lives in a different place from attribution data, budget decisions still get made on attribution data because that’s what’s in front of people every day. Incrementality becomes an interesting quarterly exercise rather than an operational input to daily decisions.

Apptrove builds incrementality measurement into the attribution layer rather than bolting it on separately.

What that means in practice:

Holdout group logic runs inside the same infrastructure measuring your campaign events. There’s no discrepancy between what the holdout test is measuring and what your attribution reports are measuring. They’re looking at the same conversion events through the same pipeline.

Fraud protection runs before incrementality calculations, not after. The conversion data feeding the test is already cleaned when the lift analysis happens.

For iOS campaigns where ATT consent limits user-level identification, Apptrove handles incrementality estimation through privacy-preserving aggregated methods and SKAdNetwork postback processing. You don’t lose the incrementality signal just because users declined tracking permission.

Incremental ROAS sits alongside reported ROAS in campaign reporting. Not in a separate dashboard you have to remember to check. In the same view where budget decisions happen.

This is what makes incrementality actionable rather than academic.

When to Actually Run an Incrementality Test

Not every campaign needs one. You need sufficient conversion volume in both groups to reach statistical significance, and the setup cost isn’t trivial.

Run a test when:

Retargeting represents significant budget. This is the highest-priority use case. If retargeting is taking a meaningful share of spend and you haven’t tested its incrementality, you’re almost certainly overpaying for organic conversions.

You’re evaluating a new channel. Before scaling a new acquisition source, an incrementality test tells you whether it’s genuinely additive or whether it’s reaching the same users who would have found you organically.

ROAS has been declining despite stable or increased spend. Classic symptom of retargeting a depleting organic audience. Incrementality quantifies whether that’s what’s happening.

You’re making a major budget reallocation decision. Incrementality data is considerably more reliable for this than last-click attribution data, which systematically flatters channels that sit at the bottom of the funnel.

Your iOS attribution quality changed after ATT. If you can’t fully trust your iOS attribution data, incrementality testing on a channel-level basis provides a directional check that doesn’t depend on user-level tracking.

Common Incrementality Testing Mistakes

Tests that run too short. If your typical conversion cycle is three weeks and your test runs for one week, you’re measuring the wrong users. Late-converting organic users haven’t had time to appear in the holdout group yet. Lift looks better than it is.

Holdout groups that are too small. You need enough conversions in both groups to reach statistical significance. A small holdout group produces wide confidence intervals. The result is a number that looks precise but has so much uncertainty around it that it can’t reliably guide decisions.

Not suppressing the holdout across all placements. If you exclude holdout users from Meta but they still see your retargeting ads on Google and TikTok, the holdout is contaminated. The control group isn’t really a control.

Testing only retargeting. Retargeting is the obvious first test. But upper-funnel campaigns are where the most surprising and commercially significant findings tend to emerge. Test across your full media mix.

Ignoring fraud before the test. Running incrementality on uncleaned conversion data produces results that are partly measuring genuine lift and partly measuring the difference in fraud exposure between test and holdout groups.

Summing Up Incrementality in Marketing

Last-click attribution isn’t lying to you exactly.

It’s just measuring something different from what you think it’s measuring.

It tells you who showed up last. It doesn’t tell you who caused the conversion. In mobile marketing, where retargeting is built to intercept users who are already converting, that difference adds up to a lot of misallocated budget over time.

Incrementality in marketing is how you find out what’s actually working. Not which campaign claimed the last click, but which campaigns genuinely moved users from “not going to install” to “installed.” That’s a different question. It has a different answer. And it leads to different budget decisions.

The teams getting this right aren’t smarter. They’re just measuring what matters instead of measuring what’s easy.

FAQs on Marketing Incrementality

What is incrementality in marketing?

Incrementality measures the genuine additional impact of your marketing campaigns: the conversions that happened because of your campaign and wouldn’t have happened without it. It separates real campaign impact from baseline conversions that would have occurred organically.

How is incrementality different from last-click attribution?

Last-click attribution assigns full conversion credit to whichever touchpoint happened last before the user converted. Incrementality measures whether those conversions were actually caused by marketing. A user who was going to install your app anyway, and clicked a retargeting ad on the way, produces a last-click conversion but zero incremental lift.

How do you measure incrementality?

The most common method is a holdout test: split your audience into a test group (sees the campaign) and a holdout group (doesn’t). Compare conversion rates after a defined measurement period. The difference in conversion rates is your incremental lift. Ghost ads and synthetic control methods are alternatives when standard holdout testing isn’t feasible.

What is a good incrementality rate?

There’s no universal benchmark to measure marketing incrementality rate. What matters is how incrementality compares across your own channels over time. Declining incrementality as you scale spend on a channel usually means you’re running out of a genuinely convertible audience and starting to pay for organic conversions. That’s the signal to act on.

How does iOS ATT affect incrementality testing?

ATT consent limitations prevent individual-level identification for users who declined tracking permission. This makes clean holdout group assignment harder on iOS. Privacy-preserving incrementality methods use aggregated signals and SKAdNetwork postback data to estimate lift without relying on user-level identification.

When should I run an incrementality test?

Prioritise testing when retargeting represents significant spend, when you’re evaluating a new channel before scaling, when ROAS is declining despite stable or increased spend, and before major budget reallocation decisions. Also run tests periodically to track whether channel incrementality is changing over time.

What’s the minimum scale needed for incrementality testing?

It depends on your conversion rate and how tight you need the confidence intervals to be. As a rough guideline, you typically need thousands of users in both test and holdout groups and enough conversions in each to produce statistically significant results. Low-volume campaigns can still use synthetic control methods to get directional signal.

from Apptrove https://apptrove.com/incrementality-in-marketing/
via Apptrove