What is incrementality in app marketing?

What incrementality means

Incrementality is the share of conversions that would not have happened without the ad.

Incrementality: incremental conversions divided by attributed conversions. If a campaign is credited with 1,000 installs and 300 of them would not have happened otherwise, its incrementality is 30%.

The idea is simple; the hard part is the word otherwise. You never get to see the same user in a world without your ad. So incrementality is always estimated by comparing two groups: one that could see the ads and one that could not. There is a short glossary definition of incrementality testing if you want the one-line version.

Is attributed the same as incremental?

No. Attribution and incrementality answer different questions, and both are useful.

• Attribution asks: which ad did this user touch before installing? It assigns credit, usually to the last click within an attribution window, so you can compare campaigns day to day.
• Incrementality asks: would this user have installed if the ad had never run? It measures cause, and it needs an experiment.

Attributed conversions: conversions credited to a campaign because a user clicked or viewed an ad first.

Incremental conversions: conversions the campaign caused. They are always a subset of attributed conversions, so incrementality sits between 0% and 100% (ignoring the noise in any one test).

The gap between the two numbers is made of people who were already on their way. They searched for the app by name and tapped the paid result, or they had seen a friend use it and an ad was simply the last thing on screen. The three users walkthrough on the start page follows one of each through the same week, and the explorable below it lets you drag the gap open and closed.

A worked example

Sample data: a made-up app to show the method, not a benchmark.

Forkly, a made-up food delivery app, gets about 30k installs a month with no ads. It spends about $30k on ads in a month.

The dashboard shows a $2.20 cost per install. The cost of each install the ads actually added is $5, more than double. That difference is what incrementality testing exists to find.

Why it matters for app marketers

• Budget goes where the credit is. Most teams move spend towards the lowest attributed CPI. If that channel is mostly collecting credit for organic users, budget flows the wrong way. The attributed vs incremental lab shows a ranking flip on sample numbers.
• Organic looks weaker than it is. Every organic install claimed by an ad lowers the organic line, so teams under-invest in the store page, referrals and brand.
• Finance asks a different question. A CFO wants to know what the spend added, not what it touched. Incremental revenue answers that; attributed revenue does not.

For a fuller treatment of the organic side, the App Attribution Guide's chapter on organic vs paid installs explains how installs get labelled in the first place.

How incrementality is measured

There are four broad approaches, from the most direct to the most modelled:

1. User-level holdout: a random slice of the audience is kept from seeing the ads. Ad platforms run these as lift studies.
2. Geo lift: ads run in some regions and stop in matched regions.
3. On/off tests: a channel is paused for a period and installs are compared with before and after.
4. Models: media mix modelling estimates each channel's contribution from aggregate data over time, ideally calibrated by tests.

Each has trade-offs in cost, precision and how much control you need. The guide to test types compares them, and the holdout simulator shows how test size decides what you can detect.

What a result looks like

A lift test reports three things:

• Lift: how much higher the test group's conversion rate was than the control group's, as a share of the control rate.
• A confidence interval: the range of lifts that fit the data. A result of +20% with an interval of +5% to +35% is very different from +20% with an interval of −10% to +50%.
• Incremental conversions: the rate difference times the size of the test group.

From those you get incrementality, incremental CPI and incremental ROAS. The lift test reader computes all of them from four numbers.

Common misunderstandings

• "Incrementality replaces attribution." It does not. Attribution runs every day at campaign level; incrementality tests run occasionally and calibrate how much to trust the daily numbers.
• "Low incrementality means the channel is useless." It means much of what the channel is credited with would have happened anyway. The remainder may still be worth the price.
• "One test settles it." Incrementality moves with spend level, season and audience. Retest when any of them changes a lot.
• "A big lift percentage means a big effect." A 20% lift on a 1% install rate is 0.2 extra installs per 100 people reached.

For the practical side of setting up a first test, this incrementality testing guide walks through a holdout from planning to readout.