--- title: Incrementality Glossary: Lift, Holdout, iROAS description: Plain definitions of incrementality terms for app marketers: lift, holdout group, confidence interval, power, cannibalisation, iROAS, geo lift and ghost ads. url: https://incrementalitycalculator.com/glossary/ --- [Start](https://incrementalitycalculator.com/) / Glossary # Incrementality glossary 21 terms, defined plainly. **Short answer** The key terms in incrementality testing, each in one or two sentences: lift, holdout, confidence interval, power, cannibalisation, iROAS, geo lift, ghost ads, MMM, MTA and more. ## Terms **Incrementality** The share of conversions that would not have happened without the ad. If a campaign is credited with 1,000 installs and 300 of them would not have happened otherwise, its incrementality is 30%. **Incremental conversions** Conversions the ads caused: the test group's conversions minus what the test group would have done at the control group's rate. **Attributed conversions** Conversions an attribution tool or ad platform credits to a campaign because a user clicked or viewed an ad before converting. Attribution records the last touch; it does not prove the ad caused the conversion. **Organic baseline** The installs or conversions an app gets with no paid push: store search, word of mouth, brand awareness, press. The baseline is what incrementality is measured against. **Cannibalisation** When a paid campaign is credited with users who would have installed anyway, so attributed installs go up while organic installs go down by a similar amount. **Holdout group** A randomly chosen slice of the audience that is deliberately not shown the ads. Also called the control group. Its conversion rate stands in for what the exposed group would have done without ads. **Test group** The part of the audience that can see the ads during an incrementality test. Also called the treatment or exposed group. **Lift** How much higher the test group's conversion rate is than the control group's, as a share of the control rate. A test rate of 1.2% against a control rate of 1.0% is a 20% lift. **Confidence interval** The range of lift values that fit the test data at a chosen confidence level, usually 95%. If the whole interval sits above zero, the test supports a real positive effect. **Statistical power** The chance a test detects a lift of a given size if it is really there. Tests are usually planned for 80% power: a real lift of the planned size is missed one time in five. **Minimum detectable effect** The smallest lift a test of a given size can reliably detect. Smaller audiences, smaller holdouts and lower conversion rates all raise it. **p-value** The chance of seeing a difference at least this large between test and control if the ads truly did nothing. Below 0.05 is the usual cut-off for 95% confidence. **Incremental CPI (iCPI)** Spend divided by incremental installs. It is always equal to or higher than attributed CPI, because incremental installs are a subset of attributed installs. **Incremental ROAS (iROAS)** Revenue from incremental conversions divided by spend. An iROAS under 1x means the campaign returns less extra revenue than it costs. **Geo lift test** An incrementality test that turns ads on in some regions and off in matched regions, then compares installs. Used when user-level holdouts are not possible. **Ghost ads** A holdout method where the platform records which control users would have been shown the ad, so test and control are compared on comparable users. Run by the ad platform, not the advertiser. **PSA test** A holdout method where the control group sees an unrelated public service ad instead of the brand's ad. It costs media money and can be skewed by how the platform delivers the PSA. **On/off test** Pausing a channel for a period and comparing installs with the period before and after. Cheap and simple, but seasonality and other changes can be mistaken for the effect. **Media mix modelling (MMM)** A statistical model that estimates each channel's contribution from aggregate spend and outcome data over time. It does not need user-level data, and lift tests are often used to calibrate it. **Multi-touch attribution (MTA)** Splitting credit for a conversion across several ad touches a user had. Like last-touch attribution, it assigns credit; it does not measure what would have happened without the ads. **Peeking** Checking a running test repeatedly and stopping the first time the result looks significant. It raises the chance of a false win well above the 5% the confidence level promises. ## Questions **What is the difference between lift and incrementality?** Lift compares conversion rates: how much higher the test group's rate is than the control group's. Incrementality usually means the share of attributed conversions the ads caused. Both come from the same test, reported two ways. **What is incrementality in app marketing?** Incrementality is the share of conversions that would not have happened without the ad. It is measured by comparing people who could see the ads with a similar group who could not, so it captures what the ads changed rather than what they were credited with. **Is attributed the same as incremental?** No. Attribution credits a campaign when a user touched an ad before installing, even if that user would have installed anyway. Incremental installs are only the ones the ads caused, so they are always equal to or fewer than attributed installs. **How big should my holdout group be?** Big enough that the control group collects enough conversions to detect the lift you care about within the time you can wait. Lower conversion rates and smaller expected lifts need bigger holdouts or longer tests; the holdout simulator on this site shows the trade-off. **Are the numbers on this site real benchmarks?** No. Every number is sample data for a made-up app, chosen to show how the method works. Your own incrementality depends on your app, channels and audience, so measure it with your own test.