Incrementality labs
Interactive sample-data labs that show how incrementality tests behave.
Four browser-only labs, each built around one idea: how big a holdout needs to be, how to read a lift result, how paid pushes cannibalise organic installs, and how incremental CPI can reverse a channel ranking. All numbers are sample data.
Which lab do you need?
- 01Holdout test simulatorPick a holdout size, an audience and a true lift, then watch a simulated test day by day until the confidence interval clears zero.
- 02Lift test readerEnter test and control results and get the lift, a 95% confidence interval, a plain-English verdict and the usual misreadings.
- 03Cannibalisation labSee how a paid push can be credited with installs that were going to arrive organically, and what that does to the organic line.
- 04Attributed vs incremental CPI and ROASTwo sample channels side by side: how ranking them on attributed CPI and on incremental CPI can point budget in opposite directions.
Questions
Can I use these labs on my own campaign data?
You can type your own numbers into the lift test reader, and it will compute the lift and interval correctly. The other labs are built to teach the method on sample data; for decisions, run a properly designed test and read it with your own analyst or the platform's study report.
What is incrementality in app marketing?
Incrementality is the share of conversions that would not have happened without the ad. It is measured by comparing people who could see the ads with a similar group who could not, so it captures what the ads changed rather than what they were credited with.
Is attributed the same as incremental?
No. Attribution credits a campaign when a user touched an ad before installing, even if that user would have installed anyway. Incremental installs are only the ones the ads caused, so they are always equal to or fewer than attributed installs.
How big should my holdout group be?
Big enough that the control group collects enough conversions to detect the lift you care about within the time you can wait. Lower conversion rates and smaller expected lifts need bigger holdouts or longer tests; the holdout simulator on this site shows the trade-off.
Are the numbers on this site real benchmarks?
No. Every number is sample data for a made-up app, chosen to show how the method works. Your own incrementality depends on your app, channels and audience, so measure it with your own test.