Start / Learn / MMM, MTA and tests

MMM vs MTA vs incrementality testing

Media mix modelling, multi-touch attribution and incrementality tests compared for app marketers: what each measures, the data needed, how they fit together.

Short answer

Attribution (last-touch or multi-touch) assigns credit for each conversion and runs every day. Media mix modelling estimates each channel's contribution from aggregate spend and outcomes over time. Incrementality tests measure cause directly but only occasionally. Most mature teams use attribution daily, tests a few times a year, and tests to calibrate a model.

Three ways to answer "what is working?"

Media mix modelling, multi-touch attribution and incrementality testing are three methods for judging marketing, and they answer different questions.

Attribution (last-touch or multi-touch): which ads did this user touch before converting, and how should credit be split?
Media mix modelling (MMM): across months of data, how much did each channel's spend move total outcomes?
Incrementality testing: in a controlled comparison, how many conversions did this channel cause?

None replaces the others. They differ in what data they need, how often they can run and how close they get to cause and effect.

Side by side

	Attribution (MTA or last-touch)	Media mix modelling	Incrementality test
Question	Who gets credit?	What did each channel contribute?	What did this channel cause?
Data	User-level touches and conversions	Weekly or daily spend and outcomes by channel	Test and control groups or regions
Measures cause?	No	Estimates it from patterns	Yes, within the test
Speed	Daily	Monthly or quarterly refresh	Weeks per test
Covers organic and offline?	Only what is tracked	Yes, as part of the model	Only the channel tested
Main risk	Credits users who would convert anyway	Correlated channels confuse the model	Small or contaminated tests

What is multi-touch attribution?

Multi-touch attribution (MTA): a set of rules for splitting credit for one conversion across several ad touches. This glossary entry on multi-touch attribution lists the common models.

MTA spreads credit more fairly than last-touch attribution, but it still divides credit among ads a user touched. It never asks whether the user would have converted with no ads. So an MTA model can over-credit retargeting and brand just as last touch can. On mobile, privacy limits on user-level data (iOS especially) also leave gaps in the touch paths MTA depends on.

What is media mix modelling?

Media mix modelling (MMM): a statistical model that relates spend in each channel to an outcome such as installs or revenue over time, allowing for seasonality, trends and other factors. This glossary entry on media mix modelling gives a short definition.

Strengths:

Works from aggregate data, so privacy limits on user-level data do not stop it.
Includes channels attribution cannot see, such as TV, outdoor and some influencer spend.
Gives diminishing-returns curves, useful for budget planning.

Weaknesses:

Needs a long history with enough variation in spend. If every channel always moves together, the model cannot separate them.
Results depend on modelling choices; two analysts can get different answers from the same data.
Refreshes slowly, so it is a planning tool, not a daily one.

Where incrementality tests fit

Tests are the only method of the three that measures cause directly, by creating a control group. Their limits are practical: each covers one channel, takes weeks, and costs the conversions you give up in the control group.

That makes tests valuable in two roles:

Settling specific questions. Is brand search incremental? Is retargeting worth its budget? The cannibalisation guide covers the usual candidates.
Calibrating the other methods. A test that finds a channel is 40% incremental tells you how much to discount its attributed numbers day to day, and gives an MMM a fixed point to check against.

Sample data: a made-up app to show the method, not a benchmark.

A made-up example of calibration: attribution credits a channel with 10k installs a month. A geo test finds 40% are incremental. Until the next test, the team reads that channel's dashboard numbers as roughly 4k real installs and plans spend on that basis. The ROAS lab shows the same correction applied to CPI and ROAS.

Which should you use?

Early stage, one or two paid channels: attribution daily, plus an occasional on/off or geo test on the biggest channel. An MMM needs more history and variety than you have.
Several channels, steady spend: attribution daily, a lift test per major channel each year, and test after any big change in spend or strategy.
Large budgets including offline: add an MMM, calibrated by tests, for annual and quarterly planning.

Whatever the mix, attribution data is the base layer every method draws on. MMPCompares has a plain explainer on what an MMP is if you are deciding where that data should come from.

Common confusions

"MMM is the privacy-safe replacement for attribution." It complements attribution; it cannot tell you which creative or campaign to change today.
"Our MTA model already shows incrementality." It shows a split of credit. Only a control group shows what would have happened without the ads.
"Tests are too slow to matter." A few well-sized tests a year can correct how you read months of daily data.

The what is incrementality guide covers the core idea if you are starting from scratch.

Questions

Is MMM better than incrementality testing?

They do different jobs. MMM estimates every channel's contribution at once from historical data; a test measures one channel's causal effect precisely. Teams with both usually calibrate the model with test results.

Does multi-touch attribution measure incrementality?

No. It splits credit among the ads a user touched, but never compares with users who saw no ads, so it cannot say what would have happened without them.

How often should I run incrementality tests?

For each major channel, about once or twice a year, and again after a big change in spend, targeting or season. Each test is a snapshot of incrementality at that spend level.

What is incrementality in app marketing?

Incrementality is the share of conversions that would not have happened without the ad. It is measured by comparing people who could see the ads with a similar group who could not, so it captures what the ads changed rather than what they were credited with.

Is attributed the same as incremental?

No. Attribution credits a campaign when a user touched an ad before installing, even if that user would have installed anyway. Incremental installs are only the ones the ads caused, so they are always equal to or fewer than attributed installs.

How big should my holdout group be?

Big enough that the control group collects enough conversions to detect the lift you care about within the time you can wait. Lower conversion rates and smaller expected lifts need bigger holdouts or longer tests; the holdout simulator on this site shows the trade-off.