Incrementality Testing

What Incrementality actually means

An incrementality test withholds marketing from a matched control group and compares outcomes. It is the only method that measures causation rather than correlation, which makes it the closest thing marketing has to proof. The usual designs are geographic holdouts, audience splits and scheduled dark periods, and the usual finding is that reported returns overstate real ones.

Worked example

A brand pauses paid search in twelve matched metro areas for six weeks. Revenue in the test markets falls 7 percent while control markets stay flat. Paid search had been claiming credit for far more than 7 percent of revenue, so its true incremental contribution is a fraction of the reported figure, and budget can now be reallocated with actual evidence behind the decision.

Why the board cares

Incrementality testing is expensive in forgone revenue and slow to run, which is exactly why boards respect the results. It converts a budget argument from a debate about attribution models into a question about experimental design. One well-run holdout per major channel per year is a reasonable operating standard for any company spending seriously on media.

Common mistakes

• Running tests too short to capture the full purchase cycle, which reads delayed conversions as lost ones.
• Using control groups that are not genuinely matched, so the result measures market differences instead.
• Testing only the channels expected to pass, which makes the program a ritual rather than a measurement.

Related terms

• Incremental ROAS (iROAS)
• Multi-Touch Attribution (MTA)
• Marketing Mix Modeling (MMM)
• Marketing Efficiency Ratio (MER)

Back to the full marketing glossary