A holdout group is a subset of users intentionally excluded from an experience, campaign, or optimisation to measure what would have happened without it.
Holdout groups are commonly used in incrementality testing.
Examples in Action
- Excluding users from personalisation
- Measuring lift from marketing exposure
- Validating long-term impact of interventions
- Assessing sustained behavioural change
Typical Outcomes / Results
- Accurate measurement of net impact
- Reduced overstatement of results
- Better understanding of true effectiveness
- Improved strategic confidence
This definition reflects common measurement approaches in optimisation and marketing.