What is it?

A holdout group is a subset of users intentionally excluded from an experience, campaign, or optimisation to measure what would have happened without it.

Holdout groups are commonly used in incrementality testing.

Examples in Action

• Excluding users from personalisation
  
• Measuring lift from marketing exposure
  
• Validating long-term impact of interventions
  
• Assessing sustained behavioural change
  

Typical Outcomes / Results

• Accurate measurement of net impact
  
• Reduced overstatement of results
  
• Better understanding of true effectiveness
  
• Improved strategic confidence
  

This definition reflects common measurement approaches in optimisation and marketing.