Split Testing
Measure lift by comparing users who see your ads with a control group that doesn't.
Split testing measures incremental impact at the user level. Your audience is divided into two groups:
- Treatment group: sees your ads.
- Control group: is held out and doesn't see your ads.
Both groups are otherwise alike. So any difference in conversions between them is the lift your ads caused.
How a split test works
- Assign users: Users are randomly split into treatment and control groups.
- Run the test: Ads are shown only to the treatment group for the test period.
- Normalize the control: The control group is adjusted to match the size of the treatment group, so results compare fairly.
- Measure lift: Conversions in both groups are compared to calculate incremental impact.
- Check significance: Results are tested to confirm the lift is real and not due to chance.
Learn more
| Article | What it covers |
|---|---|
| Statistical Significance | How to tell if your test results are reliable |
| Treatment / Control Assignment | How users are split into test and holdout groups |
| Understanding Normalised Control | How the control group is scaled for a fair comparison |
When to use split testing
- You can target and hold out audiences at the user level.
- You want a fast, direct read on a channel, campaign, or audience.
- You need clean results to calibrate your Marketing Mix Model.
When user-level tracking is limited, Geo-Testing measures lift through regional differences instead.
Updated 2 days ago
Did this page help you?