The IAB Advertising Effectiveness Council has recently released ‘A Guide to Designing Digital Ad Impact Studies’ to provide guidance on measuring digital advertising impact that will add confidence and precision to marketing investment decisions.
The guide consolidates thinking on testing digital advertising effectiveness covering all steps from creating a hypothesis, choosing the right metrics, understanding the best measurement methods, through to tips for presenting your results. The guide also includes case studies that showcase how methods have been applied to answer various digital advertising questions.
Controlled experiments sit at the core of measuring digital advertising effectiveness and offer the perfect opportunity to adopt best practice scientific methods to measure and validate digital advertising activities. Controlled experiments are used in the measurement of brand impact from digital campaigns, measuring cross-media effect and interplay of digital and other media, geo-tests of the impact of online activity on in-store sales and attribution models. They can also be used to optimise for current and future digital campaigns by providing understanding of the impact of targeting, ad context, ad format, and frequency.
One of the most common approaches is to use a Test & Control survey:
- Test respondents are identified as exposed to the campaign by tracking, then recruited for participation in a short survey.
- Control respondents are collected at random from visitors to the same sites as the exposed respondents, however they will not have seen the campaign creative.
- Simultaneous collection of both Test and Control respondents throughout the campaign period enables these studies to isolate the impact of exposure to the digital campaign as each group has equal likelihood of being exposed to other media.
Importantly controlled experiments measure for incrementality – not correlation – so that you can understand what your advertising activity did that would not otherwise have happened. Incrementality is the measure of the true value created by any business strategy, determined by isolating and measuring the results it caused, independent of other potential business factors.
The measurement of incrementality relies on best possible comparability of exposed and control experiment cells. Ensure you have:
- Adequate sample size in each cell for the precision needed to confidently answer your question (to detect smaller differences between groups you will require larger sample sizes)
- Random assignment of control and test groups
- Cells that match (e.g. demographically, attitudinally, in propensity to use your product, prior exposure etc)
- Your subjects in either the control or exposed group, not both
The metrics you use to evaluate effectiveness of advertising need to be aligned with your business objective. Success means different things to different companies and brands and there are many valid measures or short-term and longer term effect. However, avoid using easy metrics like click-through rate (CTR) that provide a narrow and potentially misleading view of advertising success.
Lastly, test and learn how to adjust your campaign to drive better results, apply and keep testing. Build benchmarks of key metrics internally and set targets that reflect the context of other measurement activities in your organisation.
To learn more, download a copy of the full whitepaper – ‘A Guide to Designing Digital Ad Impact Studies’ .
We’d love to hear your feedback, reach out and tell us about your digital advertising experiments.
