adma video testing and measurement
DESCRIPTION
The presentation discusses the significance of testing in marketing campaigns.TRANSCRIPT
![Page 1: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/1.jpg)
Australian Direct Marketing Association
Testing
![Page 2: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/2.jpg)
Australian Direct Marketing Association
What is testing?
Testing is a marketing experiment in a live environment:
• Observing what customers actually do rather than what they say they are going to do
• Monitoring how results vary as various components of a campaign are altered.
• Using the results of testing to:
– predict the outcome of future activity.
– Improve our marketing performance
Australian Direct Marketing Association
![Page 3: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/3.jpg)
Australian Direct Marketing Association
3 approaches
?
1. Test on a small scale first then roll-out
3. Same approach to all then analyse results among different segments to spot variations
2. Do testing on a portion of a live campaign
Test
Test
Roll-out
Live campaign
Live campaign
![Page 4: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/4.jpg)
Australian Direct Marketing Association
Summary of testing options
Message components • Product • Proposition • Offer • Creative • Call to action
Delivery components • Targeting & segmentation • Communication channels • Format • Timing
Australian Direct Marketing Association
Test the important things…..
![Page 5: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/5.jpg)
Australian Direct Marketing Association
Testing case study
• Two simple objectives – Improve response rates – Increase amount donated
• Understanding donor segments – Relationship to disease – Value
With permission from Salmat
![Page 6: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/6.jpg)
Australian Direct Marketing Association
• Relationship to disease – Have the disease – Parent of someone with the disease – Relative / friend of someone with the disease – No relationship to the disease
With permission from Salmat
Test different propositions for each segment
![Page 7: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/7.jpg)
Australian Direct Marketing Association
• Value – Variable donations boxes based on last donation, increased in
increments of 20%
With permission from Salmat
Personalisation case study
![Page 8: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/8.jpg)
Australian Direct Marketing Association
With permission from Salmat
Testing case study
![Page 9: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/9.jpg)
Australian Direct Marketing Association
Example: Landing Page Test Options
Page components 1. Headline 2. Secondary Headline 3 Body Copy 4 Social Proof 5/6 Call to Action 7 Navigation 8 Image 9 Above/below the Fold
Australian Direct Marketing Association
![Page 10: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/10.jpg)
Australian Direct Marketing Association
Guidelines for accurate testing
• Ensure you always have a “control” – So you have a benchmark for comparison
• Have a big enough sample to test with – To produce reliable results
• Ensure your test sample is not skewed at all (compared to control) – Use a randomly-selected sample
• Decide how you are going to measure performance: –The measurement chosen must prove/disprove your hypothesis
Australian Direct Marketing Association
![Page 11: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/11.jpg)
Australian Direct Marketing Association
The importance of a control
Australian Direct Marketing Association
May July June
Response rate
Standard offer
New offer A
New offer B
Here there is no control: - A separate offer has been run in each month - Offer A appears to have out- performed the current offer - Offer B appears to have performed worse = Offer A appears to win
Offer test
![Page 12: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/12.jpg)
Australian Direct Marketing Association
The importance of a control
Australian Direct Marketing Association
May July June
Response rate
Standard offer
New offer A
New offer B
Introduction of control: - The current offer has been run in each month as a benchmark - Offer A did not perform as well as the current offer - Offer B performed better than the current offer = Offer B is the real winner
Offer test
![Page 13: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/13.jpg)
Australian Direct Marketing Association
The “significance” of a test result
Control New Offer A
Response rate
1.7%
1.4%
The test out-performed the control by 0.3% Should you adopt New Offer A – or could it be just a chance result? We need to know whether these results are likely to be repeated if we ran the test again
![Page 14: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/14.jpg)
Australian Direct Marketing Association
Having a Big Enough Sample
• In direct marketing, your sample is typically a subset of your customer base or contact list
• In digital, your sample is typically the volume of unique visitors or unique impressions accumulated over a period of time
• You need a quantity of sample that reflects your desire to detect even small differences between your test and control groups
Australian Direct Marketing Association
50%$ 60%$ 70%$ 80%$ 90%$ 95%$ 99%$
Req
uire
d S
ampl
e Q
uant
ity
Ability to Detect Statistically Significant Differences
![Page 15: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/15.jpg)
Australian Direct Marketing Association
Not Just Statistical Significance
Do a sense-check when interpreting results: • What was the competition doing when this test was running? • Just because this worked in one location does it mean it will work in
another? • The offer was successful in Summer – would it still work in Winter? • Were there any other abnormal factors in the marketplace which
might have affected the response?
Australian Direct Marketing Association
![Page 16: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/16.jpg)
Australian Direct Marketing Association
Not Just Statistical Significance
Did we make money?
When we balance the costs of a campaign against the incremental volume of sales – was it worth it?
What should we test next? e.g. offer A performed better than offer B, what will beat offer A?
![Page 17: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/17.jpg)
Australian Direct Marketing Association
Measurement
![Page 18: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/18.jpg)
Australian Direct Marketing Association
Tracking the customer journey
Awareness Interest Action Desire
A buyer typically goes through several stages before they buy
…which means we should have measures to track performance for each stage
Reach Influence Closure
![Page 19: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/19.jpg)
Australian Direct Marketing Association
Example: eCommerce
Awareness
Interest
Action
Desire
Daily Unique Visitors
Product Page Views
Completes a Registra:on Form
Purchase
![Page 20: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/20.jpg)
Australian Direct Marketing Association
Tracking issues
How do you match response to source when:
– Your campaign runs in several channels simultaneously
– Your product is sold through a third party sales channel
– Your product has a long consideration period
![Page 21: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/21.jpg)
Australian Direct Marketing Association
Common tracking options
Unique campaign and comms channel specific phone numbers
Outbound Channel Response Channel
Offline Broadcast
Display Media
Call-Centre
In-Store
Website
Unique redemption codes
Specific Search Call to Action
![Page 22: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/22.jpg)
Australian Direct Marketing Association
Indirect sales – a missing link
Clicks/visits
Sales
Calls
Known
Known
Unknown
Browse
Enquire
Buy
![Page 23: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/23.jpg)
Australian Direct Marketing Association
Long sales cycles
Sales
Time
What does this response curve look like for your product?
• Track enquiries & sales over a long period
• Establish a pattern
• Create a rule of thumb
![Page 24: ADMA Video Testing and Measurement](https://reader034.vdocuments.site/reader034/viewer/2022052303/555a2324d8b42ab3088b47fb/html5/thumbnails/24.jpg)
Australian Direct Marketing Association
Rule of Thumb Approach
• Can be used for indirect sales as well as an ‘early read’ for long campaign cycles
• Typical approach:
1. Establish a ratio for website visits or calls to reseller enquiries/sales
2. Establish a pre-campaign baseline for calls and website visits
3. Measure the uplift in calls/visits during and following the promotion
4. Extrapolate to sales using typical ratio