slaying a sacred cow - ephmra
TRANSCRIPT
![Page 1: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/1.jpg)
When breakthrough innovation comes from where you least expected it.
Linda GrosjeanOrlando Wood
Slaying a Sacred Cow
![Page 2: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/2.jpg)
![Page 3: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/3.jpg)
q15
185 kg105 kg
200 kg 145 kg
What is the combined weight of Linda and Orlando, when dressed up as a cow?
A: B:
C: D:
![Page 4: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/4.jpg)
185 kg105 kg
200 kg 145 kg
What is the combined weight of Linda and Orlando, when dressed up as a cow?
A: B:
C: D:
q15d
![Page 5: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/5.jpg)
185 kg105 kg
200 kg 145 kg
What is the combined weight of Linda and Orlando, when dressed up as a cow?
A: B:
C: D:
a15RIGHT
£1 MILLION!
![Page 6: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/6.jpg)
1197lb
1198lb
The Wisdom of Crowds
![Page 7: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/7.jpg)
How can a crowd be as accurate as or even more accurate than an expert?
![Page 8: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/8.jpg)
We are good at anticipating others’ behaviour, but the truth is we are self-deceit machines
![Page 9: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/9.jpg)
No – not this one!
Using the Wisdom of Crowds to Slay the Sacred Cow
![Page 10: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/10.jpg)
BrainJuicer’s Predictive Markets Technique
• A crowd of 500 respondents (invited on a nationally representative basis)
• Crowd shown up to 15 ideas and asked to judge which is most likely to work in market
• Asked as a share buying/selling game
![Page 11: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/11.jpg)
BrainJuicer’s Predictive Markets Technique
![Page 12: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/12.jpg)
0
10
20
30
40
50
60
-40 -30 -20 -10 0 10 20 30 400
10
20
30
40
50
60
-30 -20 -10 0 10 20 30
Correlation 0.92
Savoury Snacks Parallel Test
Household Cleaners Parallel Test
Correlation 0.79
Predictive Market Result (Net Preference)
Traditional Monadic Result
(Purchase Intention)
Concepts green-lighted in both traditional and Predictive Markets techniques were the same
Predictive Market Picks out the Same Winners as Traditional Testing
Predictive Market Result (Net Preference)
![Page 13: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/13.jpg)
Picking out the Same Winners as Traditional Testing
Monadic Ideas Tested No. of Green Lighted Concepts
Savoury snacks 15 5 out of 5Household cleaners 10 5 out of 5Average 10 out of 10 Sequential-Monadic No. of Green Lighted Concepts
Telecoms UK 14 4 out of 5Telecoms France 14 2 out of 3Telecoms Spain 14 2 out of 3Chilled Food 11 3 out of 3Window cleaners 17 3 out of 4Chewing gum 7 4 out of 4Lighting ideas 20 9 out of 10Average 8.5 out of 10
![Page 14: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/14.jpg)
Greater discrimination vs. traditional tests
-9 -7 -10-3
-14
-26 -30
31
19 177 7
136
22
12 7
4-24
-13
-7
-35
-25
-15
-5
5
15
25
35
% o
f res
pond
ents
dou
blin
g (p
ositi
ve) v
s se
lling
(neg
ativ
e) s
hare
s
0
10
20
30
40
50
60
70
80
90
100
Sequ
entia
l Mon
adic
% D
ef/P
rob
Scores from traditional monadic test can give a flat and unhelpful result
Predictive Market is more discriminating, separating the wheat from the chaff
![Page 15: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/15.jpg)
The Crowd In Action – Linda & John!
![Page 16: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/16.jpg)
Validation of Technique in Pharmaceutical Research
• A series of experiments have been undertaken between Roche and BrainJuicer. For two of these parallel monadic data is available from traditional monadic research among health professionals (a targeted sample)
– Ad concept testing for MabThera RA in a European country
– Actemra Ad concept testing in the US
• The Actemra Ad concept testing was also conducted among Roche employees (an Internal Roche ‘crowd’)
![Page 17: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/17.jpg)
Experiment 1
Traditional480 rheumatologists in 6 countries
Face to face; followed by phone
CHF 400,000
12 weeks
Roche proprietary questions
Wisdom of the Crowd500 ordinary people in one European market
Online interview
£12,000 (25’000 CHF)
2 weeks
Standard questions
![Page 18: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/18.jpg)
Interview flow
Traditional• Assessment of awareness/
perception of current biologics
• Glimpse evaluation
• Detailed evaluation
• Fit of concept with MabThera personality
• Comparative evaluation of each concept
• Ranking of concepts vs. positioning statement
• Link with then existing campaign
Wisdom of the Crowd • Would you probably buy or probably sell
shares in each of these ideas?
• If you owned shares in these ideas, which would you.. Double .. Definitely Sell shares in?
• Which of these faces (next slide) best expresses how doctors would feel about this ad?
• Plus other diagnostic questions
![Page 19: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/19.jpg)
The crowd delivered a very similar result to the targeted sample of rheumatologists
Most encouragingLeast encouraging Encourage to find out moreLeast likely
Most likely ‘Passport
’
‘Boots’
‘Postcard’
‘Switch’
‘New route’
‘Making History’
Net Preference Score=Percentage doubling shares minus percentage
selling shares
‘Switch’
‘Making History’
‘New route’
‘Boots’
‘Passport’
‘Postcard’
= Same Result
Traditional Wisdom of the Crowd
![Page 20: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/20.jpg)
Experiment 2
Traditional202 US rheumatologists
On line interview
$120’000 (122’000 CHF)
10 weeks
Roche proprietary questions
Wisdom of the Crowd500 ordinary US people +
200 Roche Employees
On line interview
£12,000 (25’000 CHF)
Took 2 weeks
Standard questions
![Page 21: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/21.jpg)
Cement Creative Tree Creative
Experiment 2
![Page 22: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/22.jpg)
Same results – with clear winner in Cement advertising concept
Predictive MarketNet Preference Measure
Traditional ResearchPreference Measure
82%
18%
87%
13%
![Page 23: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/23.jpg)
Internal Crowd also returned the same overall result – Cement route is clear winner
Predictive Market External Crowd
Predictive MarketInternal Crowd
82%
18%
93%
7%
![Page 24: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/24.jpg)
January 09 FuturePast
30 - 50 K
Average200 - 500 K
CHFCost of ad concept
marketresearch historically
90% cost reduction feasibleif parallel pilots successful
-> Crowd method as possible new standard
Cost Savings of Predictive Market Approach
![Page 25: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/25.jpg)
3 Pharmaceutical Research Innovations in One
• Reduction of the enormous costs and time associated with ad and concept screening in pharmaceutical research
• Avoids overuse of expensive and rare doctor samples
• Ability to research sensitive subjects that should not go beyond the company walls
![Page 26: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/26.jpg)
What next for Roche and Predictive Markets?
• Continue validation of different types of “emotional research” with non-traditional respondents, including
– Name development– Name testing– Ad concept testing– Brand icon/image testing– Consumer message development and testing
![Page 27: Slaying a Sacred Cow - EphMRA](https://reader031.vdocuments.site/reader031/viewer/2022020915/619dae95c6a21151776bb643/html5/thumbnails/27.jpg)