ton wesseling which test won london 2014 keynote #tle2014 a/b-testing
DESCRIPTION
The slides of my conversion rate optimization talk on how we won so many Which Test Won awards - data & psychology - I've showed 25 cases in this presentation - thanks for being there and/or reading the slides now. If you're not able to get the PDF version - hit me on twitter http://twitter.com/tonw , I'll follow back and DM the slides! The wheel register code is "boston" (all small), not "Boston"TRANSCRIPT
London
Ton Wesseling Online Dialogue
Chief Optimization Officer
#TLE2014 The Live Event @TonW
Learn how to win 10+ WhichTestWon A/B-‐Test Awards
N o w w i t h f r e e c o n t e n t i n s i d e :
ü 2 5 c o n v e r s i o n c a s e s !
O N L I N E D I A L O G U E G u i d i n g y o u t o d i g i t a l m a t u r i t y
3 years in a row N u m b e r 1 a g e n c y i n t h e N e t h e r l a n d s
i n o u r fi e l d o f p r a c t i c e ( a n a l y t i c s & c o n v e r s i o n )
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Univé: top 3 car insurance in the Netherlands
Start cases
“Foot in the door” Get your visitors to interact asap
+23% sales
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Van der Valk Hotels (large Dutch hotel chain)
Start cases
“Chunking” Divide content in diminishing chunks of 3 or 5 pieces of information
+14% sales
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Online Dialogue: Conversion Summit landingpage
Start cases
+29% interactions
Data Driven
Data Driven
Data Driven
Data Driven
Data Driven
Data Driven
Data Driven Data Driven
Data Driven
Data Driven
Data Driven
Data Driven
Data Driven Data Driven
Data Driven
Data Driven
Data Driven
Data Driven
Data Driven @ To n W
Data Driven
Persuasion Psychologist
@ B a r t S
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Persuasion Psychologist
Data Driven Persuasion Psychology
= C R O M a g i c
+
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Kras.nl (TUI): homepage effect on sales
Award cases
“Ambiguity Aversion” Make it crystal clear what your customer will get
+20% sales
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
fonQ.nl: Ecommerce – site search sales improvement
Award cases
“Carrot & Stick” Keep showing customers what they’re buying through the whole process
+49% sales
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Freo.nl: liN in loan applicaOons
Award cases
“Sequencing” CRO is a reciprocal dialogue: Take the concepts that are active in your customers brain as your starting point
+19% applications
W h y i s t h i s w o r k i n g ?
W W W = Wo r l d W i d e Wa l h a l l a "f o r
A n a l y t i c s & B r a i n Te s t i n g
b r a i n s … B e c a u s e : y o u a re c o n v e r t i n g
W h a t i n fl u e n c e s t h a t b r a i n ?
b r a i n s … Yo u a re c o n v e r t i n g
W h a t i n fl u e n c e s t h a t b r a i n ?
b r a i n s … Yo u a re c o n v e r t i n g
D a t a D r i v e n P e r s u a s i o n P s y c h o l o g y
C re a t i n g c o n v e r s i o n b o o s t i n g & w i n n i n g A / B t e s t p l a n s =
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
This is the method we use
Fact & Act
&
Tell Combine Analyze
Test Create Analyze Filter
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Let’s start: Filter & Analyze
Fact & Act
&
Tell Combine Analyze
Test Create Analyze Filter
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Freo.nl: more liN in loan applicaOons
Filter & Analyze
“The paradox of Choice” Offer a minimum of 2 and a maximum of 5 choices
+14% applications
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
MoneYou.nl: upliN in new saving accounts
Filter & Analyze
“In Gaze Placement” Put your most persuasive content in your customers’ gaze
+23% accounts
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Freo.nl: survey to understand visitor needs
Filter & Analyze
“Foot between the lips” Directly get a forced click
Significant more applications Survey inv i ta t ion vs . no survey inv i ta t ion
winner
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
And now: Create & Test
Fact & Act
&
Tell Combine Analyze
Test Create Analyze Filter
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Autonomy: Van der Valk conversion liN
Create & Test
Autonomy: “Let your customer think he’s free in his choices”
+25% sales
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
But.. : VakanOeveilingen.nl (aucOon website)
Create & Test
Urgency: “When we’re in a hurry, we love to be guided”
Significant more submits, but..
Less registrations
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Direct Priming: our newsleYer headline test
Create & Test
Direct Priming: “RepePPon leads to quicker reacPons”
(Title first article)
(Title last article)
+42% clicks Outgoing clicks from within the newsletter by
members who opened the newsletter
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Choice Paradox: MoneYou Call-‐to-‐AcOon test
Create & Test
Choice Paradox: “We love a few, but not to many opPons”
+16% accounts + 2 5 4 % c l i c k s ( o n C TA a r e a )
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Response efficacy: Kras.nl travel booking form
Create & Test
Response efficacy: “We’re more likely to act when we believe it will actually have the desired effect”
+18% sales
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Ambiguity aversion: Freo.nl applicaOon form
Create & Test
Ambiguity Aversion: “We prefer opPons that are certain”
+10% applications
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Visuel cue: Hotel Schiphol –scroll promoOon
Create & Test
Visual Cueing: “Our aTenPon is very easily influenced”
-23% bouncerate + significant sales
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Framing: MoneYou email & landingpage test
Create & Test
Equivalence Framing: “The way things are stated or portrayed, highly influences our choices”
Significant more account opens
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Conceptual Contrast: job posOng offers
Create & Test
Ugly Brother: “The way things are stated or portrayed, highly influences our choices”
Almost everyone chooses the maximum package
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Test done? Analyze!
Fact & Act
&
Tell Combine Analyze
Test Create Analyze Filter
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Kras.nl: from most sold to a last minute list
Analyze
+58% conversions B u t n o s i g n i fi c a n t t u r n o v e r d i f f e r e n c e
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Banking: what is your interest rate posiOon?
Analyze
PosiOon targeOng: “We like basing comparisons on just a few aTributes”
Winner when
number 1
Winner when not number 1
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Product pages: CTA color?
Analyze
Goal-‐directed behavior “We need other persuasion techniques when we’re ‘geXng things done’ than when we (want to) relax
Monday winner Saturday winner
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Kras.nl: new & returning visitor differences
Analyze
Returning visitor winner
New visitor winner
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Essent Belgium: landingpage test
Analyze
Returning visitor winner
New visitor winner
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
MoneYou landing page redesign
Analyze
+27% applications B u t … ( s e e n e x t s l i d e )
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Let’s dive in the MoneYou A/B-‐test data
Analyze
Cum. Conversion % per day" Difference in cum. Conversion %"Conversion difference
stabelizes"
Conversion difference stabelizes"
Why is the conversion percentage so high on
these 2 days?"
Why is the difference between A and B at first
negative and than positive?"
Hmm, euroclix.nl
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Affiliate partner delivering external moOvaOon
Analyze
Free money if you file an application
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
MoneYou.nl without visitor funding affiliates
Analyze
Cum. Conversion % per day" Difference in cum. Conversion %"
Very stable differences"
(with conversions)"
Much shorter, difference in first days is caused by returning
visitors (which we filtered out after this analysis)"
+100% applications!
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Combine & Tell
Fact & Act
&
Tell Combine Analyze
Test Create Analyze Filter
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Combine & Tell
!"#$%&'("#)*+,-(./,"#) !"#$%&'(%)%"#*+'
CRO becomes Customer Intelligence
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Telegraaf.nl: biggest Dutch newspaper first cookie concept: almost 100% opt-‐in
Combine & Tell
Defaults: “We tend to like what is chosen for us”
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Telegraaf.nl: biggest Dutch newspaper first cookie concept: almost 100% opt-‐in
Combine & Tell
Reinforcements: “We want to hear we’re doing good”
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Based on customer intelligence: we loved to be challenged
Combine & Tell
We won J
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
We help and / or educate Kras.nl: They won!
Combine & Tell
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
We teach: Master of online Persuasion
Combine & Tell
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Ever growing our customer knowledge
3 years of being a partner
ü MoneYou groei over jaren heen
#TLE2014 The Live Event @TonW ©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Just do it
Digital Data Persuasion Psychology
How to create winning A/B test plans?
F a c t & A c t
London
Ton Wesseling Online Dialogue [email protected] +3130 4100 170
#TLE2014 The Live Event @TonW
Contact info
one more th ing…
…we’ve loca ted ove r 200 pe rsuas ion techn iques…
w w w. W h e e l o f P e r s u a s i o n . c o m
w w w. W h e e l o f P e r s u a s i o n . c o m
T h e W h e e l o f P e r s u a s i o n
Temporary Access Code: “Boston” (valid until Tuesday / June 3th )
#TLE2014 The Live Event @TonW