ton wesseling / @tonw keynote - my daily job as a growth hacker / conversion rate optmizers

O N L I N E D I A L O G U E G u i d i n g y o u t o d i g i t a l m a t u r i t y

To n W e s s e l i n g – G r o w t h H a c k e r # e l i t e c a m p E s t o n i a 2 0 1 4 / 0 6 / 1 4

M y D a i l y J o b

Growth magic

Data Driven Persuasion Psychology +

Email: [email protected] onlinedialogue.com Twitter: @TonW #elitecamp

3 things

ü  Learn to code

ü  Rethink your optimization strategy

ü  Make sure everyone knows the drill


Digita l El i te Camp


Fr iends


I t ’s a socia l th ing


I t ’s a socia l th ing

5PA1N


Amazing


Home


About me


As a k id


Your lucky number

3


But also

4


So why not

34


34 years ago


Sai l ing


Sai l ing

a very close operating team

or

just you


Winning

some big decisions

and

tweaking the details


That’s our business

#CRO

#elitecamp @TonW


Thanks to th is man


Qual i ty c i rc le


Deming said


Focus on qual i ty

“When you focus just on costs they will rise”


Test ing before rebui ld ing


Back to real i ty


Van der Valk


Van der Valk

ü  Unique website for each hotel

ü  1 centralized team

ü  Paid by % of their turnover


Onl ine Dia logue

ü  Analytics & optimization partner

They also have:

Development partner

Advertising partner


My 3 focus points

ü  Optimize new bookingsengine

ü  Testing software costs too high

ü  Let them choose for Valk


Step 1: data & control

ü  New Google Analytics main account ü  Google Tag Manager


Google tag manager & test ing tools

VWO

GTM


Data div ing

ü  Small % will visit the (rooms) bookingsengine

ü  They navigate through the room pages


Van der Valk


My 3 focus points


ü  Testing software costs too high



My 3 focus points


ü  Learn to understand behaviour



G+ as v i r tual team communicat ion


Onl ine Dia logue Yammer


Bookingsengine opt imizat ion


But first: stat ist ics - h istory

ü  4 out of 10 tests were successful

ü  With a 90% significance level

ü  Purchase cycle = 3 days

ü  Thousands room bookings per week


Power & Significance

ü  10 wrong winners every 100 tests

ü  Only +-65% of these winners were uncovered

ü  40 winners = 30 real winners, so

ü  45 winning ideas, 55 losing ideas

ü  Lucky: 75% of the winners is real


Cookie delet ion & cross device

ü  90% significance is ok

ü  We can stay at this low power level

ü  2 weeks max test-time in this case (to

reduce sample pollution)

ü  Only test 1 type of device at the same time


Ever been to Vegas?


Automate the non learning stuff !


Test ing costs t ime

Use it to LEARN


Learn

ü  If a new functionality should be build

ü  What message really impacts behavior

ü  What we should feed the bandit

Always Re-test before you decide on these 2:


Pretty h igh “cl ick back” numbers


No room avai lable


Just measure i t


No room avai lable

§  15% sees a no room available page

§  Conversion rate of them is 50% lower

§  Low number of conversions per week

§  2 weeks testing – low power warning

§  Pontentially high impact


Var iat ion


Test protocol

ü  Put the control live a week before test

ü  Always use a “copy control” as control

ü  After the test period – new traffic to

control – for again 1 week

(should be: 1 purchase cycle minimum)


Like th is


Results

ü  No significant uplift

ü  Only 30% clicks

ü  55% mouse overs

ü  The others bounce very fast from the page

ü  Without them: significant uplift!


Act ions

ü  Too little value for a re-test / new test

ü  Functionality: show other rooms

ü  Test with the dates on the widget


Booking.com - scarci ty


First – let ’s measure


Scarcity data

§  30% sees rooms with scarcity

§  Conversion rate of them is the same

§  A lot of conversions per week

§  2 weeks testing – pretty OK power


The code


The results

Significant uplift


YES!

Re-test


Re-test +


I t ’s so easy to code


The results

ü  Yes, significant result again

ü  The other 2 messages are also significant

ü  But : the combination is even significant

better than the scarcity version itself!


Act ions

ü  Build scarcity in the CMS system

ü  Re-test the combination of messaging

ü  Keep an eye on customer life time value

development from new guests through scarcity


Bonus Test story


Results

Significant worse


Fol low up test


Suddenly


My 3 focus points





My 3 focus points

ü  Test, learn and build proces




A/B-testguide

ü  Team communication

ü  Use the right aproach

ü  Store your knowledge


So – the 3 th ings you should remember

ü  Learn to code

ü  Rethink your optimization strategy

ü  Make sure everyone knows the drill


Test wait ing

O N L I N E D I A L O G U E G u i d i n g y o u t o d i g i t a l m a t u r i t y

To n W e s s e l i n g – G r o w t h H a c k e r # e l i t e c a m p E s t o n i a 2 0 1 4 / 0 6 / 1 4

M y D a i l y J o b

ton wesseling / @tonw keynote - my daily job as a growth hacker / conversion rate optmizers

Technology

elitecamp van

elitecamp focus

elitecamp sailing

elitecamp thats

elitecamp thanks

elitecamp friends

elitecamp amazing

elitecamp home