#conversion2015 amsterdam keynote ton wesseling

O N L I N E D I A L O G U E G u i d i n g y o u t o d i g i t a l m a t u r i t y
D e m y t h e n , d e l e u g e n s e n d e i l l u s i e s # C o n v e r s i o n 2 0 1 5 A m s t e r d a m A p r i l 1 4 t h 2 0 1 5
To n W e s s e l i n g
A / B - t e s t e n

E m a i l : t o n @ w e s s e l i n g . n e t !L i n k e d i n : t o n w e s s e l i n g !
T w i t t e r : @ t o n w !!
Ton Wessel ing !

onlinedialogue.com #Conversion2015 Amsterdam Ton Wesseling April 14th 2015
Webanal isten.nl

Onl ine Dia logue

Test ing.agency

Emerce magazine: conversion column

Conversion Conferences wor ldwide

Conversion speakers

Where is Craig Sul l ivan?

Replacing Craig

At least he quotes me

So..

More exper iments
If you double the number of experiments you do per year
youre going to double your inventiveness
Je Bezos, CEO Amazon 2004!

Like Amazon, Like Zalando Ge
rman
Eco
mm
erce
mar
ket,
than
k yo
u An
dr
Mor
ys

Exper iments per year
1000+

Why not?
Compare to the past?

Why can this lead to fa i lures?

A big weekly change!
26 out of 44 weeks: over 5% change!

Methods used to improve conversion

Methods companies plan to use

I t s just
HOT

Lack of resources

Test more

Importance of a structured approach

More exper iments

The negat ive effect trend
Time span!
Aver
age
effe
ct of
opt
imza
tion!
at co
mpa
nies a
pplyi
ng o
ptim
izatio
n!M
oney spend on!Optim
ization worldwide!

I t s not a solut ion
Its a method

I t s a way of working
Its DNA

So i t s a method:
When
HowWhat

Growth curve how i t was teached

Optimizat ion phases - ROAR
Time span!
Conv
ersio
ns p
er m
onth
!
Risk! + Optimization! + Automation! Re-think!
1.000 conversions!per month!
10.000 conversions per month!

Phase 1: br idge the gap Risk

Fake adwords test ing
Do theyclickon your idea?

Fake facebook test ing
Do theylikeyour idea?

Fake landing page test ing
Do theysubscribeto your idea?

Phase 1: br idge the gap Risk

Get out of the office

Risk

ROAR - Opt imizat ion
Time span!
Conv
ersio
ns p
er m
onth
!

Warning: low significance levels
Big chance the winner is not a real winner

Warning: low power levels
Big chance a real winner is not recognized

Example test set-up
90% significance level 100 conversions per variation Average conversion rate: 2%
20 out of 100 ideas is a winner with an average uplift of 10%

Average: 75% significance & 40% power

So:
100 tests will give 10 false positives (90% significance) 40% of the 20 winners are recognized (40% power)
8 real winners & 10 false positives NOT 18 improvements of 10%+

You need big numbers per var iat ion
Time span!
Conv
ersio
ns p
er m
onth
!

With 1000 conversions + a month
20+ tests a year

Tests with impact, on average
1 out of 3
Simply truth: youre just not always able to create a winner with enough impact!

So in th is case, youve impact
Every 6 weeks

Slower? Energy wi l l run out!

What to test? Business cases!
Delivery test with in stock. 4 weeks
or 2 days?!

What to test? Business cases!
Service charge, do it or not? Include in price or not?!

What to test? Big design changes

Warning: choose what to analyze up front
You just go and dig unitil you find something (you will always find something that seems true BUT is not!)

Dont stop the test once i t s s ignificant!

You just let i t run unt i l i t s s ignificant

You just let i t run unt i l i t s s ignificant
Your test has a fixed lenght!
Its even a fixed amount of test population, but we love to test business cycles and/or weeks!

ABTestGuide.com/calc

#conversion2015 amsterdam keynote ton wesseling

Business

i n e d

e d i n

b t e s t e n e

e u g e n s e n d e

t o n w e s s e

u s i e s

t w i t t e r

s t e r d

Design Skills Symposium - Ton Schaap Academy of Architecture Amsterdam

Conversion Jam 3 #cjam3 #cjam2013 Ton Wesseling Keynote - Optimize

Ton Goeman (Meertens Institute Amsterdam) Hans Van de Velde (Université Libre de Bruxelles)

Ton van Balkom VU-MC/GGZ inGeest Amsterdam

Conversion World 2015 Keynote by Ton Wesseling

An Introduction to Multigrid Methods - Peter Wesseling

Winning at optimization - Ton Wesseling

Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion rate optmizers

Innovatieve arrangementen Over slim bezuinigen Henk Wesseling, 6 jan 2011

Plan Amsterdam - Metropoolregio Amsterdam

Leanconf2014 - A/B-testing - How, When, Examples & Pittfalls by Ton Wesseling

Wesseling, Principles of Computational Fluid Dynamics,2001

2 TON, 4 TON, 6 TON, 8 TON, 12 TON, 20 TON BOTTLE JACK ... BOTTLE JACK & 12 TON AND 20 TON ... This

Zonder titel door Michel Wesseling

Conversion12 - Ton Wesseling - Online Dialogue

22-Ton, 27-Ton, 28-Ton & 35-Ton Vertical/Horizontal Log Splitters 22-Ton, 27-Ton, 28-Ton & 35-Ton Vertical/Horizontal

MULTIGRID ALGORITHMS - Generation/Wesseling - Introduction to... · 8 MULTIGRID ALGORITHMS 8.1. Introduction

Kookboekgeneeskunde Wat schieten behandelaar en patiënt op met evidence-based psychiatrie? Ton van Balkom VU-MC/GGZ Buitenamstel Amsterdam

More Than Anyone Can Do ... More Than Anyone Can Do vu university press amsterdam ton lathouwers Translated

Ochtendprogramma Namiddag programma Ochtendprogramma - Waardigheid en 2017. 11. 1.آ Harm Wesseling,