digital data tips tuesday #2 - a/b-testing: matt althauser - optimizely
DESCRIPTION
#ddtt #2 Keynote by Matt Althauser - the evolution of optimization.TRANSCRIPT
Matt Althauser GM, Optimizely EMEA
@malthauser
7,000
500,000
7,000,000,000
Join the Community:
www.optiverse.com
February 27th, 2000
Holy A/B Testing!
…?
One to Many One to Few One to One
Evolution of Optimization
What’s the best experience for the average user?
One to Many
2/27/2000-Present
=
Simple Tools
Original
Variation
+606% Engagement
Video
Images
+0.2 % Conversions
Video
Images
2006-Present
=
One to Few What’s the best experience for
segments of users?
Fire!
Fire!
Flow Analysis…
+15% Task Completion
2010-Beyond
=
Wheel!
One to One What’s the best
experience for each user on each device?
Donation Button Experiment
0.0% 0.0% 0.0%
2.3% 27.8% 16.3%
-27.8 N/A N/A
15.2% -24.6% 11.9%
8.5% 2.9% 18.4%
Donation Button Experiment Never
Signed-Up
0.0% 0.0% 0.0%
2.3% 27.8% 16.3%
-27.8 N/A N/A
15.2% -24.6% 11.9%
8.5% 2.9% 18.4%
Donation Button Experiment Never
Signed-Up Signed-Up No Donate
0.0% 0.0% 0.0%
2.3% 27.8% 16.3%
-27.8 N/A N/A
15.2% -24.6% 11.9%
8.5% 2.9% 18.4%
Donation Button Experiment Never
Signed-Up Signed-Up No Donate
Previously Donated
2%
Return Visitors 10%
Mobile Visitors 1%
Amsterdam Visitors 0.2%
New Visitors 2%
+28% Revenue/Visitor
App Store Blues
mobile.optimizely.com
@malthauser