how eye tracking research helps you run better a/b tests
TRANSCRIPT
Visual Engagement Metrics: Your Optimization Secret WeaponVisual Engagement Metrics: Your Optimization Secret Weapon
Visual Engagement Metrics » Your Optimization Secret Weapon
Visual Engagement Metrics: Your Optimization Secret WeaponVisual Engagement Metrics: Your Optimization Secret Weapon
Agenda● Quick conversion research overview ● What is visual engagement research ● How to set up an eye tracking study ● Case Studies from Sticky ● Case Studies from ConversionXL Institute
○ Ecommerce Product Pages○ Visual Cues on a Landing Page
● Q&A
Visual Engagement Metrics: Your Optimization Secret Weapon
What’s a Winning Optimization Process?● Run as many tests as possible at all times (every day without a test
running on a page/layout is regret by default)● Win as many tests as possible● Have as high impact (uplift) per successful test as possible
Visual Engagement Metrics: Your Optimization Secret Weapon
ResearchXL● Finding opportunity areas
○ Digital analytics○ Technical analysis○ Heuristic analysis
● Finding possible solutions (and more opportunity areas)○ Qualitative surveys○ User testing○ Mouse tracking / session replay
Visual Engagement Metrics: Your Optimization Secret Weapon
Marrying qualitative & quantitative dataYou’re trying to answer a few questions:
● Who is my customer?● How are they interacting with my site?● What are their possible problems?● How can we potentially fix them?
Visual Engagement Metrics: Your Optimization Secret Weapon
Continuous innovation in customer understanding
● Better and better ways to answer “how are users interacting with my site?”
● These insights help form better A/B tests (and make you more money)● Introducing Sticky.AI
Visual Engagement Metrics: Your Optimization Secret Weapon
What is Visual Engagement Analysis?It’s essentially a visual display of the user experience at a scale large enough to infer trends and insights.
Visual Engagement Metrics: Your Optimization Secret Weapon
Types Visual Engagement Analysis
<6% correlation with mouse movement and eye tracking
Scroll Click/Hover Eye Tracking
Visual Engagement Metrics: Your Optimization Secret Weapon
How Visual Engagement Analysis Helps
● See if people get stuck● See if important elements are
being missed● Know if something is being
read● Know in what order elements
are being seen● All used to optimize layout,
creative, CTA placement, and much more.
Important branding is not seen
Majority of visual attention focused on celebrity face
Visual Engagement Metrics: Your Optimization Secret Weapon
What is Eye Tracking?Sticky is leader in web-based eye tracking
Visual Engagement Metrics: Your Optimization Secret Weapon
Why Visual Engagement is AwesomeData is displayed as an aggregate which is a valuable way to understand the overall experience of visitors
Correlates with >70% of action, purchase intent, likeability, recall and click
Accurate on device measurement using built-in camera
Scalable implementation and analysis
Quick data visual output - available in just hours
Simple drag and drop setup, participant interface, no install required
500,000,000 gaze points measured
amazon.com ( press Ctrl key 3 times to turn highlighting on/off)
display ad replacement (Look for the smiley face)
Innovative Technology
Visual Engagement Metrics: Your Optimization Secret Weapon
Analyzing Norwegian Airlines’ Visual Engagement
Visual Engagement Metrics: Your Optimization Secret Weapon
According to this map, there’s too much going on.
Visual Engagement Metrics: Your Optimization Secret Weapon
The improved version includes smaller images and a more
prominent call to action
Visual Engagement Metrics: Your Optimization Secret Weapon
Results
The CTA button is now the main focus after the redesign:
● 1st thing seen
● Seen 10% more
● Seen 2X longer
● Button is noticed 2X
Visual Engagement Metrics: Your Optimization Secret Weapon
UX Research with Eye-Tracking to Create Better Tests
We combine visual engagement metrics with surveys that determine things like recall, perception, and recognition to conduct better tests and find more opportunity areas.
Information Processing Theory
Incoming info (e.g., website homepage)
sensory memory
working memory
long-term memory
Visual Engagement Metrics: Your Optimization Secret Weapon
UX Research with Eye-Tracking to Create Better Tests
We combine visual engagement metrics with surveys that determine things like recall, perception, and recognition to conduct better tests and find more opportunity areas.
Information Processing Theory
Incoming info (e.g., website homepage)
sensory memory
working memory
long-term memory
Eye-Tracking
Post-task Survey
Visual Engagement Metrics: Your Optimization Secret Weapon
Examples - 1. Optimizing the eCommerce Product page2. Visual Cue Study
Ecommerce Product Page - Three part study:
1. How Product Image Size Impacts Value Perception2. How Product Image Size Affects Attention and Engagement3. How Does Product Copy Format Affect User Engagement?
Visual Engagement Metrics: Your Optimization Secret Weapon
How Product Image Size Impacts Value Perception
● Hypothesis 1: Larger Image = Higher Value Perception● Hypothesis 2: Hypothesis 1 might change with different product classes
Visual Engagement Metrics: Your Optimization Secret Weapon
How Product Image Size Impacts Value Perception
experience good
search good
Visual Engagement Metrics: Your Optimization Secret Weapon
How Product Image Size Impacts Value Perception
experience good
search good
+ ~$2
+ ~$15
Visual Engagement Metrics: Your Optimization Secret Weapon
How Product Image Size Affects Attention & Engagement
● Hypothesis: A large product image will garner more attention than a smaller one.
Visual Engagement Metrics: Your Optimization Secret Weapon
How Product Copy Format Affects User Engagement
● Hypothesis: Product information will be viewed differently depending on how it’s formatted
● 9 Variations based on description format - Paragraph, Bullets, Nothing (control)
Visual Engagement Metrics: Your Optimization Secret Weapon
What can we do with all this data?
● Quantify high impact opportunity areas (copy format, image size, etc.)● Develop stronger hypotheses
○ “Increasing product image size will increase order volume”
● Improve testing prioritization/strategy○ “Higher lift for techy type products”
Visual Engagement Metrics: Your Optimization Secret Weapon
Visual Cues StudySettle debates with data, not years old case studies
Which visual cue works best?
Visual Engagement Metrics: Your Optimization Secret Weapon
We studied 6 variations
● Human looking away form● Human looking towards form● Arrow● Triangular ● Line● Prominent form
Visual Engagement Metrics: Your Optimization Secret Weapon
Other studies using same approach:
Social Proof
Value Proposition
Pricing Page Plan Order
Trust Seals
Homepage Banner Ads
Ecommerce Benefit List Placement
Online Reading Patterns
Visual Engagement Metrics: Your Optimization Secret Weapon
Conclusion: Tying it All Together
Visual engagement studies lower the cost of A/B testing by focusing your design changes on the areas that need it most, making the live-traffic split experiments more effective.