nux liverpool ux utopia
TRANSCRIPT
Sam Barton - Head of User Experience
NUX Liverpool - March 2016
@sam_barton @shopdirectux
@sam_barton | @shopdirectux
@sam_barton | @shopdirectux
Survival- led Business-led Customer-led
Dec ‘13
84%
TRANSFORM A
CATALOGUE BUSINESS
INTO A WORLD CLASS
DIGITAL RETAILER
@sam_barton | @shopdirectux
Digital Transformation
2003-07
Integration and
consolidation
2007-12
Catalogue to
etail capabilities
2012-today
Vision for world
class digital
@sam_barton | @shopdirectux
UX in Shop DirectWays of working
Problem solving Qualitative testing Quantitative testing
@sam_barton | @shopdirectux
Evidence led approach
Through data In our UX lab On site (100/month)
@sam_barton | @shopdirectux
Focus on learning
ExperimentsResearch
suggest hypotheses
evidence and learning
Not a hypothesis
“I think we should add this feature to
the page and measure it.”
@sam_barton | @shopdirectux
Hypothesis
“I predict change will effect metric.”
@sam_barton | @shopdirectux
@sam_barton | @shopdirectux
Rapid experiments
Reduce barriers to experiments to
push volume and innovation.
@sam_barton | @shopdirectux
Fail fast to learn better
Focus on learning through uncertainty.
90% Fail
Nearly All Fail
67% Fail
90% Fail
66% Fail
@sam_barton | @shopdirectux
“Trying new things constantly
and then abandoning them
without further study or work is
not iterating. That’s flailing.”
Laura Klein
@sam_barton | @shopdirectux
Iterate towards redesign
Incrementalism RedesignIteration
@sam_barton | @shopdirectux
Prioritise
based on
evidence
@sam_barton | @shopdirectux
Tools and technologyMix quantitative and qualitative research to focus on learning.
@sam_barton | @shopdirectux
Full UX tool kit
Experimentation
UX Lab/Guerrilla
Remote surveys (Userzoom)
Eye tracking
Session Replay (Use it Better)
Ease Measure
Quant Observation Huge sample
Qual Observation Small sample
Qual/quant Subjective Medium sample
Quant Observation Low sample
Quant Observation Big sample
Quant Observation Big Sample
@sam_barton | @shopdirectux
UX LabBring the customer into the business
@sam_barton | @shopdirectux
Eye tracking
@sam_barton | @shopdirectux
Quant UX research tools
Session Replay Ease Measure Remote testing
@sam_barton | @shopdirectux
AB Testing / Experimentation
AB Testing Analytics
@sam_barton | @shopdirectux
People and skills
UX Research
@sam_barton | @shopdirectux
Broad skills and specialisms
UX QA
UX Design
UX Testing
UX Dev (iDev)
Psychology, human behaviors
Cross device testing, automated testing
UI design, deep understanding of the sites
ABn testing and data analytics
HTML, CSS, advanced JavaScript
UX Research
@sam_barton | @shopdirectux
Team evolution2010
UX QA
UX Design
UX Testing
UX Dev (iDev)
UX Research
@sam_barton | @shopdirectux
Team evolution2010 2016
UX QA
UX Design
UX Testing
UX Dev (iDev)
@sam_barton | @shopdirectux
Broad UX vision
Building a team with broad skill sets,
with specialisms that can hone in on
the details.
@sam_barton | @shopdirectux
Notable experiments
@sam_barton | @shopdirectux
Product page:
Error messaging
£3.3m
2.9% increase in demand,
2.5% in AOV
0.4% in conversion.
@sam_barton | @shopdirectux
Gallery page:
Layout
less people viewing product page
increase in conversion from products viewed.
1.1%
@sam_barton | @shopdirectux
Multi-device test:
Sticky headers
1.4% lift in conversion on mobile
3.1% drop on desktop
@sam_barton | @shopdirectux
Mobile gallery
(0.2)%
1
2
3
@sam_barton | @shopdirectux
Persuasive messaging
2.7% significant lift in conversion.
2.7%
@sam_barton | @shopdirectux
Persuasive messaging - iteration
@sam_barton | @shopdirectux
Persuasive messaging - iteration
@sam_barton | @shopdirectux
Persuasive messaging - iteration
@sam_barton | @shopdirectux
Persuasive messaging - iteration
@sam_barton | @shopdirectux
Persuasive messaging - iteration
@sam_barton | @shopdirectux
What’s next?
@sam_barton | @shopdirectux
What’s next?Experimentation 2.0 Scaled efficient & intelligent experimentation
Connected data sources and visualisation of data
Machine learning technologies
Personalised testing
@sam_barton | @shopdirectux
In summary
• Investment in infrastructure and skills
• Environment to innovate
• Fail fast, learn faster
• The HiPPO is almost extinct!
Which
Test
Won ?
Control Challenger
@sam_barton | @shopdirectux
Gallery page layout Hypothesis accepted
1.1% increase to
conversion
Control Challenger
@sam_barton | @shopdirectux
Managing abandonment Hypothesis rejected
(2.8)% negative to
conversion
Control Challenger
@sam_barton | @shopdirectux
Product add to basket Hypothesis rejected
(1.2)% negative to
conversion
Control Challenger
@sam_barton | @shopdirectux
Collect+ messaging Hypothesis accepted
1.8% increase to
conversion
Control Challenger
@sam_barton | @shopdirectux
Default sort order Hypothesis accepted
0.5% increase to
conversion
Control Challenger
@sam_barton | @shopdirectux
Empty review stars Hypothesis rejected
(0.8)% negative to
conversion
Control Challenger
@sam_barton | @shopdirectux
Cookie messaging Hypothesis accepted
0.5% increase to
conversion
Control Challenger
@sam_barton | @shopdirectux
Enable filtering (latest offers) Hypothesis accepted
1.2% increase to
demand
Control Challenger
@sam_barton | @shopdirectux
Increased font size Hypothesis accepted
0.5% increase to
conversion
@sam_barton | @shopdirectux
Tie break question
Question: How many experiments did Shop Direct run last year?
693
Thank you
@shopdirectux | @sam_barton
shopdirect.com/careers