mastering the art of experimentation - tech circus€¦ · the lean ux process - jeff gothelf. a/b...
TRANSCRIPT
MASTERING THE ART OF EXPERIMENTATION
Chris Gibbins - Director of UX & Optimisation
Biglight.co.uk | @cjgibbins | @biglightbuzz
1
The future belongs to brands that provide remarkable customer experiences.
We exist to help them do just this.
SOME OF OUR CUSTOMER EXPERIENCE OPTIMISATION CLIENTS
WORKSHOP AGENDA
15:40
15:50
16:10
16:20
16:55
17:20
18:00
START
INTRODUCTION TO EXPERIMENTATION
THE EXERCISE
- CUSTOMER PROBLEM 1
- CUSTOMER PROBLEM 2
- CUSTOMER PROBLEM 3 (If we have time)
END
What about you?
What comes to mind when you hear the word
“TESTING” “EXPERIMENTATION”
INTRODUCTION TO EXPERIMENTATION
CREATIVE UX
INSIGHT
EXPERIMENTATION
1 Learn what really works for users
4 Iterate as you can’t get everything right in your first attempt
5 Measure the ROI of all our work
2 Be more adventurous in our ideas / innovate
3 Fail fast6 Become better UXers and Designers
EXPERIMENTATION IS KEY TO CONTINUAL IMPROVEMENT OF THE UX
“The most important single thing is to focus obsessively on the customer. Our goal is to be earth’s most customer-centric company.”
Jeff Bezos, CEO Amazon
“Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day,”
Jeff Bezos, CEO Amazon
OPTIMISATION / PERSONALISATION TOOLS
AB
C
Off-the-shelf solutions e.g. Develop your own e.g.
LIVE EXAMPLE
Control The original website
Variation 1UI enhancements to filters
Variation 2Horizontal filter layout
THE LEAN UX PROCESS - JEFF GOTHELF
A/B Testing live on the
website
Roll-out winning experiences with a much greater degree of confidence
+Calculate the uplift and ROI
Design & Build the A/B test variations
Several concepts at
a time
OUR PROCESSES ARE NOT THAT DIFFERENT
Last 4 months of UX and Optimisation activities for a client with 3 websites
35 A/B tests completed
Each one had on average of 3 design variations plus the control
12 significant winners (33% win rate)
TYPICAL PROGRAMME SHOWING HOW WE ITERATIVELY IMPROVE THE CUSTOMER EXPERIENCE
The Experimentation Exercise
Explore and investigate the customer problem making sure that you are clear on what the issues are.
Start sketching out ideas to solve the customer problem. Try to come up with at least 8 in total
Refine down to 4 ideas / variations which could be tested as an experiment. Be extra brave with the 4th ‘wildcard’ one!
1) Customer problem 2) Crazy Eights 3) Experiment design
10 mins
15 mins 10 mins
THE EXERCISE - 3 CUSTOMER PROBLEMS TO SOLVE
4) Present back
A couple of you to talk through the ideas
10 mins
IDENTIFYING CUSTOMER PROBLEMS AND OPPORTUNITIES
IDENTIFYING CUSTOMER PROBLEMS AND OPPORTUNITIES
Long product listing pages and too much choice make it hard for users to find their ideal product.
The customer problem in detail:
- Website example: COS www.cosstores.com
- Usability testing has shown time and again that users struggle when there is too much choice on very long product listing page
- User Interviews have revealed a need for more direction and also more inspiration to help users choose especially when there are a large number of products in a category
- In store research has revealed how staff members guide users to a smaller range of products that better fit the users’ needs.
The challenge
- How do we improve the overall experience on the product listings pages to help users more easily find their ideal products?
CUSTOMER PROBLEM 1.
90x Men’s Knitwear items
Takeaways- If you UXer or Product Designer who wants to make a measurable
difference, you really need to embrace online experimentation (e.g.
A/B testing)
- Push yourself and go beyond the obvious solution… be more
adventurous and you might just stumble across a solution that 10x
better for users.
- Be a little cautious of the phrase ‘best practice’. Often it’s used by
people wanting to push through their solution!
THANK YOU
Biglight.co.uk | @cjgibbins | @biglightbuzz
23