![Page 1: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/1.jpg)
THE TESTING CULTURE AT STUBHUB!
Bojan Ambrus!eCommerce Marketing Manager!
![Page 2: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/2.jpg)
AGENDA!• Introduction!• Pre-culture change!
Our product roadmap!
• Post culture change!Our new product roadmap and testing process!
• Communicating change!• Results, post culture change!
Lessons learnt!
![Page 3: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/3.jpg)
INTRODUCTION!• Who: StubHub!• When: December 2011 UK
launch!• What: A ticket marketplace
providing fans with a safe and convenient place to buy and an easy way to sell tickets!
• Why test: Increase buy/sell flow conversion, revenue and engagement!
• Test challenges: Perishable stock with a lack of control!
!
![Page 4: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/4.jpg)
WHY CREATE A TESTING CULTURE?!
“THE TRUE METHOD OF
KNOWLEDGE!IS EXPERIMENT.” !
![Page 5: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/5.jpg)
PRODUCT ROADMAP PRE CULTURE CHANGE!
Requirements and Planning • DEV, QA, DW
assigned • Item is prioritised
against other projects
Build • Development activities
completed • Feature is released to
production
Measure • Metrics collected and
published • Look Back presented
Wishlist • Requirements
submitted • Project KPIs defined • Item prioritised
Prioritised Backlog • Discovery team is
identified
Discovery • Usability, technical
feasibility and UX completed
• Analytics requirements and goals reviewed
Testing Process
![Page 6: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/6.jpg)
TESTING PROCESS
"IF YOU CAN’T MEASURE IT, YOU CANNOT IMPROVE IT."!
![Page 7: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/7.jpg)
PROCESS - HAVE DATA, NOT IDEAS!
![Page 8: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/8.jpg)
WHERE DO GREAT TEST IDEAS COME FROM?!
Build a culture of test and learn through the use of...!Source Actions Analytics (Omniture, Google Analytics, etc.)
Compare KPIs, like conversion, RPV and exit rates across top trafficked pages and sections of the buy and sell flow.
Product roadmap Align testing to the product roadmap to validate direction and design for new features and functionality.
Qualitative research (UserTesting.com)
Use qualitative research including heat maps and online user testing.
Customer feedback (NPS, Foresee, Brand Awareness Tracker, CS Contacts)
Identify customer pain points throughout the business and develop tests to improve the experience through messaging, functionality or design.
Competitive reviews (Seatwave, Viagogo, GetMeIn!)
Compare your site to your competitors as well as the sites you use and admire to gain test ideas.
![Page 9: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/9.jpg)
COMMUNICATING CHANGE!
"TELL ME AND I'LL FORGET.!SHOW ME AND I MIGHT REMEMBER.!
INVOLVE ME AND I WILL UNDERSTAND."!
![Page 10: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/10.jpg)
COMMUNICATING CHANGE!• Organise a working group!• Request testing ideas!• Share results!• Inspire ideas for change!• Create awareness!• Demonstrate value!• Outline testing process – have data, not
ideas!• Make brainstorming part of that process!
![Page 11: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/11.jpg)
![Page 12: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/12.jpg)
RESULTS!
![Page 13: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/13.jpg)
BUYER TRUST MODULE!
![Page 14: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/14.jpg)
CTA BUTTONS!
![Page 15: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/15.jpg)
VALUE PROPOSITION MESSAGING!
![Page 16: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/16.jpg)
LESSONS LEARNT!
![Page 17: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/17.jpg)
CREATING A TESTING CULTURE!
Top lessons learnt:!1. Experiment early and often!2. Question assumptions!3. Remember that every visitor is an opportunity!4. It's ok to be wrong!5. Integrate into business objectives!
![Page 18: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/18.jpg)
ANY QUESTIONS?!
![Page 19: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/19.jpg)
![Page 20: Case Study by Bojan Ambrus of StubHub - Optimizely Experience London 2014](https://reader034.vdocuments.site/reader034/viewer/2022051817/54842d0ab4af9faf0d8b4ab6/html5/thumbnails/20.jpg)