3 methods to master agile ux testing | userzoom webinar
TRANSCRIPT
1!
Webinar: 3 Methods to Master Agile UX Testing
Presented by: Ann Rochanayon,
Senior Director of UX/CX Research, UserZoom
2!
Rapidly Test Usability & Measure UX
UserZoom is the All-in-One UX Research Platform for the Data-Driven Enterprise
3!
• Chat box is available if you have any questions!
• Look out for a quick poll question!
• There will be time for Q&A at the end!
• We will be recording the webinar for future viewing!
• All attendees will receive a copy of the slides/recording!
• Continue the discussion using #UZWebinar!
Quick Housekeeping
3 Methods to Master Agile UX Testing!
4!
Ann Rochanayon Sr. Director UX/CX Research
UserZoom
Meet Your Speaker
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
5!
Agenda • What is Agile?!
• UX Research Challenges in Agile!
• 3 Methods to Master Agile UX Testing!
• Tips for UX Testing in Agile Development!
• Questions!
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
6!
What is Agile?
3 Methods to Master Agile UX Testing!
• Lightweight, iterative approach to software development that builds
software incrementally as opposed
to delivering it all at once at the end!
!
• An alternative to Waterfall development or traditional,
sequential development!
Continue the discussion using #UZWebinar
7!
Agile vs. Waterfall
3 Methods to Master Agile UX Testing!
Source: http://www.business-software.com/blog/waterfall-vs-agile-development-differ-matters/
Continue the discussion using #UZWebinar
9!
Benefits of Agile Development
3 Methods to Master Agile UX Testing!
Allows for change
Better risk management
Improved quality
Increased collaboration
Better visibility and transparency
Continue the discussion using #UZWebinar
10!
UX Challenges in Agile Development
3 Methods to Master Agile UX Testing!
• Agile is a framework for software development
• Does not necessarily take UX into account
• Speed, not enough time for ‘formal usability testing’
• May not have minimum viable product to test
• Designers may focus on immediate needs within user stories, neglect bigger picture
Continue the discussion using #UZWebinar
11!
Addressing UX Challenges in Agile
3 Methods to Master Agile UX Testing!
• Utilize remote testing techniques
• Run quick, iterative tests
• Copy prior studies, update and rerun
• Use real-time automated reporting
• Determine UX metrics to measure and benchmark the user experience
• Observe how design changes impact UX Agile metrics
Continue the discussion using #UZWebinar
13!
3 Methods for Agile UX Testing
3 Methods to Master Agile UX Testing!
Screenshot Click Test Tree Test Task-based Testing
Continue the discussion using #UZWebinar
14!
Method 1: Screenshot Click Testing
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
15!
UX Research Needs in Agile
3 Methods to Master Agile UX Testing!!
Top Agile UX Pain-Points Top UX Testing MethodsDesign – Which design has the best conversion? A/B test with screenshot click testing or task-based
testing
Image / Ad testing – Which image / ad do users prefer? A/B test with screenshot click testing or task-based testing
Attention – What grabs users’ attention? Screenshot click test
Navigation optimization – Can users efficiently find content? Screenshot click test, tree test, task-based testing, card sorting
Usability – Can users complete key tasks on the site? Task-based testing
Nomenclature / Terminology: Do users understand the terminology / naming used on the site?
Tree testing
Information architecture – How do users expect to find information? How should content be organized on the site?
Tree test, card sorting
Brand impression – How do users feel about the brand? Timeout testing, screenshot click testing, task-based testing
Continue the discussion using #UZWebinar
16!
Method 1: Screenshot Click Testing
3 Methods to Master Agile UX Testing!
• Determine findability of content
• Find out where users click first
• Evaluate first impressions
• Understand what grabs their attention
• Uncover UI flaws
• Conduct A/B testing with two or more designs
Continue the discussion using #UZWebinar
17!
Method 1: Screenshot Click Testing Metrics
3 Methods to Master Agile UX Testing!
• Success Rates
• First click testing – Difficulty
o How difficult or easy was it to find where to click on the site to find
information about _________?
• First click testing – Confidence
o How confident were you that you clicked the correct area to find
_______?
• First impression – Emotional response, overall impression
o What 3 words would you use to describe the page to a friend?
o What is your impression of the page?
Continue the discussion using #UZWebinar
18!
Method 1: Screenshot Click Testing Metrics
3 Methods to Master Agile UX Testing!
• Attention
§ Which area of the page first grabbed your attention?
• A/B Testing – Comparison / Preference
§ Which homepage do you prefer?
§ Which version of the product details page best presented the product features?
§ Which version best presented clickable links?
• Include images of the screenshots in the question to aid with recall• Give the versions neutral name; avoid old vs. new or names that imply order• Ask follow-up questions to understand why• Include a “No Preference” or “None of the above” option
Tips for Preference Questions
19!
Method 1: Screenshot Click Testing
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
20!
Method 1: Screenshot Click Test Results
3 Methods to Master Agile UX Testing!
Click Effectiveness / Click Success Rates
Continue the discussion using #UZWebinar
21!
Method 1: Screenshot Click Test Results
3 Methods to Master Agile UX Testing!
Heatmaps
Continue the discussion using #UZWebinar
22!
Method 1: Screenshot Click Test Results
3 Methods to Master Agile UX Testing!
Areas Clicked
Continue the discussion using #UZWebinar
23!
Method 2: Tree Testing
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
24!
UX Research Needs in Agile
3 Methods to Master Agile UX Testing!!
Top Agile UX Pain-Points Top UX Testing MethodsDesign – Which design has the best conversion? A/B test with screenshot click testing or task-based
testing
Image / Ad testing – Which image / ad do users prefer? A/B test with screenshot click testing or task-based testing
Attention – What grabs users’ attention? Screenshot click test
Navigation optimization – Can users efficiently find content? Screenshot click test, tree test, task-based testing, card sorting
Usability – Can users complete key tasks on the site? Task-based testing
Nomenclature / Terminology: Do users understand the terminology / naming used on the site?
Tree testing
Information architecture – How do users expect to find information? How should content be organized on the site?
Tree test, card sorting
Brand impression – How do users feel about the brand? Timeout testing, screenshot click testing, task-based testing
Continue the discussion using #UZWebinar
25!
Method 2: Tree Testing
3 Methods to Master Agile UX Testing!
• Test information architecture or navigation without visual cues, distractions, and
search
• Measure the effectiveness of a site
structure
• Evaluate the findability of content
• Determine how to organize content
Continue the discussion using #UZWebinar
26!
Method 2: Tree Testing Metrics
3 Methods to Master Agile UX Testing!
• Success rate - % of participants who successfully select the right item from
the navigation tree
• Efficiency – Success after one attempt vs. multiple attempts
• Time – Amount of time spent determining which item to select
• Confidence
§ How confident were you that you selected the right item?
• Ease of use
§ How difficult or easy was it to determine where you would click to find _____?
Continue the discussion using #UZWebinar
27!
Method 2: Tree Testing
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
28!
Method 2: Tree Test Results
3 Methods to Master Agile UX Testing!
% Clicked
% Selected
User Clicks
Continue the discussion using #UZWebinar
29!
Method 2: Tree Test Results
3 Methods to Master Agile UX Testing!
Click Effectiveness / Success Rates, Time
Continue the discussion using #UZWebinar
30!
Method 2: Tree Test Results
3 Methods to Master Agile UX Testing!
Click Summary
Continue the discussion using #UZWebinar
31!
Method 3: Task-Based Testing
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
32!
UX Research Needs in Agile
3 Methods to Master Agile UX Testing!!
Top Agile UX Pain-Points Top UX Testing MethodsDesign – Which design has the best conversion? A/B test with screenshot click testing or task-based
testing
Image / Ad testing – Which image / ad do users prefer? A/B test with screenshot click testing or task-based testing
Attention – What grabs users’ attention? Screenshot click test
Navigation optimization – Can users efficiently find content? Screenshot click test, tree test, task-based testing, card sorting
Usability – Can users complete key tasks on the site? Task-based testing
Nomenclature / Terminology: Do users understand the terminology / naming used on the site?
Tree testing
Information architecture – How do users expect to find information? How should content be organized on the site?
Tree test, card sorting
Brand impression – How do users feel about the brand? Timeout testing, screenshot click testing, task-based testing
Continue the discussion using #UZWebinar
33!
Method 3: Task-Based Testing
3 Methods to Master Agile UX Testing!
• Test navigation, design
• Test wireframe w/ hotspots, low fidelity prototypes
§ Enable links users are likely to click to track clickstreams
§ Must be hosted and web accessible
• Test user stories or tasks
Continue the discussion using #UZWebinar
34!
Method 3: Task-Based Testing Metrics
3 Methods to Master Agile UX Testing!
• Task completion / success – Defined by researcher
• Task efficiency – Time on task, Unique page views, # of clicks
• Task Difficulty / Ease of use
§ How difficult or easy was it to customize a _______?
• Satisfaction
§ Based on your experience with the prototype, how satisfied were you with the process of
customizing _______?
• Conversion
§ How likely are you to purchase from this site?
Continue the discussion using #UZWebinar
35!
Method 3: Task-Based Testing
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
36!
Method 3: Task-Based Testing Results
3 Methods to Master Agile UX Testing!
Quantitative: Task Effectiveness (Task Success)
Continue the discussion using #UZWebinar
37!
Method 3: Task-Based Testing Results
3 Methods to Master Agile UX Testing!
Quantitative: Task Efficiency (Number of Clicks, unique page views, time on task
Continue the discussion using #UZWebinar
38!
Method 3: Task-Based Testing Results
3 Methods to Master Agile UX Testing!
Quantitative: Rating scales, Net Promoter Score (NPS)
Continue the discussion using #UZWebinar
39!
Method 3: Task-Based Testing Results
3 Methods to Master Agile UX Testing!
Qualitative: Open-ended Questions
Continue the discussion using #UZWebinar
40!
Method 3: Task-Based Testing Results
3 Methods to Master Agile UX Testing!
Qualitative: Videos of participants’ screens with video and/or audio
Continue the discussion using #UZWebinar
41!
Method 3: Task-Based Testing Results
3 Methods to Master Agile UX Testing!
Behavioral: Heatmaps
Continue the discussion using #UZWebinar
42!
Method 3: Task-Based Testing Results
3 Methods to Master Agile UX Testing!
Behavioral: Clickstreams
Continue the discussion using #UZWebinar
43!
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
• Shop Direct – UK’s leading multi-brand digital retailer with annual sales of £ 1.7 billion
• Challenge: Lab-based research indicated that customers were annoyed with the checkout process
• Goal: Make checkout easier and improve conversion
• Solution: Conducted a series of remote task-based studies A/B testing two different HTML prototype designs
• Results: Identified and resolved 54 usability issues prior to release. Increased conversion by 5.7%.
Method 3: Case Study – Task-Based Testing
44!
Other Methods
3 Methods to Master Agile UX Testing!
• Card Sorting
• Understand users’ mental models for grouping content
• Use to inform site’s information architecture
• Timeout Testing / 5 Second Test
• Learn how users feel about your webpage design
• Gain insight into their emotional response
• Determine if they understand the value proposition
Continue the discussion using #UZWebinar
45!
Tips for Agile UX Testing
3 Methods to Master Agile UX Testing!
Continue the discussion using #UZWebinar
46!
Tips for Agile UX Testing
3 Methods to Master Agile UX Testing!
• Include a UX professional as part of the agile team
• Research should be conducted 1 or 2 sprints ahead of development
• Consider setting up a parallel track for UX
Continue the discussion using #UZWebinar
47!
Tips for Agile UX Testing
3 Methods to Master Agile UX Testing!
Source: http://uxmag.com/articles/the-rite-way-to-prototype
• Consider using RITE (Rapid Iterative Testing and Evaluation)
• Don’t need high fidelity prototype or
working code
• Use low fidelity prototype, “paper
prototypes”, PowerPoint mockups,
Simple black and white wireframes with HTML
Continue the discussion using #UZWebinar
48!
Tips for Agile UX Testing
3 Methods to Master Agile UX Testing!
• Have a recruitment plan
§ Consider utilizing customer list, customer council / online community, private
panel of users, or outside panel
• Take advantage of online reporting / data visualization
Continue the discussion using #UZWebinar
49!
Additional Resources
3 Methods to Master Agile UX Testing!!
• Webinars• How to Integrate UX into an Agile Environment
http://info.userzoom.com/integrate-agile-ux.html?source=Website&llr=resources&medium=&olr=resourcesDean T. Barker, Sr. Director of User Experience at Optum
• How to Master UX Testing in an Agile Design Processhttp://info.userzoom.com/how-to-master-ux-testing-in-an-agile-design-process-webinar.html?source=Website&olr=resources&llr=resourcesJon Innes, ‘The Agile UX GURU’ at UX Innovation LLCJeff Sauro, ‘The UX Measurement MASTER’ at Measuring Usability
• Formative Usability Testing in Agile: Piloting New Techniques at Autodeskhttp://info.userzoom.com/formative-usability-testing-webinar.html?source=Website&olr=resources&llr=resourcesEunice Chang, Sr. Principal User Researcher at AutodeskOlivia Williamson, Principal User Experience Designer at Autodesk
Continue the discussion using #UZWebinar
50!
Additional Resources
3 Methods to Master Agile UX Testing!!
• Articles• Waterfall to Agile UX: How Autodesk Made the Switch
http://www.userzoom.com/agile-ux-research-and-testing/waterfall-to-agile-ux-how-autodesk-made-the-switch/• How Shop Direct Improved their Conversion Rates with UserZoom• http://www.userzoom.com/successstory/shop-direct/• Dispelling 4 Common Agile UX Myths
http://www.userzoom.com/agile-ux-research-and-testing/dispelling-4-common-agile-ux-myths/Jen Kirschniok, Digital Experience Expert UserZoom
• Agile & Remote Usability Testing: A Love Storyhttp://www.userzoom.com/agile-ux-research-and-testing/agile-remote-usability-testing-a-love-story/
• 12 Best Practices for UX in an Agile Environment, Part 1 and 2http://www.uie.com/articles/best_practices/, https://www.uie.com/articles/best_practices_part2/Jeff Patton, AgileProductDesign.com
• eBook• The Essential Guide to Integrating User Experience and Usability Testing in Agile
http://info.userzoom.com/agile-user-experience-guide.html?source=website&olr=resources&llr=resourcesDean T. Barker, Sr. Director of User Experience at OptumJon Innes, ‘The Agile UX GURU’ at UX Innovation LLCJeff Sauro, ‘The UX Measurement MASTER’ at Measuring UsabilityEunice Chang, Sr. Principal User Researcher at AutodeskOlivia Williamson, Principal User Experience Designer at Autodesk