don't fear the small numbers - hxr 2016 - dan berlin and susan mercer
TRANSCRIPT
#HxR2016
Don’t Fear the Small Numbers
Why a Few Design Research Participants
Can Provide Big Insights
Dan BerlinVP, Experience Research, Mad*[email protected]@banderlin
Susan MercerExperience Research Director, Mad*[email protected]@susanamercer
#HxR2016
Topics
Prevalent requests for large numbers of participantsHow small numbers provide big insightsDetermining the “right” number of participants
#HxR2016
Large vs. Small Numbers
#HxR2016
Design Research isn’t…Market Research
#HxR2016
#HxR2016
But we’re not in the Mad Men days any more
#HxR2016
Good design follows an iterative process
#HxR2016
Design Research studies aren’t…Clinical Trials
#HxR2016
#HxR2016
Design Research
#HxR2016
In-home study
#HxR2016
Operating Room Study
#HxR2016
Five to eight users for usability
finds 80-90% of problems
Proportion of Usability Problems Found by Number of Participants
#HxR2016Source: https://www.nngroup.com/articles/which-ux-research-methods/
UX Research Methods
#HxR2016
Find 85% of the problems, update design, repeat
#HxR2016
Small Numbers
#HxR2016
UX Research is Typically Directional
#HxR2016
We conduct hundreds of interviews each year
#HxR2016
Trends emerge after a few participants
#HxR2016
Outliers provide incremental data during qualitative research
#HxR2016
Usability studies are qualitativePositive: Users were able to find the “Buy Now” button.
3 out of 8 participants mentioned that the picture had no relevance to them.Consider: replacing this with an infographic or another picture that may draw in users.
6 out of 8 participants mentioned that the text in the buttons was hard to read.Consider: increasing the contrast of the text to ease readability.
#HxR2016
Priority Finding RecommendationMedium Participants said that the text was hard to read Increase the contrast of the text
High Participants were unable to locate the product; they said they expected it under Products & Services
Move XYZ product to the Product & Services area
High Participants were unable to discern the top from the bottom of the vial
Include a visual cue as to which is the top and bottom of the vial
Medium Participants were able to figure out how to use the insulin pump, but said that the instructions were very hard to follow
Add detail throughout the instruction manual
Typical Usability Study Results
#HxR2016
The “Right” Number of Participants
#HxR2016
Keep Design RFPs Open-Ended
#HxR2016
http://bit.ly/IndiPersonas
Demographics don’t indicate reasoning
#HxR2016Source: http://howiechang.com/post/3045331253
Segment Users by Needs
Transactional Informational
#HxR2016
Project Scope Influences Number of Participants
#HxR2016
Segment Larger Projects
Application
Features Navigation Content
#HxR2016
• 12-16 participants per project segment & user group
Discovery Interviews
• 8-10 participants per project segment
Usability Studies
Very General Rules of Thumb
#HxR2016
Why? To Maximize ROI on UX Research
