agile australia 2014
TRANSCRIPT
@amir_ansari @dius_au1
UX: more than a gut feel
Amir Ansari – DiUS
@amir_ansari @dius_au2
6’3”
@amir_ansari @dius_au3 http://lyndabuitrago.com/tag/leaky-gut/
@amir_ansari @dius_au4
@amir_ansari @dius_au5
@amir_ansari @dius_au6
product and technology strategy
application-specific devices
web applications
mobile applications
@amir_ansari @dius_au7http://dralexkent.files.wordpress.com/2011/12/gut.jpg
We feel we know…• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea
@amir_ansari @dius_au8
What is UX?
@amir_ansari @dius_au9
http://www.uxisnotui.com/
@amir_ansari @dius_au10
Study and Design of ALL aspects of a person’s EXPERIENCE with a product or service
@amir_ansari @dius_au11
@amir_ansari @dius_au12
@amir_ansari @dius_au13
The sooner we can find which features are worth investing in, the sooner we can focus our limited resources on the best solution to our business problems.
Jeff Gothelf – LEAN UX
“
@amir_ansari @dius_au14
Quantitative Qualitative
What? Why?• Data is solid and repeatable• Confirm and deduce• Result-oriented• Large sample sizes
• Data is rich and deep• Explore and describe• Process-oriented• Small sample sizes
• A/B or multi-variant testing• Analytics• Remote un-moderated
usability testing• Surveys
• Focus groups• Cognitive walkthroughs• Moderated usability
testing• Card sorting
@amir_ansari @dius_au15
@amir_ansari @dius_au16www.r-66.info
Completion rate
Task time
Task level satisfactionConversions
Page viewsSystem Usability Scale (SUS)
Net Promoter Score (NPS)
@amir_ansari @dius_au17
Qualitative
Why?• Data is rich and deep• Explore and describe• Process-oriented• Small sample sizes
• Focus groups• Cognitive walkthroughs• Moderated usability
testing• Card sorting
@amir_ansari @dius_au18
How do we measure and
validate these? And When?
@amir_ansari @dius_au19
Visioning Iterations Iterations Iterations Iterations MVPLaunch
Inception & Planning
1 2 3
@amir_ansari @dius_au20
Technique 1
@amir_ansari @dius_au21
@amir_ansari @dius_au22
Visioning Iterations Iterations Iterations Iterations MVPLaunch
Inception & Planning
1 2 3
@amir_ansari @dius_au23
30 – 60 mins 3 One on one interviews
@amir_ansari @dius_au24
• Concerns• Fears• Pain points• Ideal experience• Ideal channel
OUTCOMES
@amir_ansari @dius_au25
• Some had not come prepared• Stakeholders wanted to ask
‘speculative questions’
@amir_ansari @dius_au26
• Stakeholders observing• Call centre Staff felt engaged
and included • More insights than we had
anticipated
@amir_ansari @dius_au27
Technique 2
@amir_ansari @dius_au28
@amir_ansari @dius_au29
Visioning Inception & Planning
1 2 3
Iterations Iterations Iterations Iterations MVPLaunch
@amir_ansari @dius_au30
Focus group 3
2 hours
Focus group 1
Focus group 224
@amir_ansari @dius_au31
@amir_ansari @dius_au32
OutcomesFeature 1 Feature 2 Feature 3
@amir_ansari @dius_au33
• 2 hours wasn’t quite enough• Running sessions back to back
in the same evening!
@amir_ansari @dius_au34
• Stakeholders buy-in through observation• Validate assumptions• Client was anonymous
@amir_ansari @dius_au35
Technique 3
@amir_ansari @dius_au36
Visioning Inception & Planning
1 2 3
Iterations Iterations Iterations Iterations MVPLaunch
@amir_ansari @dius_au37
1 hour One on one testing6
@amir_ansari @dius_au38
http://www.nngroup.com/articles/how-many-test-users/
@amir_ansari @dius_au39
@amir_ansari @dius_au40
@amir_ansari @dius_au41
• Refinements sometimes not subtle enough• Design ping pong
@amir_ansari @dius_au42
• Speed of testing• The feedback loop• Stakeholder buy-in
(live stream)• Rapid de-brief• One designs and one tests
@amir_ansari @dius_au43
1. Background
2. Falsifiable Hypotheses
3. Details
6. Next Actions
5. Validated Learning
4. Results
User 1 User 2 User 3 etc.
Hypothesis 1 Hypothesis 2 Etc.
• What feature or assumption to test• What questions to ask• How well does X perform• Etc. • What to test next
• What hypothesis do we want to validate
1. That majority of users think X2. That majority of users perform Y 3. That majority of users need Z4. Etc.
• Previous experiment taught us …• We want to find out…• We want to test our solutions…
1. We can proceed with X2. We need to tweak Y3. We need to pivot and get rid of Z
Title: Experiment 2: XXX Author: Created:
• From focus groups, it was clear that one barrier with other online providers has been the lack of trainer support
• That providing access to trainers in the classroom and telling students upfront about trainer availability will eliminate barriers and increase conversion
• Show users the marketing pages that talk about trainer availability
• Show users the classroom with the trainer being online
• Users indeed look for trainer availability and are comforted by knowing OTI provides access to them
• Test something else! ; )
@amir_ansari @dius_au44
bit.ly/rapid-debrief
@amir_ansari @dius_au45
@amir_ansari @dius_au46
@amir_ansari @dius_au47
@amir_ansari @dius_au48
@amir_ansari @dius_au49
@amir_ansari @dius_au50
5
One on one testingFocus groups
= $8002 FTEs
X $160X $375*
* Daily gross equivalent based on a gross annual salary of $90k
= $750
$1,550
16 = $3,2003 FTEs
X $200X $375*= $1,125
$4,325
If doing this internally…
@amir_ansari @dius_au51
@amir_ansari @dius_au52
Define why, what, when who, where how, how often
1
@amir_ansari @dius_au53
Become friends with the PM (or the person with the budget)
2
@amir_ansari @dius_au54
Keep it lean and measure often
3
@amir_ansari @dius_au55
Get stakeholder buy-in(Get them to observe)
4
@amir_ansari @dius_au56
Be consistentin how you measure and capture measurements
5
@amir_ansari @dius_au57
Define when to Quant and when to Qual
6
@amir_ansari @dius_au58http://dralexkent.files.wordpress.com/2011/12/gut.jpg
Replace with more powerful point• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea
@amir_ansari @dius_au59
Thank you
Qualitative data can in fact be converted into quantitative measures even if it doesn't come from an experiment or from a large sample size.
Jeff Sauro – Measuring Usability“
@amir_ansari @dius_au60
Resources
@amir_ansari @dius_au61http://stamfordinteractive.com.au/conversations/benefits-of-guerrilla-testing/
@amir_ansari @dius_au62
You can quantify qualitative data!
bit.ly/quantifying-qualitative bit.ly/t-distribution
@amir_ansari @dius_au63
Completion rate
Task timeTask level satisfaction
Conversions
Page views
System Usability Scale (SUS)
Net Promoter Score (NPS)
{Visioning Inception & Planning Iterations Iterations Iterations Iterations MVP
Launch
@amir_ansari @dius_au64
https://www.measuringusability.com/topics/UX