agile australia 2014

@amir_ansari @dius_au1

UX: more than a gut feel

Amir Ansari – DiUS

@amir_ansari @dius_au2

6’3”

@amir_ansari @dius_au3 http://lyndabuitrago.com/tag/leaky-gut/

@amir_ansari @dius_au4

@amir_ansari @dius_au5

@amir_ansari @dius_au6

product and technology strategy

application-specific devices

web applications

mobile applications

@amir_ansari @dius_au7http://dralexkent.files.wordpress.com/2011/12/gut.jpg

We feel we know…• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea

@amir_ansari @dius_au8

What is UX?

@amir_ansari @dius_au9

http://www.uxisnotui.com/

@amir_ansari @dius_au10

Study and Design of ALL aspects of a person’s EXPERIENCE with a product or service

@amir_ansari @dius_au11

@amir_ansari @dius_au12

@amir_ansari @dius_au13

The sooner we can find which features are worth investing in, the sooner we can focus our limited resources on the best solution to our business problems.

Jeff Gothelf – LEAN UX

“

@amir_ansari @dius_au14

Quantitative Qualitative

What? Why?• Data is solid and repeatable• Confirm and deduce• Result-oriented• Large sample sizes

• Data is rich and deep• Explore and describe• Process-oriented• Small sample sizes

• A/B or multi-variant testing• Analytics• Remote un-moderated

usability testing• Surveys

• Focus groups• Cognitive walkthroughs• Moderated usability

testing• Card sorting

@amir_ansari @dius_au15

@amir_ansari @dius_au16www.r-66.info

Completion rate

Task time

Task level satisfactionConversions

Page viewsSystem Usability Scale (SUS)

Net Promoter Score (NPS)

@amir_ansari @dius_au17

Qualitative

Why?• Data is rich and deep• Explore and describe• Process-oriented• Small sample sizes

• Focus groups• Cognitive walkthroughs• Moderated usability

testing• Card sorting

@amir_ansari @dius_au18

How do we measure and

validate these? And When?

@amir_ansari @dius_au19

Visioning Iterations Iterations Iterations Iterations MVPLaunch

Inception & Planning

1 2 3

@amir_ansari @dius_au20

Technique 1

@amir_ansari @dius_au21

@amir_ansari @dius_au22

Visioning Iterations Iterations Iterations Iterations MVPLaunch

Inception & Planning

1 2 3

@amir_ansari @dius_au23

30 – 60 mins 3 One on one interviews

@amir_ansari @dius_au24

• Concerns• Fears• Pain points• Ideal experience• Ideal channel

OUTCOMES

@amir_ansari @dius_au25

• Some had not come prepared• Stakeholders wanted to ask

‘speculative questions’

@amir_ansari @dius_au26

• Stakeholders observing• Call centre Staff felt engaged

and included • More insights than we had

anticipated

@amir_ansari @dius_au27

Technique 2

@amir_ansari @dius_au28

@amir_ansari @dius_au29

Visioning Inception & Planning

1 2 3

Iterations Iterations Iterations Iterations MVPLaunch

@amir_ansari @dius_au30

Focus group 3

2 hours

Focus group 1

Focus group 224

@amir_ansari @dius_au31

@amir_ansari @dius_au32

OutcomesFeature 1 Feature 2 Feature 3

@amir_ansari @dius_au33

• 2 hours wasn’t quite enough• Running sessions back to back

in the same evening!

@amir_ansari @dius_au34

• Stakeholders buy-in through observation• Validate assumptions• Client was anonymous

@amir_ansari @dius_au35

Technique 3

@amir_ansari @dius_au36

Visioning Inception & Planning

1 2 3

Iterations Iterations Iterations Iterations MVPLaunch

@amir_ansari @dius_au37

1 hour One on one testing6

@amir_ansari @dius_au38

http://www.nngroup.com/articles/how-many-test-users/

@amir_ansari @dius_au39

@amir_ansari @dius_au40

@amir_ansari @dius_au41

• Refinements sometimes not subtle enough• Design ping pong

@amir_ansari @dius_au42

• Speed of testing• The feedback loop• Stakeholder buy-in

(live stream)• Rapid de-brief• One designs and one tests

@amir_ansari @dius_au43

1. Background

2. Falsifiable Hypotheses

3. Details

6. Next Actions

5. Validated Learning

4. Results

User 1 User 2 User 3 etc.

Hypothesis 1 Hypothesis 2 Etc.

• What feature or assumption to test• What questions to ask• How well does X perform• Etc. • What to test next

• What hypothesis do we want to validate

1. That majority of users think X2. That majority of users perform Y 3. That majority of users need Z4. Etc.

• Previous experiment taught us …• We want to find out…• We want to test our solutions…

1. We can proceed with X2. We need to tweak Y3. We need to pivot and get rid of Z

Title: Experiment 2: XXX Author: Created:

• From focus groups, it was clear that one barrier with other online providers has been the lack of trainer support

• That providing access to trainers in the classroom and telling students upfront about trainer availability will eliminate barriers and increase conversion

• Show users the marketing pages that talk about trainer availability

• Show users the classroom with the trainer being online

• Users indeed look for trainer availability and are comforted by knowing OTI provides access to them

• Test something else! ; )

@amir_ansari @dius_au44

bit.ly/rapid-debrief

@amir_ansari @dius_au45

@amir_ansari @dius_au46

@amir_ansari @dius_au47

@amir_ansari @dius_au48

@amir_ansari @dius_au49

@amir_ansari @dius_au50

5

One on one testingFocus groups

= $8002 FTEs

X $160X $375*

* Daily gross equivalent based on a gross annual salary of $90k

= $750

$1,550

16 = $3,2003 FTEs

X $200X $375*= $1,125

$4,325

If doing this internally…

@amir_ansari @dius_au51

@amir_ansari @dius_au52

Define why, what, when who, where how, how often

1

@amir_ansari @dius_au53

Become friends with the PM (or the person with the budget)

2

@amir_ansari @dius_au54

Keep it lean and measure often

3

@amir_ansari @dius_au55

Get stakeholder buy-in(Get them to observe)

4

@amir_ansari @dius_au56

Be consistentin how you measure and capture measurements

5

@amir_ansari @dius_au57

Define when to Quant and when to Qual

6

@amir_ansari @dius_au58http://dralexkent.files.wordpress.com/2011/12/gut.jpg

Replace with more powerful point• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea

@amir_ansari @dius_au59

Thank you

Qualitative data can in fact be converted into quantitative measures even if it doesn't come from an experiment or from a large sample size. 

Jeff Sauro – Measuring Usability“

@amir_ansari @dius_au60

Resources

@amir_ansari @dius_au61http://stamfordinteractive.com.au/conversations/benefits-of-guerrilla-testing/

@amir_ansari @dius_au62

You can quantify qualitative data!

bit.ly/quantifying-qualitative bit.ly/t-distribution

@amir_ansari @dius_au63

Completion rate

Task timeTask level satisfaction

Conversions

Page views

System Usability Scale (SUS)

Net Promoter Score (NPS)

{Visioning Inception & Planning Iterations Iterations Iterations Iterations MVP

Launch

@amir_ansari @dius_au64

https://www.measuringusability.com/topics/UX