lightning talk #12:7 cognitive biases we shouldn’t ignore in research by ruth ellison

The researcher’s blind spot: 7 cognitive biases we shouldn’t ignore in research

UX Singapore 2016 Ruth Ellison, Principal User Researcher, PwC’s Experience Centre @RuthEllison

@RuthEllison from PwC’s Digital Services

Hi :)

2

@RuthEllison

Free!

$10

$7

$20

@RuthEllison

$1

$10

$8

$20

5

Kiasu

Mr. Kiasu by Johnny Lau (Source: http://www.nlb.gov.sg/sure/wp-content/uploads/2013/10/Mr-Kiasu.jpg)

@RuthEllison from PwC’s Digital Services

The Zero Price Effect is a Cognitive Bias

@RuthEllison

What is a cognitive bias? Mental shortcut

@RuthEllison 8

We are all human Photo available under a CC by 2.0 licence https://www.flickr.com/photos/yellowbookltd/14907364551/

@RuthEllison Image source: http://www.businessinsider.com.au/what-new-coke-looked-like-in-1985-2013-3

@RuthEllison

Applying it to design research

Preparing Running

Analysing

@RuthEllison from PwC’s Digital Services

Preparing for research

@RuthEllison

@RuthEllison from PwC’s Digital Services

Selection bias

@RuthEllison 14

@RuthEllison from PwC’s Digital Services

Multiple channels - mix of recruitment companies, social media, trusted networks

Avoid professional respondents

Careful screeners

Behavioural based recruitment

Dealing

with it….

@RuthEllison from PwC’s Digital Services

Running research sessions

Photo available under a CC by 2.0 licence: https://www.flickr.com/photos/gdsteam/20649386153

@RuthEllison from PwC’s Digital Services

Observer-Expectancy effect

@RuthEllison 18

Source: https://medium.com/@userfocus/the-7-deadly-sins-of-user-research-22857c5a971b#.xb38ukdtr

@RuthEllison

The way we frame our questions matter

@RuthEllison

Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 1974, 13, 585-589.

McLeod, S. A. (2014). Loftus and Palmer. Retrieved from www.simplypsychology.org/loftus-palmer.html

About how fast were the cars going when they smashed into each

other?

@RuthEllison

About how fast were the cars going when they collided into each other?

About how fast were the cars going when they bumped into each other?

About how fast were the cars going when they contacted into each other?

About how fast were the cars going when they smashed into each other?

@RuthEllison from PwC’s Digital Services

Triangulate research

Use observational methods

Keeping positive-neutral body language, watch the tone of your voice

Avoid leading questions

Dealing

with it….

@RuthEllison 23

I need a volunteer please

24

2 4 8

PwC’s Digital Services

What’s going on?

@RuthEllison from PwC’s Digital Services

Confirmation bias

@RuthEllison 27

28

Take the opposing view

29

Don’t just confirm your hypothesis, see if you can prove it wrong

Analysis of Competing Hypotheses (ACH)

30

Use open ended questions

Some people think that soft drinks are bad for you. What do you think?

What’s your opinion about soft drinks?

@RuthEllison from PwC’s Digital Services

List assumptions

Be skeptical, especially if everyone agrees with you

Remain open

Consider all evidence equally

Multiple user researchers

Leave your ego by the door.

Dealing

with it….

@RuthEllison from PwC’s Digital Services

@RuthEllison from PwC’s Digital Services

Groupthink & the Bandwagon effect

@RuthEllison 33

@RuthEllison from PwC’s Digital Services

Triangulate with other research methods (e.g. observational)

Use a mixture of individual and group exercises

Avoid stating preferences and expectations at start

Give someone the devil’s advocate role to question assumptions

Dealing

with it….

@RuthEllison from PwC’s Digital Services

Anchoring bias

@RuthEllison 36

37 https://www.flickr.com/photos/naotakem/17105160150/

@RuthEllison from PwC’s Digital Services

Consider the order of questions and designs carefully

Open ended questions

Alternate order in which participants are shown concept or design versions

Show version A first 1 3 5

2 4 6 Show version B first

Dealing

with it….

@RuthEllison from PwC’s Digital Services

Analysing

@RuthEllison 40

@RuthEllison

Apophenia

41 Source: https://au.pinterest.com/pin/249738741807276910/

@RuthEllison from PwC’s Digital Services

Clustering illusion & reporting bias

@RuthEllison from PwC’s Digital Services

Sample sizes – it’s about the WHY

Consider evidence equally – not just the ones that confirm your belief/assumption

Collaborative analysis sessions

Strive for objectivity

Dealing

with it….

@RuthEllison 44

How vulnerable are you to these biases?

@RuthEllison from PwC’s Digital Services

What percentage of the population will be more biased

than the average person?

@RuthEllison from PwC’s Digital Services

Are you more biased than the average person?

@RuthEllison from PwC’s Digital Services

Blind spot bias

@RuthEllison from PwC’s Digital Services

Listen with an open mind

Become more rational, but less rationalising

Continuous learning

48

Always assess your method, your analysis and yourself for

bias

@RuthEllison 49

We are all human

@RuthEllison Image source: https://en.wikipedia.org/wiki/Twenty_Questions#/media/File:20_questions_1954.JPG

Thank you :)

Say hello @RuthEllison

@RuthEllison

Further reading

List of cognitive biases You are not so smart: a celebration of self delusion

9 Biases In Usability Testing

52