m hawley desirability studies boston upa presentation v4
DESCRIPTION
TRANSCRIPT
Prepared by:Michael Hawley – VP Experience DesignMegan Grocki – Senior Experience Designer
June 9, 2009
Rapid Desirability TestingANALYZING EMOTIONAL RESPONSE TO A DESIGN (ON A BUDGET)
boston upa
boston upa
Agenda
• Introduction
• The Situation
• Desirability Testing Overview
• Methods Considered
• Our Selected Process
• Case Study
• Lessons Learned
2
boston upa
The Situation
4
boston upa
The Situation
Visual Designs Applied to Wireframe
5
boston upa
The Situation
Visual Designs Applied to Wireframe
6
boston upa
Desirability Testing Overview
7
boston upa
What Is Desirability Testing?
A collection of research methods intended to assess target audience’s emotional response to a design or stimulus.
8
What It Is Not
• Measure of how much people “like” something
• Figuring out which is the “best”
What It Is
• Measure of how closely a stimulus achieves the “desired” emotional response
boston upa
Positioning Desirability Studies
9
http://www.xdstrategy.com/2008/10/28/desirability_studies/
boston upa
Why Is It Important
First impressions of a design to impact a product’s or application’s perceived utility, usability, and credibility.
10
Functionality
AestheticsUsability
boston upa
Methods Considered
11
boston upa
Triading
Definition
Present three different concepts or ideas to participants, and ask them to identify how two of them are different from the third and why.
12
boston upa
Quantitative Questionnaires
Definition
Broad, experience-based questionnaires, that also include questions relating to visual appeal and aesthetics
• SUS (System Usability Scale),
• QUIS (Questionnaire for User Interface Satisfaction)
• WAMMI (Website Analysis and Measurement Inventory)
13
boston upa
Show participants a user interface for a very brief moment, then take it away. Participants recall their first impression, then moderator probes for meaning
• Helpful for layout decisions, prominence of content, even labels
• www.fivesecondtest.com
Quick Exposure Memory Tests
14
Attention designers:
You have
50 milliseconds to make a good first impression
boston upa
Physiological and Neurological Measurements
Definition
• Sensors track participants’ physiological measurements to particular designs. Changes in suggest a particular emotional response.
• Paired with attitudinal and self-reporting surveys measurements give a multifaceted view of emotional reactions to a design• Electroencephalography (EEG): Brain activity
• Electromyography (EMG): Muscles and Excitement
• Electrodermal Activity (EDA): Sweat, Excitement
• Blood Volume Pressure (BVP): Arousal
• Pupil Dilation: Arousal and Mental Workload
• Respiration: Negative Valence or Arousal
15
boston upa
PrEmo Emotional Measurement
16
http://www.premo-online.comDr. Pieter Desmet,Technical University of Delft
boston upa
Product Reaction Cards (Our Selected Approach)
17
http://www.microsoft.com/usability/uepostings/desirabilitytoolkit.doc
boston upa
Product Reaction Cards Method
18
boston upa
Before You Begin
Determine intended brand attributes (and their opposites)
19
1. Leverage existing marketing/brand materials
2. Alternatively, stakeholder brainstorm to identify key brand attributes/descriptors using full list of product reaction cards as a start
3. Tip: “If the brand was a person, how would it speak to your customers?”
boston upa
Process - Conducting
Methodology
1. Include 60/40 split of positive and negative words
2. Target 60 words, optimized to test brand
3. Simple question: “Which of the following words do you feel best describe the site/design/product (please select 5):”
4. One comp per participant, or multiple comps per participant (no more than 3)
Participants
1. Qualitative: Paired with usability testing
2. Quantitative: Target minimum of 30 per option if possible
20
boston upa
Process - Analyzing
1. Calculate percentage of positive and negative attributes per design
2. Visualize overall sentiment of feedback using “word clouds” (see wordle.net)
21
68% Positive
32% Negative
Tip: Use word list spreadsheet available at http://www.userfocus.co.uk/articles/satisfaction.html
boston upa
Case Study: Greenwich Hospital Website Redesign
22
boston upa
Case Study: Greenwich Hospital Website Redesign
Background and Goals
• Align the website with the character of Greenwich Hospital
• “luxurious, approachable, friendly, capable, multi-cultural/inclusive, established”
• Update the site after nearly 10 years
• Counter impressions that Greenwich Hospital is more than just about maternity and elder care, without damaging those notions
• Communicate that they are long-standing members of the community
23
boston upa
Case Study: Greenwich Hospital Website Redesign
Methodology
• 3 visually designed comps
• 50 people reacted to each comp (quantitative) via survey
• Additional feedback obtained via participant interviews (qualitative)
24
Hello, I am requesting feedback on a website I am working on. Your answers let me know if the site is conveying the right feel.
1. What are your initial reactions to the web site?
2. Which of the following words best do you feel best describe the site (please select 5):
Survey Questions
boston upa
Three Different Visual Designs
25
boston upa
Results: Concept 1
26
88% Positive
12% Negative“My initial reaction to this web site is that it seems kind of plain. There is not much going on in the page, and the colors seem kind of drab.”
“This is a nice looking website. It is well designed, well laid out, and is appealing to look at. It makes me want to continue to navigate the site to learn more. “
boston upa
Results: Concept 2
27
87% Positive
13% Negative
“Men don`t really go with children… where`s a baby, there must be a mother. “
“My initial reaction to the website is that it seems very clean and modern. I like the layout, it looks like its easy to find information. “
boston upa
Results: Concept 3
28
95% Positive
5% Negative
“I felt love. I saw a mother holding a child.. that`s pretty touchy. The site looks good, and it makes the hospital trustworthy.”
“My initial reaction was that the Hospital is represented by a caring, warm and friendly website.”
boston upa
Lessons Learned
29
boston upa
Lessons Learned
Methodology
• Mix of qualitative and quantitative is key. Qualitative helps provide color to the results, quantitative resonates with stakeholders and executives
• Position results as one form of input to decision-making process, not declaring a “winner”
• Simple, cost-efficient way to assess audience’s emotional response to a design
30
boston upa
Key Take Aways
The Challenge:
• Measuring emotional responses to a design important, but complex. Experiences of a visual design are multifaceted, and a number of design aspects can impact their response to a product.
• There are a number of alternatives available to measure emotional response
Our Experience:
• Leveraging Product Reaction Cards provides a low-cost, low-effort means to help us align aesthetics and general feel with desired brand attributes
31
boston upa
Thank You
Documentation
Case Study results and full presentation slides:
http://www.madpow.net
32
Have a question?
Michael Hawley
@hawleymichael
Megan Grocki
@megUX
603-436-7177
boston upa
Additional Reading
Benedek, Joey and Trish Miner. “Measuring Desirability: New Methods for Evaluating Desirability in a Usability Lab Setting.” Proceedings of UPA 2002 Conference, Orlando, FL, July 8–12, 2002. http://www.microsoft.com/usability/uepostings/desirabilitytoolkit.doc
Lindgaard, Gitte, Gary Fernandes, Cathy Dudek, and J. Brown. "Attention Web Designers: You Have 50 Milliseconds to Make a Good First Impression!" Behaviour and Information Technology, 2006. http://www.imagescape.com/library/whitepapers/first-impression.pdf
Rohrer, Christian. “Desirability Studies: Measuring Aesthetic Response to Visual Designs.” xdStrategy.com, October 28, 2008. Retrieved February 10, 2010.http://www.xdstrategy.com/2008/10/28/desirability_studies
33
boston upa
Additional Reading
User Focus. "Measuring satisfaction: Beyond the Usability Questionnaire." Retrieved February 10, 2010. http://www.userfocus.co.uk/articles/satisfaction.html
UserEffect. "Guide to Low-Cost Usability Tools." Retrieved May 12, 2010.http://www.usereffect.com/topic/guide-to-low-cost-usability-tools
Tullis, Thomas and Jacqueline Stetson. “A Comparison of Questionnaires for Assessing Website Usability.” Usability Professionals’ Association Conference, 2004.home.comcast.net/~tomtullis/publications/UPA2004TullisStetson.pdf
Westerman, S. J., E. Sutherland, L. Robinson, H. Powell, and G. Tuck. “A Multi-method Approach to the Assessment of Web Page Designs.” Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction, 2007.http:// portal.acm.org/citation.cfm?id=1422200
34
boston upa
Additional Tools
Five Second Testhttp://fivesecondtest.com/
Feedback Army http://www.feedbackarmy.com
Wordlehttp://www.wordle.net
PrEmohttp://www.premo-online.com
35