building on the usability study: two explorations on how to better understand an interface
DESCRIPTION
In this presentation, we describe two separate studies that improved our ability to understand our users’ experience of our products at salesforce.com. The first study explored a methodology of combining expert and novice performance data to yield a measure of intuitiveness. The second study created a methodology that combines both verbal and nonverbal emotion scales to better understand the emotional effect our products have on our users. We present both these methods as expansions on the standard usability study and examples of ways to better understand your users within an industry environment.TRANSCRIPT
![Page 1: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/1.jpg)
Building on the !Usability Study !
Two Explorations on How to!Better Understand an Interface
Anshu Agarwal Madhu Prabaker
Usability Analysts
HCI International 2009 San Diego, California
![Page 2: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/2.jpg)
UserExperience
TraditionalUsabilityMeasure
![Page 3: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/3.jpg)
???
UserExperience
TraditionalUsabilityMeasure MustbeValuable
&Practical
![Page 4: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/4.jpg)
UserExperience
TraditionalUsabilityMeasure Intuitiveness
![Page 5: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/5.jpg)
UserExperience
TraditionalUsabilityMeasure Intuitiveness
Emotion
![Page 6: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/6.jpg)
UserExperience
TraditionalUsabilityMeasure Intuitiveness
Emotion
![Page 7: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/7.jpg)
Measuring Intuitiveness
![Page 8: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/8.jpg)
What does it mean to be Intuitive?
“Easytouseandunderstand”
“Atechnicalsystemis,inthecontextofacertaintask,intuitivelyusablewhiletheparticularuserisabletointeracteffectively,notconsciouslyusing
previousknowledge”‐[Naumannetal]
![Page 9: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/9.jpg)
Project Goals"Measuring Intuitiveness
UserExperience
TraditionalUsabilityMeasure Intuitiveness
![Page 10: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/10.jpg)
Measuring Novice Performance on an Interface
![Page 11: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/11.jpg)
Measuring Expert Performance on an Interface
![Page 12: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/12.jpg)
Combining Expert and Novice Performance
“…interacteffectively,notconsciouslyusingpreviousknowledge.”
![Page 13: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/13.jpg)
Combining Expert and Novice Performance
Anintuitiveinterfacecanbethoughtofasonethat“achievesnovicetaskperformanceascloseaspossibletoexperttaskperformance”
![Page 14: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/14.jpg)
Combining Expert and Novice Performance
![Page 15: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/15.jpg)
Combining Expert and Novice Performance
![Page 16: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/16.jpg)
Measuring Emotion
![Page 17: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/17.jpg)
Project Goal"Understanding Emotional Impact
UserExperience
TraditionalUsabilityMeasure
Emotion
![Page 18: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/18.jpg)
Existing Ways to Measure Emotion"The Common Tools
Physiological
Verbal
Non‐Verbal
![Page 19: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/19.jpg)
Our Approach"Combine the Best from Both Worlds
Physiological
Verbal
Non‐Verbal
![Page 20: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/20.jpg)
Our Verbal Measure"PAD Semantic Differential Scale
Pleasure,Arousal,andDominance(PAD)
SemanticDifferentialScale
[Mehrabian&Russell]
PADDimension BipolarPairs
Pleasure
Annoyed‐Pleased
Unsatisfied‐Satisfied
Despairing‐Hopefully
Tense‐Relaxed
Friendly‐Unfriendly
Arousal
Relaxed‐Stimulated
Calm‐Excited
Sleepy–WideAwake
Unaroused‐Aroused
Dominance
Controlled‐Controlling
Influenced‐Influential
Submissive‐Dominant
Guided‐Autonomous
![Page 21: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/21.jpg)
Our Non-Verbal Measure"Emo-Card
ExcitedPleasant
CalmPleasantCalmUnpleasant
ExcitedUnpleasant
‐Pleasure
‐Arousal
+Pleasure
+Arousal
![Page 22: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/22.jpg)
Validating the Technique"Empirical Study Design
22Participants(13male,9female)
InterfaceA InterfaceB
7Tasks(Randomized)
StandardMeasures EmotionalMeasures
TimeonTask Emocard
#ofErrors PADScale
Post‐TaskInterview
![Page 23: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/23.jpg)
Validating the Technique"Empirical Study Results
Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B
InterfaceA InterfaceB
PAD Scale and Emocard showed some clear differences in how users experienced the interfaces
![Page 24: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/24.jpg)
Validating the Technique"Empirical Study Results
Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B
InterfaceA InterfaceB
PAD Scale and Emocard showed some clear differences in how users experienced the CRM Systems
![Page 25: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/25.jpg)
Validating the Technique"Empirical Study Results
Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B
InterfaceA InterfaceB
PAD Scale and Emocard showed some clear differences in how users experienced the interfaces
![Page 26: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/26.jpg)
Conclusion
UserExperience
TraditionalUsabilityMeasure Intuitiveness
Emotion
![Page 27: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/27.jpg)
Conclusion
UserExperience
TraditionalUsabilityMeasure
???Whatareyoumissingthat’s
Valuable&Easy
![Page 28: Building on the Usability Study: Two Explorations on How to Better Understand an Interface](https://reader034.vdocuments.site/reader034/viewer/2022051610/548710dfb47959ec0c8b5393/html5/thumbnails/28.jpg)
Agarwal, A. and Meyer, A. 2009. Beyond usability: evaluating emotional response as an integral part of the user experience. In Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009).
Anshu Agarwal [email protected]
Madhu Prabaker [email protected]