toolkit support for usability evaluation 05-830 spring 2013 – karin tsai 1
TRANSCRIPT
![Page 1: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/1.jpg)
Toolkit Support for Usability Evaluation
05-830 Spring 2013 – Karin Tsai
1
![Page 2: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/2.jpg)
Overview
• Motivation• Definitions• Background from Literature• Examples of Modern Tools
2
![Page 3: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/3.jpg)
Motivation
• To improve or validate usability• Comparison between products, AB tests, etc.• Measuring progress• Verify adherence to guidelines or standards• Discover features of human cognition
3
![Page 4: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/4.jpg)
Usability Attributes
• Learnability – easy to learn• Efficiency – efficient to use• Memorability – easy to remember how to use• Errors – low error rate; easy to recover• Satisfaction – pleasant to use and likable
4
![Page 5: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/5.jpg)
Evaluation Categories
• Predictive– psychological modeling techniques– design reviews
• Observational– observations of users interacting with the system
• Participative– questionnaires– interviews– “think aloud” user-testing
5
![Page 6: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/6.jpg)
Challenges and Tradeoffs
• Quality vs. Quantity– “Quality” defined as abstraction, interpretability, etc.– User testing – high quality; low quantity– Counting mouse clicks – low quality; high quantity
• Observing Context • Abstraction
– Event reporting in applications places burden on developers
– Complicates software evolution
6
![Page 7: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/7.jpg)
• Evaluation Type: Predictive• Description: Uses a predictive human
performance model (“cognitive crash dummy”) to evaluate designs.
7
CogTool
![Page 8: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/8.jpg)
8
CogToolPros ConsFree Limited in “realisticness”Good for getting a baseline evaluation of prototypes
Quite confusing at first (extremely high learning curve)
Instantly accessible (not limited by participant availability or completion of the system’s functionality)
Documentation is “daunting”
Neat concept and insight into human cognition Limited usefulness
Overall Score: 6.5/10
![Page 9: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/9.jpg)
• Evaluation Type: Observational• Description: Aggregates developer-defined
event data in useful ways.
9
Mixpanel
![Page 10: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/10.jpg)
10
MixpanelPros ConsVery powerful built-in analysis tools High learning curveGood API for automated scripting Expensive
Scalable Application events = developer burden/maintainability issues
Flexible to fit needs of developers Rate-limited (one request at a time)
Overall Score: 9.5/10
![Page 11: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/11.jpg)
• Evaluation Type: Observational• Description: Real-time data visualization.
11
Chartbeat
![Page 12: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/12.jpg)
12
ChartbeatPros Cons
Data is real-time Does not scale well (financially) with huge sites
Captures data that is hard to obtain via events (reading, writing, idling, active time, referrals, social integration, etc.)
Limited in the data it captures (have to “hack” it if you want event-like data)
Great for site monitoring Only records “page-level” interactionsReally awesome visualization Limited historical data accessEasy to use Not built for usability evaluation
Overall Score: 7/10
![Page 13: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/13.jpg)
• Evaluation Type: Participative• Description: Watch a user complete a task on
your system while thinking aloud.
13
User Testing
![Page 14: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/14.jpg)
14
User TestingPros ConsProbably best method for catching usability issues Small sample size (hit or miss)
Most thorough recording of user interaction with the system Not easily scalable (expensive)
“Think aloud” allows data insights not otherwise attainable from just user interactions
Limited user availability
Can observe certain demographics without requesting personal information in the system itself
Sometimes, it’s painful to watch…
Overall Score: 8.5/10
![Page 15: Toolkit Support for Usability Evaluation 05-830 Spring 2013 – Karin Tsai 1](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649db15503460f94aa026f/html5/thumbnails/15.jpg)
15
Questions?