UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

Download UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

Post on 10-May-2015

557 views

Category:

Technology

4 download

TRANSCRIPT

  • 1.UX Assessment TechniquesJen Romano Bergstrom December 11, 2013 NOVA UX @forsmarshgroup @romanocog

2. Measuring the UX the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. ISO 9241-11 + emotions@forsmarshgroup @romanocog2 How does it work for the end user? What does the user expect? How does it make the user feel? 3. Why is it important? Put it in the hands of the end user. Things may seem straightforward to you but maybe not to your users.@forsmarshgroup @romanocog3 4. Why is it important? Put it in the hands of the end user. Things may seem straightforward to you but maybe not to your users.@forsmarshgroup @romanocog4 5. Why is it important? Put it in the hands of the end user. Things may seem straightforward to you but maybe not to your users.@forsmarshgroup @romanocog5 6. Where to test LABORATORY Controlled environmentAll participants have the same experienceRecord and communicate from control roomObservers watch from control room and provide additional probes (via moderator) in real timeIncorporate physiological measures (e.g., eye tracking, EDA) No travel costs@forsmarshgroup @romanocog6REMOTE Participants in their natural environments (e.g., home, work)Use video chat (moderated sessions) or online programs (unmoderated)IN THE FIELD Participants tend to be more comfortable in their natural environmentsRecruit hard-to-reach populations (e.g., children, doctors)Conduct many sessions quicklyModerator travels to various locationsRecruit participants in many locations (e.g., states, countries)Bring equipment (e.g., eye tracker)Natural observations 7. Where to test LABORATORY Controlled environmentAll participants have the same experienceRecord and communicate from control roomObservers watch from control room and provide additional probes (via moderator) in real timeIncorporate physiological measures (e.g., eye tracking, EDA) No travel costs@forsmarshgroup @romanocog7REMOTE Participants in their natural environments (e.g., home, work)Use video chat (moderated sessions) or online programs (unmoderated)IN THE FIELD Participants tend to be more comfortable in their natural environmentsRecruit hard-to-reach populations (e.g., children, doctors)Conduct many sessions quicklyModerator travels to various locationsRecruit participants in many locations (e.g., states, countries)Bring equipment (e.g., eye tracker)Natural observations 8. How to test ONE-ON-ONE SESSIONS In-depth feedback from each participantFOCUS GROUPS No group think Can allow participants to take their own route and explore freelyParticipants may be more comfortable with othersInterview many people quicklyOpinions collideNo interferencePeer reviewRemote in participants environmentQualitativeFlexible schedulingQualitative and Quantitative@forsmarshgroup @romanocog8SURVEYS RepresentativeLarge sample sizesCollect a lot of data quicklyNo interviewer biasNo scheduling sessionsQuantitative analysis 9. When to test@forsmarshgroup @romanocog9 10. What to measure EXPLICITOBSERVATIONAL+ Post-task satisfaction questionnaires + In-session difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial+ Ethnography + Time to complete task + Reaction time + Selection/click behavior + Ability to complete tasks + AccuracyIMPLICIT + Facial expression analysis + Eye tracking + Electrodermal activity (EDA) + Behavioral analysis + Linguistic analysis of verbalizations + Implicit associations + Pupil dilation @forsmarshgroup @romanocog10 11. Case 1 Problems: What do users want? Does the new design work? Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking@forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started.11 12. Case 1 Problems: What do users want? Does the new design work? Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking@forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started.12 13. Case 2 Problem: What parts of the form do people actually read? Method: In-lab usability testing with eye tracking@forsmarshgroup @romanocog13 14. Case 2 Participants did not read the instructions in their entirety (page 1: left; page 3: right); rather, they skimmed and then moved on to the form where they needed to enter information.120Time (seconds)100 80 60 40 20Aggregate fixation count heat map across all participants, Page 1. Participants looked at Purpose of Form section the most often.@forsmarshgroup @romanocog140 Page 1Page 2Page 3Length of time spent on each page of the instructions before working on form. 15. Case 3 Prototype Grid@forsmarshgroup @romanocog15Old Grid 16. Case 4 Gaze Plot: After getting an error message, the participant had to search all over the screen to find the missing field.SmartphoneHow do I advance to the next screen? It seems like it's stuck on the screen. @forsmarshgroup @romanocog16Tablet 17. Many ways to evaluate the UX Surveys Focus groups In-person one-on-one with eye tracking Analytics@forsmarshgroup @romanocog17 18. Obstacles to UX Testing There is no time. Start early in development process One morning a month with 3 users Krug 12 people in 3 days Anderson Reimer 12 people in 2 days Lebson & Romano Bergstrom Cant find representative users Everyone is important Travel Remote testing We dont have a lab Test anywhere @forsmarshgroup @romanocog18 19. Thank you! Twitter: @forsmarshgroup LinkedIn: http://www.linkedin.com/company/fors-marsh-group Blog: www.forsmarshgroup.com/index.php/blogJennifer Romano Bergstrom @romanocog jbergstrom@forsmarshgroup.com NOVA UX