user interface evaluation usability inquiry methods zzj/usabilityhome.html
TRANSCRIPT
User Interface Evaluation
Usability Inquiry Methods
http://jthom.best.vwh.net/usability/http://www.cs.umd.edu/~zzj/UsabilityHome.html
Usability Inquiry Methods
Usability experts learn about the users’ likes, dislikes, needs, etc. of the system through: Observation Verbal questioning Written questioning
Widely used in practice.
Different methods have different costs, but in general, this is relatively cheap.
Usability Inquiry Methods Contextual Inquiry
Field Observation
Questionnaires
Interviews
Focus Groups
Logging Actual Use
Contextual Inquiry Contextual Inquiry is also known as proactive
field studies.
Applicable Stages: Requirements & Design
Personnel Usability Experts, approximately 1. Developers, 0. Users, 2.
Contextual Inquiry Usability Issues Covered
Effectiveness: No Efficiency: No Satisfaction: No
Quantitative Data is NOT collected.
Can NOT be conducted remotely.
Can be used on any system.
Contextual Inquiry What is it?
Before designing the system, the expert(s) visit the users’ workplace and question them.
This should occur before any design has been done.
Contextual Inquiry How can I do it?
Determine who your users are.
Go visit them where they work.
Contextual Inquiry How can I do it?
Talk to them about the system How do they currently do their job? How would you like to do your job? What do you like about the current system/method? What don’t you like about the current
system/method?
http://jthom.best.vwh.net/usability/context.htm http://www.cs.umd.edu/~zzj/FieldStu.htm
Field Observation Applicable Stages:
Test & Deployment
Personnel Usability Experts, approximately 1. Developers, 0. Users, 2.
Field Observation Usability Issues Covered
Effectiveness: Yes Efficiency: No Satisfaction: Yes
Quantitative Data is NOT collected.
Can NOT be conducted remotely.
Can be used on any system.
Field Observation What is it?
Usability experts observe users in the field using the system/product.
Field Observation How can I do it?
Go to the users’ workplace and simply observe.
Things to look for: What is the user’s mental model? Are the users using it the way you expect? You don’t want them to know you are evaluating
them.
http://www.cs.umd.edu/~zzj/FieldObs.htm
Questionnaires Applicable Stages:
Any stage of development (depending on the questions)
Personnel Usability Experts, approximately 1. Developers, 0. Users, 2.
Questionnaires Usability Issues Covered
Effectiveness: Yes Efficiency: No Satisfaction: Yes
Quantitative Data is NOT collected.
Can be conducted remotely.
Can be used on any system.
Questionnaires What is it?
Written lists of questions that you distribute to your users.
Questionnaires How can I do it?
Develop a list of questions on paper, web, email, etc. and give the questionnaire(s) to the users.
The users will answer the questions and return the questionnaires to you.
http://jthom.best.vwh.net/usability/question.htm http://www.acm.org/~perlman/question.html
Interviews Applicable Stages:
Any stage of development (depending on the questions)
Personnel Usability Experts, approximately 1. Developers, 0. Users, 1.
Interviews Usability Issues Covered
Effectiveness: Yes Efficiency: No Satisfaction: Yes
Quantitative Data is NOT collected.
Can be conducted remotely.
Can be used on any system.
Interviews What is it?
You interview users and ask them questions.
Interviews How can I do it?
Develop a list of questions for the users.
Meet with the users, individually.
Ask them the questions and log the responses Written and/or taped
Interviews How can I do it?
Interview Tips: Clearly define this is an interview. Ask open ended questions to get the user talking.
Yes-No questions are bad. Begin with less demanding topics and progress to
more difficult topics. Don’t ask questions to support your belief or
hypothesis. Do not answer your own questions. Do not agree or disagree … remain neutral.
Interviews How can I do it?
Probes: used to encourage the subjects to continue speaking, or to guide their response in a particular direction
Addition Probe Encourages more information or clarifies certain
responses from the test users. Either verbally or nonverbally the message is, "Go on, tell me more," or "Don't stop."
Interviews How can I do it?
Reflecting Probe Uses a nondirective technique, encourages the test
user to give more detailed information. The interviewer can reformulate the question or synthesize the previous response as a proposition.
Directive Probe Specifies the direction in which a continuation of
the reply should follow without suggesting any particular content. A directive probe may take the form of "Why is the (the case)?"
Interviews How can I do it?
Defining Probe Requires the subject to explain the meaning of a
particular term or concept.
http://jthom.best.vwh.net/usability/surveys.htm
http://www.cs.umd.edu/~zzj/Interview.htm
Focus Groups Applicable Stages:
Test & Deployment
Personnel Usability Experts, approximately 1. Developers, 0. Users, 6.
Focus Groups Usability Issues Covered
Effectiveness: Yes Efficiency: No Satisfaction: Yes
Quantitative Data is NOT collected.
Can be conducted remotely with email, bulletin boards, conference calls, etc.
Can be used on any system.
Focus Groups What is it?
A group of users are gathered to talk about the system.
The expert acts as the moderator.
Should conduct more than 1 focus group.
Focus Groups How can I do it?
Bring a group of users together and begin.
Collect data
http://www.cs.umd.edu/~zzj/FocusGro.htm
Logging Actual Use Applicable Stages:
Test & Deployment
Personnel Usability Experts, approximately 1. Developers, 0. Users, 6.
Logging Actual Use Usability Issues Covered
Effectiveness: Yes Efficiency: Yes Satisfaction: Maybe
Quantitative Data is NOT collected, in most cases.
Can be conducted remotely.
Can be used on most systems.
Logging Actual Use What is it?
The computer automatically collect usage data.
You could ask the user to log their usage, but that’s not practical.
Logging Actual Use How can I do it?
Usability Logger http://www.usabletools.com/
Automatic capture of keyboard, mouse, etc.
VideoCam and other products.
Logging Actual Use Facts On Logging Actual Use
You know exactly what the user is doing. You don’t know why, but you do know what, when,
where. You don’t know how the user feels.
http://www.cs.umd.edu/~zzj/Logging.htm
Usability Inquiry Methodsmost useful
Contextual Inquiry
Field Observation
Questionnaires
Interviews
Focus Groups (least useful)
Logging Actual Use