ux india 2016 workshop: research methods to measure design impact - techniques for rapid decision...
TRANSCRIPT
![Page 1: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/1.jpg)
Research Methods to Measure Design Impact:
Techniques for rapid decision- making
Steve Fadden, Ph.D.Director, Analytics UX Research,
SalesforceLecturer, School of Information, UC
Berkeley
Workshop delivered at UX India 2016 - Impact by Design, October 22, 2016, Hyderabad, India
![Page 2: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/2.jpg)
Agenda
1. Introductions
2. Research process
3. Methods
4. Activity
5. Debrief & Discuss
![Page 3: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/3.jpg)
Introduction
![Page 4: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/4.jpg)
About me (@sfadden)
![Page 5: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/5.jpg)
About you
Focus:
▸ Design▸ Research▸ Engineering▸ Marketing▸ Management▸ Other?
Experience:
▸ >5 years▸ 2-5 years▸ <2 years
Image: https://pixabay.com/en/people-group-crowd-team-isolated-309099/
![Page 6: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/6.jpg)
Process
![Page 7: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/7.jpg)
Generative Formative Summative
Research process
Images: https://commons.wikimedia.org/wiki/File:Light_bulb_(yellow)_icon.svg ; https://commons.wikimedia.org/wiki/File:Korean_Traffic_sign_(Left_Turn_and_Right_Turn).svg ;https://pixabay.com/en/chart-line-line-chart-diagram-trend-148256/
![Page 8: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/8.jpg)
Formative methods inform progress
▸ Developing ideas, stories, flows, designs
▸ Any level of fidelity
▸ Focused on improving
▸ Informs decision-making
▸ “Lightweight” techniques → faster decisions
Image: https://commons.wikimedia.org/wiki/File:Korean_Traffic_sign_(Left_Turn_and_Right_Turn).svg
![Page 9: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/9.jpg)
Designing involves lots of decisions
Images: http://www.ichanical.com/wp-content/uploads/2015/08/agile-dev.png ; https://crowdsourcedtesting.com/resources/wp-content/uploads/2016/07/agile-methodolody_695x260.jpg ; http://www.softwaretestingandistqb.com/wp-content/uploads/2015/06/Waterfall-Development-Model.jpg
![Page 10: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/10.jpg)
“Fail faster to succeed sooner.”
David Kelley, IDEO
Quote / Image: https://ssir.org/articles/entry/fail_faster_succeed_sooner ; https://churchill.imgix.net/images/profiles/Kelley_David.jpg?auto=format&h=512&w=512&q=30
![Page 11: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/11.jpg)
Methods
![Page 12: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/12.jpg)
Understanding problems
![Page 13: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/13.jpg)
Background
Image: https://en.wikipedia.org/wiki/Lockheed_L-1649_Starliner
![Page 14: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/14.jpg)
Ask Critical Incident questions
Gain insights:
▸ Evidence of problems▸ Opportunities to solve
Based on:
▸ Recent events▸ Specific details▸ Feelings and perceptions▸ Future behaviors and responses
![Page 15: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/15.jpg)
Critical Incident process
1. Identify time since last experience
2. Gather details:a. Descriptionb. Actions takenc. Feelingsd. Outcomee. Future actions/responses desired
Reference: http://www.usabilitynet.org/tools/criticalincidents.htm
![Page 16: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/16.jpg)
Example prompt
“Consider the last time you had to share something online. How long ago did this happen? What did you share? Describe the steps you took to share, and highlight any surprises or problems (if any) that happened.”
![Page 17: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/17.jpg)
Example result
▸ Validates problem
▸ Identifies opportunities
▸ Clarifies expectations
▸ Details scenarios
▸ Builds empathy
“I needed to share a PDF with a friend, and we use Dropbox, but I hadn’t used it in a while. I logged in through my browser, dragged the PDF to Files, and then saw the PDF open. I expected Dropbox to start uploading. I hit back, created a folder in Dropbox, uploaded the PDF to it, clicked share to add my friend, and sent her the link.”
![Page 18: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/18.jpg)
Investigating concepts
![Page 19: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/19.jpg)
Background
Image / Reference: Rosson, M.B., & Carroll, J.M. (2002). Usability Engineering: Scenario-Based Development of Human-Computer Interaction. Academic Press; Carroll, J.M. (1999). Five Reasons for Scenario-Based Design. In: Proceedings of the 32nd Hawaii International Conference on System Sciences.
![Page 20: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/20.jpg)
Concept evaluation with commenting
Gain insights:
▸ Initial confusion▸ Acceptability of process
Based on:
▸ Description of goal and flow▸ Illustration of process: storyboard, steps,
video
![Page 21: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/21.jpg)
Commenting process
1. Present overall scenarioa. Ensure understandingb. Capture open concerns/questions
2. Show steps of flow (or interaction), capturing feedback about:a. Concerns, confusionsb. Benefits, positivesc. Open questions
3. Capture final comments at end
![Page 22: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/22.jpg)
Example prompt
“Consider the data export concept presented on the next 4 slides. As you read through the concept, comment on anything you find to be confusing, problematic, useful, or appealing about the concept.”
![Page 23: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/23.jpg)
Example flow (comments gathered after each slide presented)
1.
2.
3.
4.
100%
![Page 24: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/24.jpg)
Example feedback, slide 1
1.
“Makes sense so far. No comments yet.”
![Page 25: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/25.jpg)
2.
Example feedback, slide 2
“Doing this would require a lot of clicks, even for a small number of columns.”
![Page 26: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/26.jpg)
3.
Example feedback, slide 3
“You should embed best practices for naming here. Otherwise, the result could be messy.”
![Page 27: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/27.jpg)
4.
100%
Example feedback, slide 4
“Will we be able to save the mappings? That could save time in the future.”
![Page 28: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/28.jpg)
Example: Final comments
1.
2.
3.
4.
100%
“It’s great that you don’t have to jump around different parts of the system to do this. Very valuable to be able to complete this from one place.”
“Hi, I wanted to follow up to reiterate that this is a REALLY COOL idea and it fills a much needed requirement for our use of the product. Please consider me for future studies like this, because we need this functionality!”
![Page 29: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/29.jpg)
Prioritizing needs
![Page 30: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/30.jpg)
Kano model
Absent Present
Dissatisfaction
Satisfaction
Reference: http://uxmag.com/articles/leveraging-the-kano-model-for-optimal-results
![Page 31: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/31.jpg)
Kano model
Absent Present
Dissatisfaction
Satisfaction
Performance
(more is better)Basic
(required)
Exciting(truly unexpected)
![Page 32: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/32.jpg)
Kano model (automobile)
Absent Present
Dissatisfaction
Satisfaction
Seat-warmersBluetooth
Intuitive GPS
Horsepower
Fuel economyAirbags
Adjustable seats
![Page 33: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/33.jpg)
Kano question process
1. Assemble list of features
2. Pose realistic scenarios for each
3. For each question, assess:a. Satisfaction if availableb. Satisfaction if not availablec. Importance of feature
Scenario: “Imagine you are using a mobile app to find a restaurant.”
Question 1: “How would you feel if the app allowed you to filter results by distance?”
Question 2: “How would you feel if the app did not allow you to filter results by distance?”
Question 3: “How important is this function to you?”
![Page 34: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/34.jpg)
Kano results
:( if present :) if present
:) if absent
:( if absent
Map answers to “product allowed you to...” questions
Map answers to “product did not allow you to...” questions
![Page 35: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/35.jpg)
Kano results
:( if present :) if present
:) if absent
:( if absent
![Page 36: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/36.jpg)
Kano results
:( if present :) if present
:) if absent
:( if absent
“We should explore and reconsider these...”
![Page 37: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/37.jpg)
Kano results
:( if present :) if present
:) if absent
:( if absent
“Do we emphasize what people want (X-axis), or what they expect (Y-axis)?”
![Page 38: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/38.jpg)
Activity (part 1)
![Page 39: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/39.jpg)
“A leading transportation provider has announced a competition to design a product to improve passenger engagement with the community.
Work with your team to consider a concept.
![Page 40: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/40.jpg)
Activity: Improving passenger engagement with community
Process
1. Discuss possible concepts
2. Plan questions to understand needs/problems
3. Conduct interview on non-team member
Scenario
A leading transportation provider has announced a competition to design a product to improve passenger engagement with the community. Work with your team to consider a concept and make decisions to improve it.
![Page 41: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/41.jpg)
Identifying Expectations
![Page 42: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/42.jpg)
Expectation testing
Gain insights:
▸ Mental model▸ Categories and labels▸ Areas of confusion
Based on:
▸ Nonworking interface▸ No text/labels▸ Basic task information
Reference: Thomas S. Tullis, 1998. A method for evaluating web page design concepts. CHI 98 conference summary on human factors in computing systems
![Page 43: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/43.jpg)
“Greeking” technique
1. Identify important task
2. Write scenario
3. Write “first step” question
4. Gather feedback
5. Ask for group/category names
![Page 44: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/44.jpg)
Example
Scenario: “You are exploring colleges, and need to make sure the courses look interesting and financial support is available.”
Instruction: “Indicate where you would click to access a list of courses?”
![Page 45: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/45.jpg)
Try it!
Where would you click to
1. ...access a list of courses?
2. ...determine if financial support is available?
3. ...contact the department in charge of Athletics?
What would you call each group?
![Page 46: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/46.jpg)
Sample results
Where would you click to
1. ...access a list of courses?
2. ...determine if financial support is available?
3. ...contact the department in charge of Athletics?
What would you call each group?
12
3
“Academics” “Administrative”
“Connections”
![Page 47: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/47.jpg)
Uncovering impressions
![Page 48: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/48.jpg)
First impressions matter
Impressions are formed by 50ms
▸ Primarily based on visual appeal▸ Do not change with additional viewing time▸ Are not influenced by actual usability issues
5-second test reveals usability impression
1. Show interface for 5 seconds2. Gather feedback on appeal and usability
Reference: Phillips, C. & Chaparro, B.S. (2009). Visual Appeal vs. Usability: Which One Influences User Perceptions of a Website More? http://usabilitynews.org/visual-appeal-vs-usability-which-one-influences-user-perceptions-of-a-website-more/
![Page 49: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/49.jpg)
Initial prompt
“You will be shown an interface for 5 seconds. After viewing the interface, indicate your response to the following questions.”
![Page 50: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/50.jpg)
Prompt after viewing
Mark how you feel about the interface you just saw. The interface is:
Very VeryAttractive - - - - - - - - - Unattractive
Very VeryEasy - - - - - - - - - Hard
Very VeryEfficient - - - - - - - - - Inefficient
![Page 51: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/51.jpg)
Responses found to correlate with System Usability Scale
Strongly Strongly Agree Disagree
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.3. I thought the system was easy to use.4. I think that I would need the support of a
technical person to be able to use this system.5. I found the various functions in this system were
well integrated.6. I thought there was too much inconsistency in this
system.7. I would imagine that most people would learn to use
this system very quickly.8. I found the system very cumbersome to use.9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.
![Page 52: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/52.jpg)
Assessing sentiment
![Page 53: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/53.jpg)
Use semantic differential scales to quickly assess sentiment
Gain insights:
▸ How people feel about your concept, interface, experience
▸ Alignment with intended goals
Based on:
▸ Exposure to concept, interface, or task▸ Marking experience on 5-point scale
Reference: Source: Johnson, F. 2012. Using semantic differentials for an evaluative view of the search engine as an interactive system, EuroHCIR2012.
![Page 54: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/54.jpg)
Semantic differential process
1. Determine goal: a. Granularity: Overall concept, interface,
task performance
b. Criteria: utility, efficiency, satisfaction
2. Administer as survey or interview prompt
3. Analyze based on participant characteristics, interface differences, tasks
![Page 55: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/55.jpg)
Sample prompt and responses
“Mark how you feel about this experience.”
Powerful - - - - - SimplisticAttractive - - - - - UnattractiveValuable - - - - - Not valuableRelevant - - - - - IrrelevantSatisfying - - - - - FrustratingFast - - - - - SlowPredictable - - - - - UnpredictableIntuitive - - - - - RigidEasy - - - - - Difficult
![Page 56: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/56.jpg)
Example result from interface A
“I selected these values because I really like how the interface looks, but I’m not sure what to do with it. What would I ever use this for?”
Utility
Efficiency
Control
Affect
Appearance
![Page 57: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/57.jpg)
Example result from interface B
“Even though the interface looks really plain and boring, I like that I have complete control when doing this task, and it’s a lot faster than how I need to accomplish this today.”
Utility
Efficiency
Control
Affect
Appearance
![Page 58: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/58.jpg)
Combined results (A & B)
Utility
Efficiency
Control
Affect
Appearance
“Utility and Appearance are both critical for this product -- we need to design for both!”
![Page 59: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/59.jpg)
Evaluating usability
![Page 60: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/60.jpg)
Usability evaluation
Goals
▸ Explore concepts▸ Assess effectiveness▸ Compare competitors
Measures
▸ Learnability▸ Efficiency▸ Effectiveness (Errors)▸ Satisfaction
Process
1. Plan (goals, participant criteria)
2. Prepare (recruit, test materials)
3. Conduct (active observers)
4. Analyze (issues, insights)
5. Report (fixes, future opportunities)
![Page 61: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/61.jpg)
Usability evaluation
Rapid Iterative Testing & Evaluation (RITE) - identify and fix problems
Discount usability - identify and prioritize problems
Traditional testing - benchmark, summarize performance
Aim to:
● Test representative users● Follow script with actual tasks● Use high-fidelity interface
Key elements:
● Observation by key decision makers (PM, design, development)
● Agree on critical problems and make changes to interface after each session
Reference: Medlock, M.C., Wixon, D., Terrano, M., Romero, R., & Fulton, B. (2002). Using the RITE method to improve products; a definition and a case study, Presented at the Usability Professionals Association.
![Page 62: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/62.jpg)
Example result
● RITE with 4 participants over 2 days
● Key changes: Terminology, embedded instructions, icons
● Stakeholder gratitude and desire for more
“I haven’t selected the columns yet! Why can’t I go back?”
“It says ‘save’ but I’m just modifying the settings. Unless I can re-use this later?”
![Page 63: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/63.jpg)
Activity (part 2)
![Page 64: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/64.jpg)
“A leading transportation provider has announced a competition to design a product to improve passenger engagement with the community.
Work with your team to refine concept.
![Page 65: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/65.jpg)
Activity: Improving passenger engagement with community
Process
1. Sketch a flow or interface
2. Plan methods to:a. Refine conceptb. Refine interface
3. Try one!
▸ Problem: Critical incident questions
▸ Concept: Commenting▸ Expectation: Greeking▸ Impression: 5s test▸ Sentiment: Semantic
differential▸ Prioritization: Kano
questions▸ Usability: RITE
![Page 66: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/66.jpg)
Debrief
![Page 67: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/67.jpg)
Report-out
1. Your concept2. Research plan3. Refinements you made4. Lesson(s) learned
![Page 68: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/68.jpg)
Tips & Tricks
![Page 69: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/69.jpg)
Recruit participants for the research purpose
Image: https://pixabay.com/en/think-about-salaried-worker-1184858/
![Page 70: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/70.jpg)
Order research activities intentionally and don’t overdo it
Image: https://pixabay.com/en/photos/aim/
![Page 71: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/71.jpg)
Pilot with each other
Image: https://pixabay.com/en/men-team-women-blue-craftsmen-158690/
![Page 72: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/72.jpg)
Triangulate, triangulate, triangulate
Image: https://commons.wikimedia.org/wiki/File:3_triangles.svg
![Page 73: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/73.jpg)
Others?
Questions?
Image: https://pixabay.com/en/hand-fingers-silhouette-human-294018/
![Page 74: UX India 2016 workshop: Research methods to measure design impact - techniques for rapid decision making](https://reader031.vdocuments.site/reader031/viewer/2022022414/5871fbcf1a28ab5c348b798d/html5/thumbnails/74.jpg)
Thank you!
@sfadden on Twitter
slideshare.net/stevefadden1