tips and tricks for measuring the user experience
TRANSCRIPT
![Page 1: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/1.jpg)
Tips and Tricks for Measuring the User Experience
Tom Tullis and Bill AlbertFidelity Investments
Usability and User Experience 2008
UPA-Boston's Seventh Annual Mini UPA Conference
May 28, 2008
![Page 2: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/2.jpg)
2
Who we are
Tom TullisSenior Vice President, User Insight, Fidelity [email protected]
Bill AlbertDirector, User Insight, Fidelity [email protected]
![Page 3: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/3.jpg)
3
Our Top 10 List
1. Know your data 2. Show your confidence (intervals)3. Deal with binary success data (appropriately)4. Compare means 5. Consider using expectation measures6. Use the System Usability Scale (SUS)7. Show frequency distributions8. Combine different metrics9. Use appropriate tools 10.Present data appropriately
![Page 4: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/4.jpg)
4
Tip 1: Know your data(and what you can & can’t do with it)
![Page 5: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/5.jpg)
5
Independent & Dependent Variables
Independent variables: The things you manipulate or control for, e.g.,
Characteristics of participants (e.g., age, sex, relevant experience)Different designs or prototypes being testedTasks
Dependent variables: The things you measure, e.g.,Task SuccessTask TimeSUS scoreetc.
![Page 6: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/6.jpg)
6
Types of Data
Nominal (aka Categorical)e.g., Male, Female; Design A, Design B.
Ordinale.g., Rank ordering of 4 designs tested from Most Visually Appealing to Least Visually Appealing.
Intervale.g., 7-point scale of agreement: “This design is visually appealing. Strongly Disagree . . . Strongly Agree”
Ratioe.g., Time, Task Success %
![Page 7: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/7.jpg)
7
Ordinal vs. Interval Rating Scales
Are these two scales different?
Top scale is ordinal. You should only calculate frequencies of each response.
Bottom scale can be considered interval. You can also calculate means.
![Page 8: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/8.jpg)
8
Statistics for each Data Type
![Page 9: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/9.jpg)
9
Tip 2: Show your confidence (intervals)
![Page 10: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/10.jpg)
10
Confidence Intervals
Assume this was your time data for a study with 5 participants:
Task Time (secs)P1 43P2 53P3 59P4 68P5 49
Does that make a difference in your answer?
![Page 11: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/11.jpg)
11
Calculating Confidence Intervals
Task Time (secs)P1 43P2 53P3 59P4 68P5 49Mean: 54.4Std Dev: 9.6N: 5
95% Confidence Interval:8.4
=CONFIDENCE(<alpha>,<std dev>,<n>)
<alpha> is normally .05 (for a 95% confidence interval)
<std dev> is the standard deviation of the set of numbers (9.6 in this example)
<n> is how many numbers are in the set (5 in this example)
Excel example
![Page 12: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/12.jpg)
12
Show Error Bars
Mean SUS Scores: Apollo Program Websites(Error bars represent 95% confidence interval)
0
10
20
30
40
50
60
70
80
NASA Wikipedia
SUS
Scor
e
Excel example
![Page 13: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/13.jpg)
13
How to Show Error Bars
![Page 14: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/14.jpg)
14
Tip 3: Deal with Binary Success Data (appropriately)
![Page 15: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/15.jpg)
15
Binary Success
Pass/fail (or other binary criteria)1’s (success) and 0’s (failure)
Participant Task 1 Task 2 Task 3 Task 4 Task 5P1 1 0 1 0 0P2 1 0 1 0 1P3 1 1 1 1 1P4 1 1 1 1 1P5 0 0 1 1 1P6 1 0 0 1 1P7 0 1 1 1 1P8 0 0 1 1 0P9 1 0 1 0 1P10 1 1 1 1 1P11 0 1 1 1 1P12 1 0 1 1 1Average 67% 42% 92% 75% 83%Confidence Interval (95%) 28% 22% 29% 29% 29%
![Page 16: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/16.jpg)
16
Confidence Interval for Task Success
When you look at task success data across participants for a single task the data is commonly binary:
Each participant either passed or failed on the task.
In this situation, you need to calculate the confidence interval using the binomial distribution.
![Page 17: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/17.jpg)
17
ExampleTask 1
P1 1P2 1P3 1P4 0P5 1P6 1P7 0P8 1
1=success, 0=failure.So, 6/8 succeeded, or 75%.
Easiest way to calculate confidence interval is using Jeff Sauro’s web calculator: http://www.measuringusability.com/wald.htm
![Page 18: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/18.jpg)
18
Chi-square
Allows you to compare actual and expected frequencies for categorical data.
=CHITEST(<actual range>,<expected range>)
Excel example:
![Page 19: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/19.jpg)
19
Tip 4: Compare Means
![Page 20: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/20.jpg)
20
Comparing Means
T-test:Independent samples (between subjects)Paired samples (within subjects)
Excel example:
Apollo websites, task times
Excel example:
Haptic mouse study
![Page 21: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/21.jpg)
21
T-tests in Excel
Independent Samples: Paired Samples:
=TTEST(<array1>,<array2>,x,y)
y = 2 (independent samples) y = 1 (paired samples)
x = 2 (for two-tailed test) in almost all cases
![Page 22: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/22.jpg)
22
Comparing Multiple Means
Analysis of Variance (ANOVA)
“Tools” > “Data Analysis” > “Anova: Single Factor”Excel example: Study comparing 4 navigation approaches for a website
![Page 23: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/23.jpg)
23
Tip 5: Consider Using Expectation Measures
![Page 24: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/24.jpg)
24
Expectation Measure
Based on “expectancy disconfirmation” theory from Market ResearchBasic approach:
Ask the users how easy/difficult they think each task is going to be, before doing them.Ask them how easy/difficult it actually wasimmediately after each task.Same 7-point scales in both cases:
Very Difficult to Very Easy
![Page 25: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/25.jpg)
25
Expectation Measure: Before/After
Before:
After:
![Page 26: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/26.jpg)
26
Expectation Measure
For each task, calculate two averages:Average expectation scoreAverage experience score
Visualize the data as a scatterplot with two axes:
Expectation ratingExperience ratingThe four quadrants are interesting
![Page 27: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/27.jpg)
27
Expectation MeasureAverage Expectation and Experience Ratings
by Task
1
2
3
4
5
6
7
1 2 3 4 5 6 7
Average Expectation Rating
Avg
. Exp
erie
nce
Rat
ing
“Fix it Fast”
“Promote It”
“Big Opportunity
“Don’t Touch It”1=Difficult …
7=Easy
1=Difficult; 7=Easy
![Page 28: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/28.jpg)
28
Tip 6: Use the System Usability Scale (SUS)
![Page 29: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/29.jpg)
29
SUS
Developed at Digital Equipment Corp.Consists of ten items.Adapted by replacing “system” with “website”.Each item is a statement (positive or negative) and a rating on a five-point scale of “Strongly Disagree” to “Strongly Agree” .
For details see http://www.usability.serco.com/trump/documents/Suschapt.doc
![Page 30: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/30.jpg)
30
SUS
![Page 31: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/31.jpg)
31
SUS ScoringSUS yields a single number representing a composite measure of the overall usability of the system being studied. Note that scores for individual items are not meaningful on their own.
To calculate the SUS score:Each item's score contribution will range from 0 to 4.
For items 1,3,5,7,and 9 the score contribution is the scale position minus 1.
For items 2,4,6,8 and 10, the contribution is 5 minus the scale position.
Multiply the sum of the scores by 2.5 to obtain the overall SUS score.
SUS scores have a range of 0 to 100.http://www.measuringux.com/SUS_Calculation.xls
![Page 32: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/32.jpg)
32
SUS Scoring Example
Total= 22 SUS Score= 22 * 2.5 = 55
![Page 33: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/33.jpg)
33
SUS Usage
“SUS has been made freely available for use in usability assessment, and has been used for a variety of research projects and industrial evaluations; the only prerequisite for its use is that any published report should acknowledge the source of the measure.”
![Page 34: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/34.jpg)
34
SUS Data from 50 Studies
Frequency Distribution of SUS Scores for 129 Conditions from 50 Studies
05
101520253035404550
<=40 41-50 51-60 61-70 71-80 81-90 91-100
Average SUS Scores
Freq
uenc
y
Percentiles:10th 47.425th 56.750th 68.975th 76.790th 81.2Mean 66.4
http://www.measuringux.com/SUS-scores.xls
![Page 35: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/35.jpg)
35
Tip 7: Show Frequency Distributions
![Page 36: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/36.jpg)
36
Frequency Distributions
SUS Score per Person
0
1
2
3
4
5
6
7
8
9
<=40 41-50 51-60 61-70 71-80 81-90 91-100
SUS Score
Freq
uenc
y
OriginalRedesign
Comparison of SUS scores cross two tests in an iterative process:(LeDoux, Connor, & Tullis, 2005)
Frequency distributions are a useful way of visualizing variability if you have a reasonably large number of data points.
![Page 37: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/37.jpg)
37
Frequency Distributions
Frequency Distribution of SUS Scores
0
2
4
6
8
10
12
<=10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100
SUS Score
Freq
uenc
y
NASAWikipedia
Excel sample
![Page 38: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/38.jpg)
38
How-toMust be entered by selecting a range of cells for the formula and then enter with <ctrl> <shift><enter>.
=FREQUENCY(<data array>, <bins array>)
![Page 39: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/39.jpg)
39
Tip 8: Combine Different Metrics
![Page 40: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/40.jpg)
40
Combined Metrics
Often it’s useful to combine different metrics to get an overall usability measure.Challenge is combining metrics that have different scales, e.g.
Task completion: % correctTask time: SecondsSubjective rating: SUS score
Two common techniques:Combine using percentagesCombine using z-scores
![Page 41: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/41.jpg)
41
Combine Based on Percentages
Basic idea is to convert each of the metrics to a percentage and then average those together.For each metric to be transformed, you want:
0% to represent the worst possible score100% to represent the best possible score
Some metrics already are a percentage:SUS scores% correct tasks
![Page 42: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/42.jpg)
42
Sample Data
Original data:
Participant #Time per Task (sec)
Tasks Completed
(of 15)Rating
(0-4)1 65 7 2.42 50 9 2.63 34 13 3.14 70 6 1.75 28 11 3.26 52 9 3.37 58 8 2.58 60 7 1.49 25 9 3.8
10 55 10 3.6Averages 49.7 8.9 2.8
![Page 43: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/43.jpg)
43
Sample Data
Excel spreadsheet
Original data with percentage transformations added:
Participant #Time per Task (sec)
Tasks Completed
(of 15)Rating
(0-4) Time % Tasks % Rating % Average1 65 7 2.4 38% 47% 60% 48%2 50 9 2.6 50% 60% 65% 58%3 34 13 3.1 74% 87% 78% 79%4 70 6 1.7 36% 40% 43% 39%5 28 11 3.2 89% 73% 80% 81%6 52 9 3.3 48% 60% 83% 64%7 58 8 2.5 43% 53% 63% 53%8 60 7 1.4 42% 47% 35% 41%9 25 9 3.8 100% 60% 95% 85%
10 55 10 3.6 45% 67% 90% 67%Averages 49.7 8.9 2.8 57% 59% 69% 62%
![Page 44: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/44.jpg)
44
Combine Using Z-scores
Another method sometimes used is z-score transformation:
Convert each participant’s score for each metric to a z-score.
Z-scores are based on the normal distribution.They have a mean of 0 and standard deviation of 1.Use the “standardize” function in Excel.
Average the z-scores for each person to get an overall z-score.
Make sure all scales go the same direction.Must decide whether each score is going to be given equal weight.
![Page 45: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/45.jpg)
45
Z-score Transformation Example
=standardize(B2,$B$12,$B$13)
![Page 46: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/46.jpg)
46
A Real Example: Older UsersOverall Usability Index
-5.00
-4.00
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
4.00
0 10 20 30 40 50 60 70 80 90
Age (in years)
Perf
orm
ance
Z-S
core
Study 1 Study 2Linear (Study 1) Linear (Study 2)
![Page 47: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/47.jpg)
47
Tip 9: Use Appropriate Tools
![Page 48: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/48.jpg)
48
Morae and Camtasia
Excellent tools for capturing video, logging issues, and creating highlight tapes (www.techsmith.com).
![Page 49: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/49.jpg)
49
Data Logger
Free Excel program used to collect and analyze usability test data. Records task success, time-on-task, survey questions, and automatically generates charts.
(www.userfocus.co.uk/resources/datalogger.html)
![Page 50: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/50.jpg)
50
Keynote (formerly Vividence)
Data collection and analysis tools designed to run automated usability studies. Metrics include task success, time-on-task, clicks, and click-stream.
(www.keynote.com)
![Page 51: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/51.jpg)
51
UTE (Usability Testing Environment)
Bob Bailey at Mind Design Systems (www.mindd.com)Comprehensive tool for capturing and analyzing usability dataAutomatically calculates success rates, time-on-task, and many other metrics
![Page 52: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/52.jpg)
52
Ergo Browser
Modified Internet browser designed to collect usability data including time-on-task, clicks, pages, and scroll usage. Reasonably priced and easy to use. (www.ergolabs.com).
![Page 53: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/53.jpg)
53
The Observer from Noldus
Sophisticated software for usability data collection, analysis and presentation. Ability to integrate multiple video feeds, analyze eye-tracking data, and other physiological measurements.
(www.noldus.com)
![Page 54: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/54.jpg)
54
Ovo Studios
Software for collecting and analyzing usability data. Allows you to log issue, collects time-on-task, create video highlights, and helps in report generation.
(www.ovostudios.com)
![Page 55: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/55.jpg)
55
“SUM” CalculatorAnother free tool developed by Jeff Sauro to calculate a single usability score “SUM” (www.measuringusability.com).
![Page 56: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/56.jpg)
56
Tip 10: Present Data Appropriately
![Page 57: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/57.jpg)
57
Top 10 Mistakes in Data Graphs1. Not labeling axes or units.2. Implying more precision in your data than it deserves.3. Not showing confidence intervals when you can.4. Not starting a bar graph at the lowest possible value
for the y-axis (usually 0).5. Using a line graph when it should be a bar graph.6. Using 3-D when it doesn’t add any value.7. Trying to include too much.8. Poor labeling of pie charts.9. Using color coding as the only way to convey
information.10. Not knowing when to use stacked bar graphs.
![Page 58: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/58.jpg)
58
What’s Wrong With This?
Successful Task Completion Rate
0%
10%20%
30%
40%
50%60%
70%
80%
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8
![Page 59: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/59.jpg)
59
What’s Wrong With This?
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8
![Page 60: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/60.jpg)
60
Better Version
Successful Completion Rate, by Task(Error bars represent 90% confidence interval)
0%
10%
20%30%
40%
50%
60%70%80%
90%
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8% o
f Use
rs W
ho S
ucce
ssfu
lly C
ompl
eted
![Page 61: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/61.jpg)
61
What’s Wrong With This?
% of Pages with Accessibility Errors
0 Errors1 Error2 Errors3-4 Errors5-6 Errors7-8 Errors9-10 Errors11-15 Errors> 15 Errors
![Page 62: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/62.jpg)
62
Better Version
% of Pages with Accessibility Errors
0 Errors10%
1-2 Errors12%
3-10 Errors53%
>10 Errors25%
![Page 63: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/63.jpg)
63
How Could This Be Better?
Breakdown of Task Completion Status
0%
10%
20%
30%
40%
50%
60%
70%
80%
NASA Wikipedia
% o
f Tas
ks CorrectIncorrectGave Up
![Page 64: Tips and Tricks for Measuring the User Experience](https://reader035.vdocuments.site/reader035/viewer/2022071523/613d0aab736caf36b7589ebd/html5/thumbnails/64.jpg)
64
Better Version
Breakdown of Task Completion Status
58%71%
18%
11%
24% 17%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
NASA Wikipedia
% o
f Tas
ks Gave Up
Incorrect
Correct