![Page 1: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/1.jpg)
Are Users the Gold Standard for Accessibility Evaluation?
7th April - W4A’14
Amaia Aizpurua1, Myriam Arrue Simon Harper2, Markel Vigo3
U. of the Basque Country University of Manchester
http://dx.doi.org/10.1145/2596695.2596705
1: @amaiaaizpurua 2: @sharpic3: @markelvigo
![Page 2: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/2.jpg)
Motivation
• User testing for accessibility evaluation
• Encouraged by the community
• No established procedures– What?– When? – How?
27th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
![Page 3: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/3.jpg)
In practice
• Evaluating web accessibility for blind users
– User testing method• Effectiveness in identifying accessibility problems
[Mankoff et al. 2005]
– Accessibility guidelines• Partial coverage of user problems [Power et al. 2012]
37th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
![Page 4: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/4.jpg)
Study results• Accessibility: conformance vs. perceived
Website AA conformance level Perceived
47th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
Median Mode SD
6 7 1.95
6 6 1.42
2 1 1.62
6 6 1.95
1: very inaccessible
7: very accessible
Satisfied SC Non-satisfied SC
73% 27%
69% 31%
52% 48%
36% 64%
![Page 5: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/5.jpg)
Why this mismatch?
• Guidelines vs. user problems
• Eliciting users’ problems– Experience/report problems differently: web
expertise– Interaction context, tasks type
• Identifying accessibility issues– Combination of causes, contextual
information– Under the influence of the evaluator effect
57th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
![Page 6: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/6.jpg)
Discussion
• Inclusive participatory evaluation– Following the considerations of co-operative
evaluation– Evaluator and user evaluating a website
collaboratively– Instead of interpreting data afterwards, work
on problems during the session
• Improve effectiveness
67th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
![Page 7: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/7.jpg)
Conclusions
• User testing may not be the gold standard for web accessibility evaluation
• Users are a gold mine
• Co-operative accessibility evaluation
• Bridge the gap?
77th April - W4A’14 Are Users the Gold Standard for Accessibility Evaluation?
![Page 8: Are Users the Gold Standard for Accessibility Evaluation?](https://reader036.vdocuments.site/reader036/viewer/2022062319/558cc8efd8b42afe7b8b46d8/html5/thumbnails/8.jpg)
Thank you!
87th April - W4A’14
Contact
[email protected] | @amaiaaizpurua
[email protected] | @sharpic
[email protected] | @markelvigo