accessibility metrics accessibility data metrics and reporting – industry best practices
TRANSCRIPT
Accessibility MetricsTed Drake and Sagar Barbhaya Intuit Accessibility CSUN 2017 Accessible version: http://www.last-child.com/a11y-data-metrics/
Current State• Manual Testing and Evaluations
• Continual improvements
• Documentation
• Community Outreach
• Stakeholders
• Centralized Accessibility Leadership
What we have• Product database with evaluations, scores,
stakeholders, and documents
• Bug Tracking
• Scattered Employee Resources
• Centralized documentation
• Non-integrated automation
What we need• Dashboards to surface data and trends
• Connections between platforms
• Automation and unit testing for developers
• Shared test results
• Connected resources
• Support channel integration
Future Goal• Measurable lifecycle from tests/evaluation to
production
• Deep consumer feedback integration
• Diverse workforce with support channels
• Developer ownership of testing and solutions
• Product ownership of process and success
Product Quality Metrics
• Issues by Priority – 77%
• Issues by WCAG category – 54% • last time product had a manual evaluation – 46%
• Closed Captioning support – 38% • Number of accessibility bugs opened per month/
year – 31%
Priority Levels
• Impact on Customer
• Template / Module / Unique
• Density of Issues
• High Traffic, Key Flows
Company Metrics•AODA compliance – 55% •Number of media inquiries about product/company
accessibility – 45% •Conference talks by employees on accessibility – 45%
•Product VPAT Coverage – 45% •Compliance towards 21st Century Communications Act
(CVAA) – 36%
Human Resources
• Section 503: Self-Identification
• Accommodations Requests
• Ergonomic Evaluations
• Service/Perk Requests
Customer Metrics•Usability participants with a disability – 63%
•Feedback via customer support – 50%
•Usability activities, including user personas,
with disabilities – 38%
•Customer support calls - 38%
•Tickets filed by customer support – 38%
JIRA Queries
• labels = accessibility
• labels=accessibility and status=Open
• labels=accessibility and status = blocked
• labels=accessibility and status=Closed and resolved > “-4w”
Integrated Testing
• Developer testing: Tenon, aXe, Nemo
• Unit tests include accessibility
• Export test results to JIRA
• Tracking tool usage
Finding Outliers
• Extremely critical to prioritize
• Outliers need to be identified and fixed first
• Example - Turbo Tax in season production bugs
Customer Support Logs
• Search for Deaf and HOH customer requests: [ LEVEL:SENTENCE TYPE:KEYWORD {deaf, hearing}] [ LEVEL:SENTENCE TYPE:AND_KEYWORD {capability, impaired, challenged, hard}]
• “Accessibility” topic within Salesforce for reports
• Microsoft: Dedicated Support Channel