©2016 Infostretch. All rights reserved. 1
Adam CarmiCo-Founder and CTOApplitools
BEST PRACTICES OF AUTOMATED
VISUAL TESTING
GETTING IT RIGHT FOR THE BEST UX
Manish MathuriaFounder and CTO Infostretch
©2016 Infostretch. All rights reserved. 2
Agenda
• What is Visual Testing?
• Visual Testing Tools Requirements
• What kinds of apps must be tested Visually?
• Visual Testing Process
• Best Practices
• Demo
©2016 Infostretch. All rights reserved. 3
Visual Testing in the Lifecycle
IndispensableLate in the
lifecycle
Manually
Intensive
Does not
Capture
Visual
Defects
©2016 Infostretch. All rights reserved. 4Image Source: http://baymard.com/blog/bugs-and-user-experience
Visual Bugs
©2016 Infostretch. All rights reserved. 5
©2016 Infostretch. All rights reserved. 6
Visual Testing - What do Testers Want?
Incremental Efforts / Automated Validation
• Effort for baseline and maintenance of the same
No False Positives
• Support for Dynamic Content
• Smart validation: ignore/validate viewport portions, comparison with tolerance level
Effective Error Reporting
• Visual Defects Reporting Integrated into Test Results
• Failure Analysis, video reports
Technical Capabilities
• Ignoring Dynamic Content
• Integration Into existing test automation framework
• Layout verification
• Screen resolution independent
©2016 Infostretch. All rights reserved. 7
Visual Testing – Hi-Pot Candidates
•
•
•
•
•
Responsive
High Form
factors
Content
Rich
Device
fragmentatio
n
©2016 Infostretch. All rights reserved. 8
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
©2016 Infostretch. All rights reserved. 9
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
©2016 Infostretch. All rights reserved. 10
©2016 Infostretch. All rights reserved. 11
©2016 Infostretch. All rights reserved. 12
APPLITOOLS EYES
• Handles anti-aliasing, pixel offsets, color similarity, and image scaling
• Dynamic and moving content
• Compare images of different sizes
• No error ratio configuration required
• Validates full UI pages
• Fast!
• Supports layout / structure matching
©2016 Infostretch. All rights reserved. 13
DEMOIMAGE MATCHING
©2016 Infostretch. All rights reserved. 14
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
©2016 Infostretch. All rights reserved. 15
DEMO+ +
©2016 Infostretch. All rights reserved. 16
FULL PAGE VALIDATION
• More coverage
• Catch unexpected bugs
• Enables codeless baseline maintenance
– Everyone can do it
– Immediately
– Avoid element locators maintenance
©2016 Infostretch. All rights reserved. 17
TEST INTEGRATION OPTIONS
• Write dedicated visual tests
– Maximal control over visual coverage and AUT manipulation
– Moderate effort
• Add visual checkpoints to your existing functional tests
– More ROI out of existing tests
– Moderate to light effort
• Add implicit visual validations to your test framework
– Trivial to implement
– Full page validation
– May result with duplicate validation points
– Exclude tests that are heavily parameterized on data
©2016 Infostretch. All rights reserved. 18
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
©2016 Infostretch. All rights reserved. 19
DEMOBASELINE MAINTENANCE
©2016 Infostretch. All rights reserved. 20
SPECIAL OFFERS!
Infostretch
Get a free 4-hour assessment of your visual testing strategy:
http://bit.ly/infostretch-get-free-4
Applitools
Join our free Visual Testing Master Class – learn more advanced features and in-depth
capabilities: https://goo.gl/awXPAg