agile web application testing - siliconindia...•a good continuous integration set-up...
TRANSCRIPT
![Page 1: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/1.jpg)
Agile Web Application Testing
Technologies and Solutions
V. Narayan Raman
Tyto Software
![Page 2: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/2.jpg)
Goals
Rapid feedback on the quality of software
![Page 3: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/3.jpg)
Problem in Web App Testing
• Many Browsers
• Many Operating Systems
• Browsers have quirks
• Browser technologies are evolving at an amazing pace
![Page 4: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/4.jpg)
Functional or Unit Testing?
• Functional tests
– How the end user engages with the system
– Very important
• Unit tests
– Does not matter to the end user
– Necessary from a dev perspective
Functional or AND Unit Testing
![Page 5: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/5.jpg)
Automation vs. Manual
• Automation
– Can rapidly go over known scenarios and paths
• Manual
– Can be creative and go over new scenarios and explore new ways of interacting with the system
Automation is a time saving tool for Manual testers
![Page 6: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/6.jpg)
Agile
• Ability to rapidly adapt to changes in business requirement
– Automation scripts can break due to evolution of web application
– Good engineering practices are required to ensure effectiveness
![Page 7: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/7.jpg)
Engineering practices
• Frequent check-ins
• Frequent releases
• A good continuous integration set-up
– CruiseControl, Jenkins etc.
• Ensure Automation scripts are green before delivering the build to QA for testing
– Needs good interaction between Testers and Devs
![Page 8: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/8.jpg)
Automation Tools
• Everything on the desktop
– Good for broad level testing
– Ineffective for complex web applications
• Only web applications
– Good for web app testing
– Tools: Sahi, Selenium, Watir
![Page 9: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/9.jpg)
Technology choices of Tools - Factors
• Identification of elements
• Event Simulation
• Reliability of playback
• Library Dependencies
• Recorders/Object Spys
![Page 10: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/10.jpg)
Identification of elements
• Ids, names and other DOM properties – Best if you can have them. Not always possible though
• XPaths – A very popular but amazingly bad way of writing tests
• CSS Selectors – Better than XPaths but still depends too much on code
structure
• User Interface Relations – Best way to identify elements without depending on
code structure
![Page 11: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/11.jpg)
Examples
• DOM
– _textbox(“username”)
• XPath
– "/html/body/table[@id='rootfragment']/tbody/tr[2]/td/table/tbody/tr[2]/td[4]/table/tbody/tr[1]/td/table/tbody/tr[1]/td/div/table/tbody/tr/td[2]/table/tbody/tr/td*1+/strong“
![Page 12: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/12.jpg)
Examples
• CSS Locator
– "css=div#myID > a:contains(Confirm Transaction)“
• User Interface Relations
_button(“Delete”, _near(_cell(“Ram”)))
_cell(0, _near(_div("My product one")), _under(_div("Cost")))
![Page 13: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/13.jpg)
Good Identifiers
• Should be resilient to structural changes in UI
– XPaths and CSS Selectors are dependent on code structure
• Should remain constant even when moved across frames/iframes
• Should be able to resolve one element with respect to another
– _checkbox(0, _under(_tableHeader(“Choose”)))
![Page 14: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/14.jpg)
Event Simulation
• OS level events
– Theoretically more correct
– Needs focus of window/element (prevents reliable parallel playback)
• Browser level events
– Vary between browsers
– Does not need focus of browser
![Page 15: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/15.jpg)
Reliable Playback
• Problem – Variable load times for AJAX, Page/Frame/Iframe
• Solutions – Explicit waits
• _wait(30000) • Bad practice
– Explicit conditional waits • _wait(30000, _isVisible(element)); • Needs more code and knowledge of application
– Implicit Waits • Tool decides how much to wait • Much better and more reliable
![Page 16: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/16.jpg)
Library Dependencies
Usage of common development libraries for testing Eg. JQuery, Sizzle
• Problem – Conflict between tool’s version and web application’s
version
– 2 web apps with different library versions cannot be tested with the same tool
• Solution – Avoid dependence on libraries which are used in your
web app development
![Page 17: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/17.jpg)
Recorder/Object Spy
• Recorder
– Nice to have
– But scripts should be refactored
• Object Spy
– Absolute must have
– For rapid feedback and trouble shooting across browsers
![Page 18: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/18.jpg)
Automation Approach 1 Protocol Level
• Protocol level request response validation
– May use a proxy
– Cannot simulate javascript events reliably
– Does not work for modern AJAX applications with lots of snippets of request responses
![Page 19: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/19.jpg)
Automation Approach 2 Proxy Based Javascript Injection
• Javascript injection via Proxy
– Uses technologies like Proxy and Javascript which are guaranteed to be implemented by all browsers (with minor differences)
– Works across browsers and can easily support a new browser
– Con: Needs configuration of Proxy, SSL certificate management
![Page 20: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/20.jpg)
Automation Approach 3 Browser Extensions
• Very tedious to write extensions for every browser on every OS
• Maintenance problem for project developers
• Common libraries for javascript needs to be injected similar to proxy based approach
![Page 21: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/21.jpg)
Comparison of common Tools
Criteria Sahi Selenium 1 Selenium 2
Identification DOM, UI Relations (XPath, CSS Selector also possible)
DOM, XPath, CSS Selector
DOM, XPath, CSS Selector
Event Simulation (Predominant)
Browser Browser (but does not simulate all events)
Native
Waits (Predominant)
Implicit Explicit + Conditional
Half way
Library dependencies
None by default. Needed for XPath and CSS Selector
For XPath and CSS Selector
For XPath and CSS Selector
Recorder/Object Spy
Yes on all browsers Yes on Firefox only In development for Firefox only
Automation approach
Proxy based Proxy based + Plugin based
Native extensions for browsers
![Page 22: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/22.jpg)
Other features to look out for
• Integration with build tools and CI systems
• Parallel playback
– Saves time
• IDE support (Real ones, not Se IDE)
• Refactorable test scripts
• Easily understandable and learnable by testing team
• Inbuilt Reporting
![Page 23: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/23.jpg)
Note to Developers
• A testing tool is used mostly by testers
– Remember how long you took to be good at any programming language.
– Don’t force your programming language onto your testers. It will hamper productivity
• Really, don’t think like a developer when testing
– Bad practices like XPath usage have come in because of thinking in code
– Think like the End User
![Page 24: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/24.jpg)
Things to check out
• Tools
– Sahi, Selenium, Watir
• Behavior Driven Development (BDD)
– Cucumber, Behat, EasyB, Twist
• Performance and Load Testing
– PushToTest TestMaker
• Continuous Integration
– Cruise Control, Jenkins (Hudson)
![Page 25: Agile Web Application Testing - SiliconIndia...•A good continuous integration set-up –CruiseControl, Jenkins etc. •Ensure Automation scripts are green before delivering the build](https://reader034.vdocuments.site/reader034/viewer/2022042203/5ea4b8ad090bb108e6407346/html5/thumbnails/25.jpg)
Q & A
Thank You
Twitter: @narayanraman
Website: http://sahi.co.in/