case study: how to remove the testing bottleneck case... · 2018-04-26 · how t-systems releases...
TRANSCRIPT
Case study:How to remove the testing bottleneck
April 2018
Agenda
2
The testing bottle neck
How T-Systems releases the testing bottleneck - case study
Demos
01
02
03
Summary & Q&A04
7
15
15
10
6
Business priorities impact development and delivery goals
7
Source: Forrester: Master DevOps For Faster Delivery Of Software Innovation, 2017
Industrial scale quality: an automated assembly line
8
< / >
Developer
Test engineer
DevOps engineer
Reliable, comprehensive dataConsistent testingConstant new releases
5 best practicesfor releasing the testing bottleneck
Simple, low maintenance test creation
Wide technology coverage
(platforms, application types, device manufacturers, OS)
Intelligent large scale execution – as part of CI
Rapid feedback, data based decisions
DevOps approach – open, collaborative,
automated
Agenda
10
The testing bottle neck
How T-Systems releases the testing bottleneck - case study
Demos
01
02
03
Summary & Q&A04
7
15
15
10
12
s.Oliver Fashion AppSituation and goal
• Existing webshop with backend
• Create native iOS and android app, connection to existing backend
• Develop agile with 2 week sprint
• After Launch:
- New App Update About Every 4 Weeks
- Web Shop Releases Every 2 Weeks
• Controlled QA
• Efficiency with shop releases
Situation QA Goal
13
Goals and Approach
Integrate QA from beginning
Deliver High Quality App
Continuous integration approach with automation
and build chains
Flexible and Agile
Efficient testing on backend changes
Integrate with Existing Webshop
14
Approach to Testing
Combination of Manual and Automated Testing
Test on Real Devices
Reduce Overhead Through Automation
Collaboration with R&D for Testability
15
Approach to TestingMOBILE DEVICE CLOUD
REAL DEVICES, ACCESS WITH BROWSER OR CODE
Combination of Manual and Automated Testing
Test on Real Devices
Reduce Overhead Through Automation
Collaboration with R&D for Testability
16
Combination of manual and automated Testing
Execution Grid for AutomationManual Testing in Browser
New Features Regression
DECISIONS
o Use Java with SeeTest Automation
o Locate elements with XPath (Dev support)
o Explicit Assertions on various Levels, lot of Screenshots
o Run continuously in Build Server
o Leverage central device lab
18
Automation Approach
o Maintainable
o Reliable & Robust
o Very expressive results
o Remote
AUTOMATION GOALS
Agenda
19
The testing bottle neck
How T-Systems releases the testing bottleneck - case study
Demos
01
02
03
Summary & Q&A04
7
15
15
10
20
Demo
Developing highly maintainable tests
21
Automation ApproachPage Object Pattern
Test Script
Page A
Android Page A
Page B
iOS Page A
Android Page B
iOS Page B
OS independent
OS specific
79% of the Code
21% of the Code
22
23
Demo
Continuous integration approach to test execution
24
25
T-system reporting system
26
Demo
Cross Browser testing
Automation Approach
27
Page Object Pattern
Test Script
Page A
Mobile Page A
Page B
Web Page A
Mobile Page B
Web Page B
OS independent
OS specific
28
Agenda
29
The testing bottle neck
How T-Systems releases the testing bottleneck - case study
Demos
01
02
03
Summary & Q&A04
7
15
15
15
30
Simple, low maintenance test creation –
• R&D involvement
• Abstraction for code reuse
• XPath for stable object identification
Wide technology coverage
• Abstraction for code reuse across platforms
• Device cloud
Intelligent large scale execution
• Cross platform execution
• Integrated with the CI build chains
Data driven
• Use of reports and data for decision making
DevOps approach
• Early consideration of QA needs
• High level of automation
• Frequent testing for rapid feedback
5 best practicesfor releasing the testing bottleneck
What we did
Device Cloud & Grid Execution (mobile and web)
SeeTest Digital Assurance Platform
Test Development
SeeTestAutomation / Appium Studio
31
Tools used today
Questions?
34