how to build an appium continuous testing pipeline to... · analyst designer analyst tester test...
TRANSCRIPT
![Page 1: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/1.jpg)
How to Build an Appium Continuous Testing Pipeline
Step-by-Step TutorialNovember, 2017
![Page 2: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/2.jpg)
Today’s speakers
Guy Arieli,
CTO,
Experitest
Ruth Zamir
Marketing Director
Experitest
![Page 3: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/3.jpg)
Agenda
3
Why do we need continuous testing?
The continuous testing environment
Step-by-step demo
01
02
03
Q&A04
15 min
30 min
5 min
10 min
![Page 4: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/4.jpg)
Digital transformation impacts business
4
Frequent app releases (mobile: 2 weeks, web: daily,
even hourly)
Digital is a keysurvival strategy
Customer empowerment
R&D organizations under pressure
more, faster, better
![Page 5: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/5.jpg)
Unique mobile market characteristics
• Fast technology innovation & complex
integration (Alexa/Siri, IoT, AR..
• Device matrix large and constantly evolving
• High cost of error - App/play stores create a
distribution gap that makes rollback impossible
5
Scope * Velocity * Risk
![Page 6: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/6.jpg)
SDLC Paradigm shift
Speed, agility and risk reduction
6
Agile CI/CDDevOps
“Develop smaller chunks - we will probably change our mind”
“get your team talking…”
“make sure you didn’t break anything…”
![Page 7: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/7.jpg)
What is Continuous Integration?
“a software development practice where members of a team integrate their work frequently...
Each integration is verified by an automated build… to detect integration errors as quickly as possible.”
![Page 8: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/8.jpg)
There is no Continuous Integration without Continuous Testing
![Page 9: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/9.jpg)
The estimated* impact of a shortened feedback loop
R&D productivity will increase from 10% to 35% since more time will be invested
on Dev-test activities
Dev-test
10%
Project
related
15%
Context switching
20%Bug reproduction
25%
Root cause analysis
25%
Administration
5%
R&D productivity - 1 week feedback
Dev-test
35%
Project
related
20%
Context
switching
14%
Bug reproduction
13%
Root cause analysis
13%
Administration
5%
R&D productivity - 1 hour feedback
* Data source: Experitest customers
![Page 10: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/10.jpg)
Rapid feedback significantly increases R&D effectiveness
0
10
20
30
40
50
60
1 week 1 day 6 hours 2 hours 1 hour 1 minute
Time to feedback
R&D Effectiveness
![Page 11: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/11.jpg)
The dev-test process
Epic Story thinning PrototypeAcceptance
Criteria
Code (incl. White box Test)
Build & DeployAutomated
testingAcceptance
testing
Showcase / Demo Sign-off
Dev Engineer
Dev & Test EngineerDevOps EngineerDev & Test Engineer
Functional Spec
Product Owner
Analyst Product Owner
Analyst Designer Analyst Tester Test Engineer
TesterProduct Owner
CI/CD cycle
![Page 12: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/12.jpg)
12
High scales simultaneous testing of different use cases, devices and network conditions
Highly stable test suites
Test environment management requires IT / Security expert
Rapid result analysis and actionable feedback
Continuous testing in mobile - implications
![Page 13: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/13.jpg)
Agenda
13
Why do we need continuous testing?
The continuous testing environment
Step-by-step demo
01
02
03
Q&A04
![Page 14: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/14.jpg)
Development Engineer
The Continuous Testing process
Commit tests
Trigger test execution
Load app to cloud
Test Engineer
Commit app code
Consolidate test results
Continuous Integration
reporttest
build Version approved?
no yes
Debug / update test suite
1Pull app source2
Build app3
Pull test source4
Build test project5
6 7 8
9
10
Trigger build
![Page 15: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/15.jpg)
Today’s demo environment
• Appium tests
• TestNG framework
• Jenkins server
Pre-prepared
• Appium* Studio for Eclipse
• Eclipse*
• Jenkins*
• Git
• Gradle
• Experitest Digital Device Lab
• Experitest Reporter
Tools used
• Build Application (Jenkins)
• Pull tests from Git and build test project (Jenkins + Gradle)
• Upload application to the cloud - Jenkins (shell script)
• Run Tests (Jenkins + Digital Device Lab)
• Link test results to Jenkins
What we will do
* Any test framework, IDE and CI tools can be used
![Page 16: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/16.jpg)
Agenda
16
Why do we need continuous testing?
The continuous testing environment
Step-by-step demo
01
02
03
Q&A04
![Page 17: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/17.jpg)
Configuration and setup
17
Demo
![Page 18: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/18.jpg)
Executing from Jenkins
18
![Page 19: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/19.jpg)
Recap
• Appium*
• TestNG*
• Eclipse*
• Jenkins*
• Git
• Gradle
• Appium Studio for Eclipse
• Experitest Digital Device Lab
• Experitest reporter
Tools / frameworks used
• Used CI server to build an applicationfrom repository
• Used CI server to build a test project from repository
• Used CI to upload application to digital device lab
• Used CI and digital device lab to installed application numerous devices and run tests in parallel on
• Used Reporter to link test results to CIand analyzed test results
What we did
* Any test framework, IDE and CI tools can be used
![Page 20: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/20.jpg)
How the SeeTest platform support hands free continuous testing
20
Epic Story thinning PrototypeAcceptance
Criteria
Code (incl. White box Test)
Build & DeployAutomated
testingAcceptance
testing
Showcase / Demo Sign-off
Dev Engineer
DevOps EngineerDev & Test Engineer
Functional Spec
Product Owner
Analyst Product Owner
Analyst Designer Analyst Tester Test Engineer
TesterProduct Owner
Dev & Test Eng.
Digital assurance Lab (SeeTestCloud)
Grid Execution
Test development
Test Analytics
Manual Testing Grid Execution Dev &
DebugTest Analytics Test Analytics
RESTful API to mass
deploy application on
targeted devices
Dev-Test Feedback cycle – 1 hour
![Page 21: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/21.jpg)
Agenda
21
Why do we need continuous testing?
The continuous testing environment
Step-by-step demo
01
02
03
Q&A04
![Page 22: How to Build an Appium Continuous Testing Pipeline to... · Analyst Designer Analyst Tester Test Engineer Product Tester Owner Dev & Test Eng. Digital assurance Lab (SeeTestCloud)](https://reader031.vdocuments.site/reader031/viewer/2022040608/5ec5bb10f7a1a0769a34f2a6/html5/thumbnails/22.jpg)
Thank You!
22