build your own mad-lab - for mobile test automation
TRANSCRIPT
![Page 1: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/1.jpg)
MAD-LABMobile Automation Devices LAB
Anand BagmarSoftware Quality Evangelist
![Page 2: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/2.jpg)
About Me
@BagmarAnand
Blog – essenceoftesting
about.me/anand.bagmar
![Page 3: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/3.jpg)
What do you expect from this session?
![Page 4: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/4.jpg)
Context
![Page 5: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/5.jpg)
EcosystemPlatforms
• Android (> v4.4.2)
• iPhones (> v5)
• Desktop / WAP
Regions• Indonesia
• Malaysia
• India
• Middle-East
• ...
![Page 6: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/6.jpg)
Dynamic Content
![Page 7: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/7.jpg)
Search
![Page 8: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/8.jpg)
Download
![Page 9: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/9.jpg)
Offers
![Page 10: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/10.jpg)
ADs
![Page 11: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/11.jpg)
What this means ... Reality!• Unique installs (Android) - > 1.75M
• Unique # of devices (Android) - > 13K
![Page 12: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/12.jpg)
What this means ... Product!• One product, One experience
• Unique / Regional / Local content
• User detection & offers based on origin
• Region
• Carrier network
• ...
![Page 13: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/13.jpg)
What this means ... Testing!• Video Quality, Dynamic Content, Download Expiry
• Simulating Carriers / Networks
• Consumer focus!• Experience
• Device / OS combinations
![Page 14: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/14.jpg)
Business Requirements• Provide visibility into -
• What is Tested
• Overall Quality of Product
![Page 15: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/15.jpg)
APIs
Payment partners
CMS
....
....
Architecture
![Page 16: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/16.jpg)
APIs
Payment partners
CMS
....
....
Where are these?
![Page 17: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/17.jpg)
How would you approach
Automation for such a product?
![Page 18: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/18.jpg)
What Metrics make sense for
determining Quality?
![Page 19: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/19.jpg)
Automation Strategy
![Page 20: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/20.jpg)
Automation Criteria• E2E tests should focus on Business Rules & User Scenarios
• Reports should reflect the above
• Understood by all Business & Technical stakeholders
• Generated automatically
• Should run frequently
• Every new change in the product-under-test
• On a variety of device / OS combinations
![Page 21: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/21.jpg)
What to Automate?Validate what was working, continues to work!
• Identify user journeys / scenarios
• Prioritize it!
![Page 22: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/22.jpg)
Run Automation against...Usage patterns from Analytics / PlayStore / App Store
• Identify devices that matter
• Identify devices that are “automation-friendly”
![Page 23: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/23.jpg)
Automation Tech Stack• Cucumber-JVM à For specifying Business Rules
• Appium à To allow integration with Android / iOS
• Gradle / Groovy à As build tool scripting language
• Jenkins à Continuous Integration Server
• TTA - Test Trend Analyzer à Trend & Failure Analysis
![Page 24: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/24.jpg)
MVP / POC
![Page 25: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/25.jpg)
Scale&
Repeat
![Page 26: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/26.jpg)
Experiment, Learn & Evolve
![Page 27: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/27.jpg)
Experiments• Run tests on Emulators
• Use Cloud-based services (Local Vs Remote)
• SauceLabs
• TestObject
• Amazon Device Farm
• pCloudy
• Devices
![Page 28: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/28.jpg)
Some Experiments didn’t work• Emulators
• Video Player
• Location
• Cloud-based services• Location
• Tech Stack
• Devices
![Page 29: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/29.jpg)
MAD-LAB
![Page 30: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/30.jpg)
MAD-LAB – Mobile Automation Devices LAB
![Page 31: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/31.jpg)
Innovations in MAD-LAB• Device management (selection, cleanup, app install and
uninstall)
• Parallel test execution (at Cucumber scenario level) -maximising device utilisation)
• Appium server management
• Adb utilities (periodic ADB server disconnects)
• Custom reporting using cucumber-reports
• Video recording of each scenario and embedding in the custom reports
• Analytics Automation
![Page 32: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/32.jpg)
Conclusion
![Page 33: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/33.jpg)
Learnings (so far)Blue-print is important!
• Stepping-stone approach
Experiment!
• Know your objective(s)
• Fail-fast!
Make the Product Testable!!
![Page 34: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/34.jpg)
References• Sample code
• https://github.com/anandbagmar/cucumber-jvm-appium-infra
• Blog posts• https://essenceoftesting.blogspot.com/search/label/madlab
• TTA (Test Trend Analyzer)• https://essenceoftesting.blogspot.com/search/label/tta
![Page 35: Build your own MAD-LAB - for Mobile Test Automation](https://reader033.vdocuments.site/reader033/viewer/2022052117/5a67676a7f8b9a0a2e8b4887/html5/thumbnails/35.jpg)
Thank you!
@BagmarAnand
Blog – essenceoftesting
about.me/anand.bagmar