shift left mobile application testing

W14 Track Session 4/20/2016 2:00 PM

"Shift Left Mobile Application Testing"

Presented by: Satyajit Malugu

GoDaddy

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ info@techwell.com ∙ www.techwell.com

Satyajit Malugu GoDaddy A senior SDET at GoDaddy, Satyajit Malugu is focused on mobile testing. His work involves automation of native and hybrid apps, and providing test perspective to his team that recently converted from waterfall to agile. As a testing leader in a company that is deploying a suite of native apps, Satyajit is involved with strategizing, executing, and mentoring other testers and stakeholders on best practices for native mobile testing. Previously, Satyajit worked at Urbanspoon and gained SDET knowledge at Microsoft. He frequently presents on mobile topics. Read his blog and follow him on Twitter at @malugu.

Shift left Mobile Application TestingSatyajit Malugu

Urbanspoon

Specification Design Development Maintenance

Domain Finder

Specification Design Development Maintenance

Still had lot of problems

• Testing unable to fit into agile dev cycle

• UI automation unable to keep up with constant changes

• A very low coverage after weeks of playing catchup

Test pyramid reversed

http://adventuresinqa.com/2015/10/19/mobile-test-pyramid/#more-3194

What is shift left

• Testing processes start along with development processes

Godaddy Investor

Specification Design Development Maintenance

Why Shift left

• Cost of fixing bugs exponential across SDLC

Specification Design Development Maintainance

Enabling Agile

• Agile is all about adaptability to change

• Enabled through fast feedback loops

• Test early and often

Challenges for shift left in mobile

• Tool support not mature

• Device fragmentation

• Culture of native development teams

Tool support is getting better

• Companies dedicated to mobile testing

• Real device clouds

• Network variations

• Sensor simulation

Android tool support

• Unit tests – JUNIT

• Code coverage – Jacaco

• Continuous integration – Jenkins

• Functional tests – Robotium, roboelectric, Espresso etc

• Continuous deployment – Alpha, beta releases in play store

• Device emulation – adb, genymotion

• Performance testing – rendering profiler, traceview

iOS tool support

• Unit tests – XCTest,

• Code coverage – Xcode server with bots

• Continuous integration – Jenkins/Xcode server

• Functional tests – EarlGrey

• Continuous deployment – Testflight alpha, beta

• Device emulation – Xcode

• Performance testing – XCTest, instruments

Reclaiming the pyramid

Step I – Measure unit test code coverage

• What you can’t measure, can’t be improved

• Get the team to agree on a good coverage %

• Send coverage report everyday

Slack bots reporting daily coverage

Step II– Create a CI infrastructure

• For running unit, functional, UI tests

• Build various environments

• Code coverage reports

Pull Request triggers unit and UI tests

Jenkins UI test run sample

Step III– Replace some UI tests with unit tests

• As a principle prefer lower level test over a higher one

• Don’t repeat a test - DRY

Step IV- Partial integration testing

• Create test hooks for navigating into app directly

• Deep links, shortcuts

• Level – 1 verifications

• Not everything need to be verified through UI

Step V- Re-architech E2E tests

• Should be minimal• Run them continuously• On a cloud provider

• Should be atomic not interdependent• Enables running in parallel

• A single test can be triggered in multiple ways• Different language

• Different screensize

• Different OS

Specification Design Development Maintenance

1. Continuous testing2. Crash Monitoring3. Performance Monitoring4. App store reviews monitoring5. Engagement monitoring

1. Functional tests2. UI Tests

1. Testability hooks2. Accessibility ids3. API level testing

1. Test infrastructure2. CI/CD setup3. Device lab

Summary

On the horizon – React Native

• Bringing web like deployment capabilities to native world

• Learn once, write anywhere

• Very high importance to testing