advanced mobile testing - ravindran antonysamy
DESCRIPTION
Ever wonder why some mobile applications loved by millions ? Ever wonder why few mobile applications stand on top of the list of million apps ? Its the quality and usability of the product that drives them to the top of the -already-crowded- marketplace. do they stand on top just because the makers do app store marketing and SEO ? or Its just the genre of the application that is need of the hour ? How can you deliver an application that is useful and also usable ? How can you deliver an application that won't crash on most of the phones and run flawlessly ? How can you provide the maximum test coverage of your mobile app, that is practically impossible to achieve ? Do I have to perform expensive usability testing ? do I have to buy mobile clouds ? do I have to perform emulator testing along with real device testing ? Why should I have a social media channel that engage real users ? Why should I crowd source my mobile application testing ? upto what extend I can crowd source ? how much lab testing is sufficient and upto what % of defects can be captured using lab testing ? do I have to consider the power consumed by my mobile app ? what level of mobile application testing automation I shall be able to perform ? what are all the tools I shall be using ? This webinar will try to address most of the questions. Mobile testing OS / Market Coverage / Challenges in Mobile testing World Available Testing techniques (Lab testing, Beta user testing (Internal and external) , Usability testing, Field testing, Functional Automation testing) Emulator testing Vs Real device testing Practical Usability testing Mobile Testing Functional Automation (Open Source Tools) Is it possible to provide 100% coverage in Mobile testing ? Who should attend: Mobile Application testers, Mobile Handheld testers, Mobile Usability testers, Product Managers, Test and Tech LeadsTRANSCRIPT
Advanced Mobile Testing !!
1
Ravindran Antonysamy, QA Head, Nimbuzz !!!http://www.slideshare.net/ravindranantonysamy/advanced-mobile-testing
Youtube Video of the Webinar : http://www.youtube.com/watch?feature=player_embedded&v=-LYtG45HXb8
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
2
Who Am ITarget AudienceMobile Handset / Application ECO System / Smartphone marketOEM / Handset testing Vs Application testingNative Vs Hybrid Vs Web AppsMobile App Test Strategy - Emulator Testing Vs Real Device
- Ambassador testing- Crowdsourcing
- Social Media channels - Mobile Clouds - Usability testingTest CasesMOS in Mobile Application testingPower Consumption testingMobile Application testing Automation Tools – Calaba.sh MobileQAZone.comLinksUltimate Cheat sheet for Mobile TestingNimbuzz !!
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
This is In
dex I say
3
Who Am I & What I love
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
4
Who Am I & What I love
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
Robotics
Renewable Energy
Open Source
Disc:$
I may be wrong !
I may be missing things !
Feel free to disagree with me and argue with me !
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
5
6
Target Audience
Fresh College Pass OutsSoftware Testers and ‘Wanna’ be software / mobile testersAutomation TestersTest Leads, Tech LeadsProduct and Project ManagersFriends and Family.
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
7
Some Mobile Facts
Total Number of Mobile Phones in the World : 6 billionMobile Phone internet usage is surpassed PC internet users – 2013Total Mobile apps in Android Play store 900,000, Apple store 800,000Smartphone users spend at least 30 hours in their phones in a month
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
8
Mobile Handset / Application ECO System – P1OEM’s: Samsung, Apple, Nokia, ZTE, Blackberry, Huawei, HTC, Motorola, Sony, Toshiba, Lenovo, Sagem, JCB,
Network Equipment : Ericsson, NSN, Huawei, Juniper
Chipset / Board Vendors : ST-Ericsson, Qualcomm, MTK, M-Star, Broadcom, Atheros, NVIDIA, Intel, TI, Freescale,
Service Provides : Three, Vodafone, Airtel, MTN, Singtel, Telstra, AT&T
Mobile Operating Systems : Android, iOS, Windows Phone, Java - MIDP, Blackberry, Symbian, Bada (deprecated), Web OS (Probably deprecated), Jolla, Ubuntu Edge, Firefox OS
App Makers, Developers : Whatsapp, Nimbuzz, Real Racing (EA Games), Runtastic, Running with Friends (Zynga)
Ad Networks : AdMob, MadsOne, Smaato, AdNear, Innteractive, Ripple, iAds, Enpocket, MobYD, LeadBolt, AdInch, AdFonic, Amobee
Moible Test Clouds : Device Anywhere (keynote), Perfecto Mobile, Scirocco Cloud, See Test Cloud, Cloud Monkey
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
9
Mobile Handset / Application ECO System – P2Appstores : Google Play, Apple App Store, Windows Phone Marketplace, Blackberry App World, OVI Store, getjar, Mobango, Amazon app store
Mobile App test crowdsourcing : Mob4hire.com, utest.com, Testcovery.co, 99tests.com
Hybrid App Making Tools : Titanium (Appcelerator), Phone Gap, Sencha Touch, RHO Mobile
Authorities / Forums / Bodies : 3GPP, 3GPP2, OMA, OHA, ITU-T, GSMA, Wi-Fi Alliance, Bluetooth SIG, NFC Forum, GCF, PTCRB, FCS, IEEE
Test Automation Tools (Open source / Paid / Native / Cross Platform) : Robotium, Calaba.sh, Appium.io, SeeTest, MonkeyTalk, Telerik, Android Monkey, UI Automator for Android, Frank, KIF, Anteater, TouchTest, UI Automation for iOS, Zucchini, Test Plant (egg plant), ZAP-Fix, Silk Test for Mobile, UFT Mobile, Selenium Web driver,
Other : LIMO, Linaro, MobileIRON (Enterprise Mobility), Mobile Antivirus, Mob VAS, Flurry, Localytics
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
10
OEM Handset testing Vs Application Testing
OEM Handset testing performed by OEM’s.
Mobile Application Testing (Native / Hybrid / Web) Performed by Application Vendor / Service Companies or Developers.
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
OEM Testing Types – Drop Testing
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
11
OEM Testing Types – Keypad stress test
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
12
OEM Testing Types – Bending test
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
13
Upto 100 nuwtons applied
OEM Testing Types – Drop water test
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
14
OEM Testing Types – Dust box test
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
15
OEM Testing Types – Twisting test
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
16
OEM Testing Types – Wearing test
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
17
Field Testing (with tools like TEMS)Call Performance Testing (with tools like Optis Innowireless)Battery testing (Test of the battery itself)Native Application testing (SMS, PB, Native apps)Power consumption testing (Power consumed by various apps / features / calling / media playback – Using Agilent / Anritsu signaling testers)Bluetooth, Wireless LAN, NFC / RF Integration TestingVideo / Call Quality testing (with Tools like SwissQual)
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
18
OEM Testing Types – Other Tests
19
OEM / Handset testing Vs Application testing
- Only around the OS / Device- No need to test the Native Apps (like SMS, Camera)- Only around the App- Functional testing, Integration testing, Performance testing, Security testing, Usability testing, UAT
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
20
Native Vs Hybrid Vs Web Apps
- Native Apps : Made by Native SDK’s (Android SDK, Xcode, WP SDK, BB SDK)- Hybrid Apps : Made by Hybrid tools (Phone Gap, Appcelerater Titanium, RHOMobile)- Web Apps (HTML 5 / CSS)
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
21
Mobile App Test Strategy
- How many devices I should buy ?- My DUT’s should have what OS Version ?- Do I have to test in Emulator or Device or both ?- Do I have to use Ambassadors or Social Media ?- Do I have to crowd source my app testing ? - How can I share my test app to users ? (test flight / appbundle / knappsack / Hockeyapp)- What to do with the comments I receive in app stores ?- What to do with the crash reports from users on Dev dashboard ? (http://postimg.org/image/yom95zfmp/)- Do I have to use automation ? Upto what extend ? What tools to use ? - Performance testing ? Benchmarking ? - Security testing ? Encryption ? - Do I have to perform Usability testing ? Is that expensive ?- Do I have to use device Clouds ? How much It would cost me ? - How many SIM Cards I should have for testing ? - Will social media helps me in testing my mobile app ?
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
22
Test Cases and Test Preparation
What kind of test cases I should write ? What is MMI (Man Machine Interface)Why I should have a Pre-Condition ?What is Interoperability testing ? (OMA Test Fest)SIM Cards, Data PacksInterruptsConsidering Mobile QoS Test Data / Screen Sizes (http://www.quirksmode.org/m/tests/widthtest.html)
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
23
MOS in Mobile Application testing
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
Parameter Score
User Interface 5
Performance / Sluggishness 4
Functionality 4.5
Connectivity 4
Usability 4
Call Quality 4
Battery Consumption 4
Circulate this kind of sheet within the organization, friends, family before releasing the app. Make sure everyone feels as great as your developers. you are good to go !
24
Power Consumption testingOld Method- Start the timer- Charge the mobile 100%- Perform different scenarios- Note down the battery bars- Once battery dead- End the timer
New Method- Start the timer- Charge the mobile 100%- Perform different scenarios- Check the settings for battery % in each scenario- Once battery dead, - End the timer
Fool Proof MethodUse Agilent 8960 or Anritsu 8475A to setup the data acquisition unit / dummy battery / power supply setup !!
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
ToolsPower Tutor
GSAMBattery Life ProEZ Battery Life
25
Practical Usability Testing
- What is sample size and Why Its important ?
- What is test protocol in Usability testing ?
- Who will be the users ?- Who will assess the results ?- What is Feedback forms ?
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
26
Mobile Application testing Automation
- Is it possible to automate all the test cases in a mobile app ?- Is there reliable open source tools available ? - Hardware / software setup ?- Do I need to know Java to be a mobile application automation tester ?
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
27
Tools – Calaba.sh
- Easy to setup- you can write test scripts in readable english- Easy to update- Easy to maintain the scripts- Open source community support
Watch this : http://www.youtube.com/watch?v=mvzGAs9aD20 – Mancef http://www.youtube.com/watch?v=9FAjxMLyTco – Karl Krukow
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
28
Mobile Browser Testing Tools
- iPhony- W3C Mobile OK Checker - http://validator.w3.org/mobile/) - iPADPeek- Modify Headers Add On in Firefox- Adobe Device Central CS5- Google Mobilizer - http://www.google.com/gwt/n - Opera Mini http://www.opera.com/developer/opera-mini-simulator
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
29
MobileQAZone.comA Community dedicated for mobile testers. Discuss new tools. Chat with other mobile testers. Solve problems. Contribute.
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
30
Some Links
www.gsmarena.com www.gsma.com www.openmobilealliance.org www.quirksmode.org http://www.mobileqazone.com/www.3gpp.org http://calaba.shhttp://appium.io http://phonegap.com https://sellanapp.com/http://www.google.com
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
31
Ultimate Cheat Sheet
Ultimate Cheat Sheet for Mobile Testing
How to start learning mobile testing ?
Start by pointing your browser to android.com, download the SDK and Eclipse. Do read the documentation (Its very comprehensive) – Create your first AVD (Android Virtual device) and run your first mobile application in the emulator. You are all set.
How to test iPhone applications ?
You must have an iPhone. Or else, a MAC & XCode (IDE for making iPhone Apps).
Do I have to buy a tool for mobile testing automation ?
The Answer is No. you can manage with Open source tools. (I am not working for a mobile test automation tool making company yet, so this is how I will answer)
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
32
Ultimate Cheat Sheet
Ultimate Cheat Sheet for Mobile Testing
How can I take my application logs when I am performing android manual testing ?
Install eclipse (download it from android.com where you can get the full SDK pack.) Keep USB debugging ON in your mobile, go to DDMS view in your eclipse, you will be able to see the logs. You can also use an app called alogcat (search it in play.google.com), It will also show all the logs.
How to run adb from anywhere in the command prompt?
From Android platform location, Add platform-tools path (where ADB is located) to your environment variables – path. Now see the magic. (go to command prompt and type adb)
How can I stress test my android mobile application ?
Use Android Monkey. In your command prompt say adb shell monkey –p package name –v verbos (Example : adb shell monkey –p com.nimbuzz –v 50000)
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
33
Ultimate Cheat Sheet
Ultimate Cheat Sheet for Mobile Testing
I don’t have a MAC. How can I take logs for my iPhone App ?
Use iPhone Configuration Utility for Windows. (google It I say)
How to Sceen cast my Android application from my device to my PC ?
Use Androidscreencast.
Do I need developers support for testing iPhone applications ?
Yes, they need to add your UUID into their build system / certificate. Only then you will be able to test the iPhone application coming from the developer.
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
34
Ultimate Cheat Sheet
Ultimate Cheat Sheet for Mobile Testing
What is the test setup for Windows Phones ?
In-order to Install the .xap (WP executable), you need Zune + XAP Installer. Also you need to have the developer account. Without Windows Developer account, you will not be able to install the XAP.
How to test the BlackBerry .cod file.?
Forget it. BB is dead. Even Symbian in that context. Android, iOS, Windows Phone is what we will work in the future. Jolla, Ubuntu Edge, Firefox OS’s will run android apps.
Define mobile testing strategy?
You cover different networks, different OS, different screen sizes, different processor speeds, different OEM vendors, different ways of approaching your functionality.
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
35
Ultimate Cheat Sheet
Ultimate Cheat Sheet for Mobile Testing
Do I have to worry about NDK ?
No. Developers will take care of that.
Do I need to know Java to perform Mobile test automation ?
Knowing that is good. But Its not required. Calabash scripts can be written in readable English. You read English right ?
Tell me about Telerik for iOS ?
Great record and playback tool. Free too. Worth trying for Web apps. But for native apps, calabash is best !
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
36
Ultimate Cheat Sheet
Ultimate Cheat Sheet for Mobile Testing
What is the Smart phone OS Market Coverage ?
80% of the smart phones running in Android. 15% iPhones. 4% Windows Phones. Rest of them are running (what the heck is to run ?)
Never heard about ZTE.
Go to hell. They are the 4th biggest phone makers in the world.
Can I hire you ?
Send me an Offer !
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
37
Nimbuzz !!Download Nimbuzz from www.nimbuzz.com (If you are in a PC)
Or if you have a smart phone, use your mobile browser to point http://get.nimbuzz.com (Nimbuzz will automatically find your phone and give you the software)
I want you to try this out and send your feedbacks to me !!
Join Nimbuzz Beta Testers Facebook Group and post your views…
https://www.facebook.com/groups/nimbuzz.beta/
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
07/04/2023 Go get Nimbuzz from http://get.nimbuzz.com
38
Ha Ha Ha !! I just gave a
Medal to the Medal.
Thanks for Watching
Divert your feedbacks to:http://twitter.com/senthazalravi
3907/04/2023 Go get Nimbuzz from
http://get.nimbuzz.com