ian smith - mobile software testing - facing future challenges

Unissons nos Talents

T O G E T H E RT A L E N T E D

1Mobile Software Testing – Facing The Future Challenges - EuroSTAR 2008 (R1)

Mobile Software TestingFacing The Future Challenges

Ian F. Smith Principal Consultant [email protected] Group Ltd. Tel: +44(0)131 332 3311

Robert Campbell Senior Developer [email protected] Group Ltd. Tel: +44(0)1698 574 200

2Mobile Software Testing – Facing The Future Challenges - EuroSTAR 2008, 12/11/2008

Overview

Introductions

Mobile Systems

Trends

Challenges/Problems

Approaches

Testing Mobile Systems

Example/Case Study

Summary


Introductions

Ian F. Smith Principal Consultant – Testing Division Over 28 years experience in software and system development and testing Strong background in telecoms and real-time/embedded systems

Robert Campbell Senior Developer – Testing Division 28 years experience in embedded development, manufacturing and test Focussed on testing telecoms monitoring equipment for the last 6 years

Sopra Group Sopra Group, a major European IT Services Company Total of 11,000 staff and group 2007 Revenue of 1 Billion € Offices throughout UK and Europe + and a fully integrated offshore

Development and Testing Centre in Noida, Delhi Services:

Application Outsourcing, Systems Integration, Testing + Specialist services (Consulting, Information Security, Business Intelligence)

Key markets: Finance, government, retail / manufacturing, energy & utilities, health,

telecoms + media


Mobile Systems

What’s a typical mobile system? Various types of mobile system User operates mobile devices Supports access/operation in various locations and can often connect by various methods

Key Characteristics of Mobile Systems Elements:

Mobile Devices

Wireless/Wired connection

Server + Network Infrastructure

Mobile applications Can run on either device or server, depending on architecture and device capability

More internet based apps/services are being now provided for mobile use

Increasingly, core business systems are providing mobile application interfaces


Mobile Systems - Examples

Navigation/Route Planning Mobile Application Server: Navigation/Route Planner Handset software: Navigation Handset Client Content: Route information Access Methods: GSM/3G

Email Mobile Application Server: Email Server Handset software: Email Client Software Content: Email information Access Methods: Desktop, Laptop, GSM, WiFi, …

Voip Service Mobile Application Server: Voip Network System Handset software: Handset Voip Features + Agent software Content: Voip calls data + SMS Access Methods: WiFi, 3G


Trends

Some key stats 1.28 billion mobile handsets to be sold in 2008

[Source: Gartner]

32.2 million smartphones sold in Q1 2008 [Source: Gartner, May 2008]

Better support for data (networks and devices)

3G data services WiFi

Increasing power and sophistication of mobile devices

More powerful processor and storage Software capabilities Connectivity Audio/Camera/Video Sensors/Accelerometers GPS

Application trends Web 2.0 applications Increasing expectations

Desktop applications on mobile devices Device capabilities

e.g. LBS


Trends (Cont.)

Handsets gaining laptop/PDA capabilities Processing, storage and software

Mobile OSs Established players …

Symbian, Windows Mobile, RIM BREW, Linux Mobile, …

New Players Apple, OHA/Android, LiMo, …

Landscape is changing Symbian Foundation Open source Possible consolidation??

Development Environments Varied environments

Open source Proprietary

IDEs/SDKs Improving (but still with limitations)

Server application development


Challenges/Problems

Cellular network services can vary significantly Data offerings Performance

Mobile device platforms Many different software models + SDKs/IDEs/Toolsets Changing platform landscape (Symbian Foundation, Android, iPhone, …) Device specific firmware issues

Application complexity Meeting expectations for current desktop based applications on mobile Application/content access

Access Diversity Ubiquitous access …

GSM/UMTS WiFi/WiMax Wired

Security Trust models and signing Increasing amount of sensitive data on devices


Approaches

Manage Complexity Application and system architecture Component partitioning

Divide and conquer Multi-level testing (unit, component, integration, system levels) Get value from unit testing

Layer UI to simplify testing and support automation

Maximise code reuse Important for multi-platform developments Different platform models can limit this

Consider signing testing or manufacturer specific testing Signing can be expensive and time consuming


Approaches (Cont.)

Using emulation Doesn’t address all testing needs (telephony, performance) Combine with GUI based test automation tools

Device Testing Confirm compatibility (device firmware issues) Cover device specific features

Security Testing Threat models change constantly, so review security testing regularly

Consider full operational lifecycle Deployment, operation, upgrade, removal


Building Mobile Systems

Choosing platforms Implications

System Architecture DfT – Design for Test Create test support features

Handset software Engine (portable) + (thin) UI layer Different software models Application signing

Consider application efficiency Power efficiency/management is a major factor for mobile systems

Development environments SDKs are getting better, but building and testing will still be challenging

Design for offline use Connectivity is improving – but devices won’t always be connected Also, some OSs don’t support application concurrency, so consider this


Testing Mobile Systems

Planning testing Server and core system tests Mobile device software testing Using “devices” in system testing

Design testing to cover different handset platform needs Aim for reuse, where practical

Server testing Existing web or other test tools may be used as mobile device client

capabilities approach those of desktop client systems.

Network & Infrastructure testing Understand provider capabilities and limitations

Tools Consider whether additional support tools can be procured/created

to assist in testing (set-up, reporting, diagnosis, etc).

Emulators Can support application level testing Not good at emulating telephony or real-time operations Combine with GUI test tools


Automating Testing

Quick wins

Unit test can be well automated Continuous Integration tools Even some handset code!!

Automating device testing This can be tough – different environments, user interfaces, etc. Design UI to support automation (allow virtual user actions driven by tools)

Test Automation Tools

Performance Testing Automation is essential to effective performance testing Consider this in design-for-test activities.

Skills


Example

Webraska Navigation System Handset based navigation system

Route information from server

GPRS connection

Bluetooth GPS Handset software testing project Evaluate handset software

Quality

Performance

Usability


Case Study

Mobile Call Generator Project Develop proof of concept test call generation system Key project objectives:

Automated control of voice calls, messaging and data functions Scalable solution Remote control

RemCall System Architecture Software

Handset Server(s)

Development Challenges Achievements Limitations


Case Study (Cont.)

Multi-level Testing Unit, Component and System level test

Handset Agent Software Piloted automation for unit testing Introduced CI with CruiseControl

Handset Controller PC Software

Quality Center Server Software


Summary

Mobile systems offer many major opportunities but developing and testing them pose significant challenges.

The pressure to supply mobile access to existing business applications and systems is growing as device capabilities improve to support these.

New and existing businesses are bring innovative mobile systems to market.

The handset landscape is changing rapidly, with established players evolving and new entrants joining. This change will continue.

Consider platforms carefully – development implications and future market.

Architecture and component partitioning can have a marked effect on system development and testability.


Questions

?Thank You

www.sopragroup.co.uk

ian smith - mobile software testing - facing future challenges

Software

mobile handsets

mobile use

3g6mobile software testing

mobile systems whats

testing telecoms

typical mobile system

email server handset

testing centre