the enterprise goes mobile

The Enterprise Goes Mobile

Thomas Gronbach

Tobias Dreyschultze

Keynote

February 6th 2013

© 2013 Keynote Systems

About Keynote

• Founded May 1995

• Revenue: FY2012 – 124M / Q1FY2013 – 34M

• Public, NASDAQ: “KEYN” since Sept, 1999

• 450+ employees

• 4,000 customers

2

February 8, 2013

• On-demand infrastructure of over 7,000

measurement computers and mobile devices

in more than 275 locations around the world.

• DeviceAnywhere: #1 Mobile Testing Platform

• Named by Forbes as ‘One of the Best 100

Companies in America’ with Under $1 Billion

in Sales.

Keynotes Leadership


Agenda

3

� Mobile is a TOP Priority for Enterprises

� Who are Enterprises?

� What does it Mean TOP Priority?

� What does it Mean for Mobile Development?

� Effect of TOP Priority on Mobile Development

� The (formerly Web) and now Mobile Developer

� Mobile Development Puzzle

� Enterprises adopting Mobile Testing

February 8, 2013


Some Mobile Enterprise Profiles

Main website (HTML)

Mobile-optimized website (HTML)

Tablet-optimized website (HTML)

Commercial banking app

(Native: iOS, Android, BB)

Investment app

(Native: iOS, Android, BB)

Mobile payment app

(Native: iOS, Android)

Financial Institution

Consumer Applications Internal Applications

Internal corporate website (HTML)

Mobile-optimized corporate website

(HTML)

Trading app (Native: BB)

Expense app (HTML)

Type of Company

Main website (HTML)

Mobile-optimized website (HTML)

Mobile app (Hybrid – native with some

web content: iOS, Android)

POS app (Native: iOS)

Time tracking (HTML)

Inventory tracking (HTML)

Who chooses this as a TOP Priority? Who is an Enterprise?

Retailer

February 8, 2013


What does it mean TOP Priority?

5

Enterprises Investments

� Secure Mobile Apps

� Qualitatively Sound Mobile Apps

� Reliable Mobile Apps

� Available Mobile Apps

� Mitigate Mobile Risk

February 8, 2013


What does it mean for mobile development?

6

Get S, Q, R, A, M into your work

� Good quality development

� Test before production

� Test well before production

� Test a lot before production

� Test during production

� Test before development

� Don’t forget that testing gets S,Q,R,A,M

into your work

February 8, 2013


The Mobile Developer

7

Developers are facilitator for Enterprise’s TOP Priority

� Thought leader

� Differentiated developer do things first

� Real world testing

� Real device testing

� Simple, easy access for everyone

February 8, 2013


Mobile Development Puzzle

8

Testing in a highly heterogeneous and diverse environment

QA Effort Portion of Development

Timeline

Eff

ort

(T

ota

l Ho

urs

Sp

en

t)

Traditional Products Mobile Products

Requirements/Design Development QA

� App type: native, web, hybrid

� OS variety

� Device variety

� Differing display sizes

� Differences in processing speed,

memory

� Android customizations

� Device life cycle – 12 to 18 months vs.

2 – 3 years

� Connectivity

� Network latency

� Connectivity fluctuation

February 8, 2013


Keynote’s DeviceAnywhere Platform

February 8, 2013

© 2013 Keynote Systems 10

February 8, 2013

DeviceAnywhere platform has over 1,000 devices* on all OS’ including…

• Android

• iOS

• Windows Mobile

• BlackBerry

Just announced…

• Support of iPhone 5

• – available now!

• Windows 8 – coming soon!

• BlackBerry 10 – coming soon!

*Number of devices available includes duplicate models.

Market Leader in Devices Supported


February 8, 2013

How it Works - Keynote’s DeviceAnywhere Mobile Testing Datacenter


Rack mountable

tabs

Live Mobile Device

Hardwiring Harness Device Standard I/O

Status/Service

Indicators

Power Switch

SIM Switcher (USB)Data Cable

Connection (USB)Power Server Connection

(USB)SIM Card Slot

RF Antenna

February 8, 2013

How it Works


Changing the Mobile Testing landscape

DeviceAnywhere Free

• Lets users spot check mobile

websites on real, remote

smartphones.

• Eliminates the cost of device

ownership, management, and

data plans.

• Introduces development and QA

teams to capabilities of cloud-based

testing solutions.

• No cost. No commitment.

February 8, 2013


Consumer Banking division’s QA team initially signed up for annual

subscription on shared testing environment

Later added other divisions and replaced cloud service with a licensed

dedicated enterprise system

Also added test automation for enterprise style testing of multiple test

cases integrated with their QA tools

delivering Strategy powered by having a portfolio of testing products – delivering

in both cloud and systems configurations.

Large US BankThree Years Experience in Mobile Testing

February 8, 2013


Humana

� Fortune-100 healthcare company with over 40,000 employees

� Healthcare operations in all 50 US states

� Goal

� Provides consumers and businesses with mobile access anywhere

any channel across various healthcare operations

� HTML5-based web apps, mobile websites and native mobile apps

Approach

� Technical drivers� Support emerging mobile technologies

� 150 different test cases

� Licensed 50 different devices/OS

� Business driver

� Humana corporate initiative: expand mobile access to patient information

� Develop mobile process improvements to support upcoming mobile web and applications portfolio additions

� Device flexibility supporting changing market and project requirements

� Provide reliable and expedient access to consumers’ health information whenever needed

Humana: Improving the Quality of their Mobile Apps & Websites

February 8, 2013


Solution

• Keynote’s DeviceAnywhere TCE Monitoring platform provides remote access to TomTom

devices, including its latest generation devices and the iPhone 4

• Connection to a live mobile network in France

• Housed in a data center in Paris, Keynote created a customized environment, providing

GPS connectivity supporting functionality and performance monitoring scenarios

Benefit

• TomTom’s operations receives real-time alerts about software, hardware and network

issues that could affect the end-user experience.

• Alerts are pre-defined and require no additional involvement from TomTom’s

development team

• Focus on developing the next-generation features and services

TomTom: Monitoring their

February 8, 2013


Meet Keynote and get hands-on experience with DeviceAnywhere

17

� Keynote at M-Days

� Hands-on demo at booth IN15

� Humana case study – request 3 pager for details

� Sign up for DeviceAnywhere FREE

� www.keynote.com/dafree

� Free Mobile Testing Event in Munich, February 19th 2013

� Hands-on Sessions

� Sign up today: www.keynote.com/munich

� Mobile World Congress, Feb 25-28, 2013, Barcelona

� Booth 6C71

February 8, 2013


Demo

Thank You

Thomas [email protected]

www.keynotedeviceanywhere.com

http://mobileapplicationtestingtimes.wordpress.com/

https://twitter.com/DevAnywhere

February 8, 2013

Appendix


Mobile is a TOP Priority

20

CIOs provided their feedback in a survey at Gartner’s 2012 CIO Leadership Forum, themed

Amplify the Enterprise

Mark P. McDonald, Ph.D, Group VP & Head of Research, Gartner Executive Programs – January 2012


Elements of a Test Strategy

21

DevicesDevice Rationalization

� Optimize coverage (or minimize risk)

� Control costs (time, equipment)

Testing Platform

� Real device testing platform

� Remote access to enable geographically diverse teams

� Flexible device deployment options

� Security and management

Test Automation

� When and where to automate

� Efficient automation of web, hybrid, and native apps

� Flexible scripting options to enable QA engineers with different skillsets


Real Device Testing Platform: Remote Access to Real Devices

22

What device input and output capabilities are required?

� Keys; tapping and swiping touch screens

� Opening and closing devices, connecting/disconnecting the battery, accelerometer support,

audio, backlight, camera

� Viewing the LCD with accurate pixel-by-pixel displays

Benefits

� Enables management and storage of

test assets and test results

� Collaboration environment for QA


Multiple Options for Accessing Real Devices

23

DevicesShared Devices

� 24X7 access to hundreds of smart devices

� Use for compatibility testing, “untrusted

device” testing

Private Devices

� 24X7 access to your devices

� Enables geographically diverse team

� Inside corporate firewall or external

Local Devices

� Plug your smart devices on hand directly into

your local computer

� Make use of existing assets, great for local

teams

Keynote Confidential


Approach to Rationalizing Test Devices

� Prioritize key criteria

� Choose the target # of devices

� The Device Planner generates a device list

based on device popularity, by finding devices

that provide coverage across key criteria

� Example: The Samsung Galaxy SIII is chosen

first as the most popular Android device. The

Galaxy Nexus is another popular device, but as

its OS is also Android 4.0, it moves to another

device, e.g. the LG Spectrum on Android 2.3,

to maximize coverage across OS versions.

� The Device Planner can be found at

tce.deviceanywhere.com

Device Planner: Optimized Device List

Reduce required test devices by defining key criteria (e.g. OS version, screen resolution, CPU, etc.) that will

impact the performance of your application and optimize for popular devices that provide coverage across

these criteria

Device Planner: Key Criteria

Device Planner: a free tool by Keynote


Test Automation

25

As with software QA, test automation of mobile apps both save

money and improve the effectiveness of your QA test process

� Save Money

�Automate smoke tests and regression tests to reduce cost of manual testing

� Find bugs earlier in the process when it is cheaper to fix them

� Execute more tests in less time, improving time to market and/or

increasing test coverage

�Execute 2.5 – 3X faster than manual tests

� Schedule tests to run offline; utilize up to 24 hours of each day

� Improve Quality

� Find more bugs: increase test case coverage, reduce tester fatigue

� Offload testers from repetitive tasks, allowing them to focus on product

quality (e.g. exploratory testing)

� Increase device coverage

� Address quality in an increasingly fragmented device market

� The more models you test against, the more efficiency you gain


Mobile Application Testing Stages: Where to Automate?

26

Goals

Test Cases

Frequency

Methodology

Basic acceptance,

build acceptance

Functional testing

across all existing

areas of product

Compatibility

across devices

Very high value

to automate!

High value to

automate

Medium value to

automate

# of Devices

Optimal Test

Strategy

Automated

Testing

Automated

Testing

Manual or

Automated Testing

Smoke

Testing

Regression

Testing

Compatibility

Testing

Deep testing of

new areas of

functionality

Low value to

automate

Manual Testing

New Feature

Testing

Not automated

Exploratory

Testing

Exploration of

functionality from

customer viewpoint

Manual Testing

February 8, 2013


Use Case: Automation

27

One customer had a series of 100 test cases run across one Android and one

iOS device that were required to verify each new build. Two QA engineers

were effectively completing 40 test cases per day, or requiring 1 week of

elapsed time (10 man-days)

Customer utilized DeviceAnywhere Test Center Enterprise Automation to

automate all 200 test cases, which can now be scheduled to automatically

run overnight when a new build is generated.

As a result, the customer improved time to market by 1 week, while

freeing up QA resources for other testing efforts.


Test hybrid and web applications, from a single platform, allowing you to:

• Test on real devices, ensuring test accuracy

• Enable all of your remote and local employees, with or without existing scripting skills

• Save time by automating native, web, and hybrid application testing

DeviceAnywhere Platform – Enabling Testing of Mobile Apps

THANK YOU!

the enterprise goes mobile

Technology

mobile access

mobile websites

mobile devicesin

mobile developerdevelopers

agenda mobile

mobile testing platform

mobile testing landscape

upcoming mobile web