state of mobile benchmark report - mwc 2015 edition

Mobile Experience Benchmark - MWC 2015 Edition

Introduction

2

This report provides an overview of three key aspects to consider when developing mobile apps:

1.  State of the mobile ecosystem, across operating systems and devices

2.  The digital customer experience for mobile apps, across app store categories

3.  Differences in key metrics for wireless networks, across different markets & geos

The data in this report is aggregated from Crittercism’s leading mobile application performance management (mAPM) solution that processes over 50,000 requests per second from 1 billion mobile users in over 120 countries, every day.

What’s the state of the mobile ecosystem?

3

Why the state of the mobile ecosystem matters

4

“The smarts of Android on screens big & small” “iOS 8 is the biggest iOS release ever — for developers and everyone else” Knowledge about the installed base of devices & OS inform design, feature and architecture decisions.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15

5

iOS 7

iOS 8

iOS 6

Other

iOS 8 passing 50% of iOS app load traffic in 50 days Apple ships a new OS bundled with a new flagship device, and pushes the OS update immediately to the entire installed base of devices, resulting in quick uptake.

Apple iOS, Share of App Load Traffic by Version, April 2014 – January 2015

50% share of app load

traffic

Nov 5th Sept 17th

50 days

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Apr/14 May/14 Jun/14 Jul/14 Aug/14 Sep/14 Oct/14 Nov/14 Dec/14 Jan/15 Feb/15

iPhone 6 Plus

iPhone 6

iPhone 5S

iPhone 5C

iPhone 5

iPhone 4S

iPhone 4

Other

6

iPhone 6 / 6 Plus reaching ~20% app load traffic within ~20 weeks iPhone 5S as the previous flagship device achieved largest share of a single iOS device, indicating how good Apple is at providing latest computing technology to consumers.

Apple iPhone, Share of App Load Traffic by Device Generation, April 2014 – January 2015

19% share of app load

traffic

Feb1st Sept 17th ~20 weeks / 4.5 months

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Lollipop 5.0

KitKat 4.4

Jelly Bean 4.3

Jelly Bean 4.2

Jelly Bean 4.1

Ice Cream Sandwich 4.0

Other

7 Google Android, Share of App Load Traffic by Version, April 2014 – January 2015

Android Lollipop <2% share of app load traffic, KitKat still growing Google, device OEMs, carriers provide phased upgrades to the existing installed base of Android devices, resulting in a uptake of a new Android version that’s slower by an order of magnitude compared to iOS.

<2% share of app load traffic Feb1st Nov 12th 81days

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Droid Razr HD

Motorola Moto G

Samsung Galaxy Note 2

Samsung Galaxy S3

Samsung Galaxy Note 3

Samsung Galaxy S3 Mini

Samsung Galaxy S4

Samsung Galaxy Tab 2

Samsung Galaxy S5

8

Samsung dominates Android, with S5 supplanting S3 Data indicates that owners of Samsung devices follow a 2-year upgrade cycle (e.g. from S3 to S5) along with carrier subsidies.

Android, Share of App Load Traffic by Device Type, April 2014 – January 2015

20% share of app load traffic

Nov 1st Apr 1st ~30 weeks / 7 months

OS updates can cause unforeseen problems

9

Despite extensive testing & preparation by OEMs and carriers, things can still go wrong.

Source: https://twitter.com/azkop2010/status/569312645527908352; https://twitter.com/dianaturner/status/569971859573358592

What’s the digital customer experience of my mobile apps ?

10

Why the digital customer experience for mobile apps matters

11

Badly functioning apps are the source for customer frustration: “Your app has one job: online check in, so how come it’s never available?”

Source: https://twitter.com/British_Airways/status/560238312540364800; https://twitter.com/British_Airways/status/560055280604835840

12

iOS 8 crash rates stabilized within a single quarter Initial iOS 8 crash rate of 3.3%, dropping to 2.2% and converging with crash rates of iOS 6 (2.2%) and iOS 7 (2.0%). Improvements due to Apple quickly providing OS updates and developers fixing errors & bugs.

1.5%

1.7%

1.9%

2.1%

2.3%

2.5%

2.7%

2.9%

3.1%

3.3%

3.5%


iOS 8

iOS 6

iOS 7

iOS Crash Rates by Version, April 2014 – February 2015

Crash rate declines from 3.3% to 2.2%

Dec 14th Sept 17th 12 Weeks

1.2%

1.4%

1.6%

1.8%

2.0%

2.2%

2.4%


13

Anatomy of an iOS release cycle: Maturity of “old” OS Crash rates are continuously declining as developers get more acquainted with the installed OS, continuously improving the quality of their apps.

iPhone 4

iOS 7 maturing, with crash rates continuously going down and reversely the quality of the

in-app experience goes up

iPhone 6 plus

iPhone 5S

iPhone 5C

iPhone 6

iPhone 5

iOS Crash Rates by Device, April 2014 – February 2015

iPhone 4S

1.2%

1.4%

1.6%

1.8%

2.0%

2.2%

2.4%

Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14 Jan-15 Feb-15

14

Crash rates spike following the introduction of a new OS version New OS features – while tested - are breaking apps, as end points are hit in a new way. Developers need at least two new builds to fix errors, implying two App Store review cycles with ~20 days of wait time.

iPhone 4

iOS 8 launch on Sept 2014

iPhone 4S

iPhone 6 plus

iPhone 5S

iPhone 5C

iPhone 6

iPhone 5


1.2%

1.4%

1.6%

1.8%

2.0%

2.2%

2.4%

Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14 Jan-15 Feb-15

15

New cycle of developer productivity starts, driving down crash rates Quality of apps goes back up as Apple stabilizes the OS and developers continuously fix errors and optimize the delivery of their apps. Dev efforts are also shifting away from older devices such as iPhone 4.

iPhone 4

iPhone 6 plus

iPhone 5S

iPhone 5C

iPhone 6

iPhone 5

Developers getting more acquainted with the new OS, creating higher quality apps, but also moving on from older hardware platforms in

the case of iPhone 4


Devs moving on from iPhone 4

iPhone 4S

16

App Store category crash rates Apps that consume a lot of resources across the stack (e.g. CPU, GPU, bandwidth) like Photo & Video and Games tend to crash more frequently than other categories.

2.5% 2.4%

2.1%

1.8% 1.8% 1.8% 1.6%

1.5%

Photo & Video Games Social Networking

Health & Fitness

Business Entertainment Lifestyle News

0.05% 0.02%

Best-in-class apps operate within a crash rate band of 0.02-0.05% on both iOS and Android

AppStore Crash Rates by Category – December 2014

5.3%

3.4% 3.3% 3.3% 2.8%

2.5%

1.8% 1.7%

Entertainment Business Photography Shopping Games News & Magazines

Social Health & Fitness

17

Google Play category crash rates On average higher crash rates per category than for the App Store, same phenomenon for resource-intensive apps like Entertainment and Photography.

0.05% 0.02%

Best-in-class apps operate within a crash rate band of 0.02-0.05% on both iOS and Android

Google Play Crash Rates by Category – December 2014

What are key differences for wireless networks across different markets & geos?

18

Source: https://twitter.com/wuhelp/status/539823712766144512

Why networks & geographies matter for mobile apps

19

Carrier-enforced firmware for Android devices can cause an app to drain battery life; network settings can lead to unexpected consequences for the availability of mobile apps: “Yes, it does work on wifi but does not work over AT&T’s network.”

20

France

€26

€17

€23

€23

Orange

SFR

Free

Bouygues

274ms

293ms

262ms

250ms

Wireless Carriers Ranked by Speed

ARPU Carrier Avg. Latency

Relative Split between iOS & Android

iOS Android

1.

2.

3.

4.

37%

35%

41%

40%

63%

65%

59%

60%

Bouygues scores best on latency among all French carriers, with the highest ARPU. France has the fastest networks and with it the highest ARPUs in the Eurozone countries.

= highest ARPU in a market Split between iOS and Android calculated based on app usage / traffic.

21

Germany

€15

€14

€11

€9

O2

E-Plus

T-Mobile

Vodafone

320ms

448ms

317ms

300ms




iOS Android

1.

2.

3.

4.

45%

44%

69%

77%

55%

56%

31%

23%

Vodafone with the fastest network and the highest ARPU; E-Plus by far the slowest, with lowest ARPU


22

United Kingdom




iOS Android

£19 / ~€26

£18 / ~€24

£19 / €26 EE

Vodafone

O2 1.

2.

3. 347ms

344ms

327ms 82%

87%

87%

18%

13%

13%

O2 with the fastest network and the highest ARPU. All carriers with highly similar OS split and latency metrics, leading to similar ARPUs.


23

Italy

€11

€12

€13 Vodafone

TIM

WIND

343ms

339ms

282ms




iOS Android

1.

2.

3.

7%

55%

55%

93%

45%

45%

No clear correlation between low relative latency / high relative ARPU.


24

Spain

€15

€13

€18

€15

Vodafone

Movistar

Yoigo

Orange

358ms

431ms

334ms

314ms




iOS Android

1.

2.

3.

4.

8%

23%

32%

26%

92%

77%

68%

74%

No clear correlation between low relative latency / high relative ARPU; Vodafone with highest ARPU and highest share of iOS usage.


25

United States

$44

$49

$55

$42

45%

44%

69%

77%

55%

56%

31%

23%

Verizon

AT&T

Sprint

T-Mobile

356ms

362ms

340ms

327ms




iOS Android

1.

2.

3.

4.

AT&T with the slowest network and the lowest ARPU in the market.


Mobile Experience Benchmark - MWC 2015 Edition