enhancing facebook app experience

AN ANALYSIS OF NETWORK AND APPLICATION RESEARCH, TESTING AND OPTIMIZATION IN THAILAND

ENHANCING FACEBOOK APP EXPERIENCE

In collaboration with:

contents

Page 3. Executive summary

Page 5. App experience is everything

Page 6. Measuring quality engagement

Page 8. Applying the methodology to the DTAC network

Page 9. Putting the test into practice

Page 12. Results

Page 14. Conclusion

Page 16. Glossary

ENHANCING FACEBOOK APP EXPERIENCE • CONTENTS 2

In today’s data-hungry world, a positive app experience on smartphones is central to user satisfaction.

Greater smartphone use has led to a growth in data revenue, in many operators exceeding voice revenue. This makes developing a quality app experience important for developers, operators and users alike; yet measuring this experience within data-centric applications is no easy task – and, if it is difficult to measure, how can it be improved?

Networks have traditionally been tuned to improve voice services and large video/data downloads using established key performance indicators (KPIs). As networks have matured, they have adapted to primarily deliver data services; but, the current network enhancement approach does not address mobile application performance, and data and network signaling needs are different.

In particular, quantifying the success or failure of a user’s app experience is much more complex than with traditional voice services – which either successfully connects and maintains a call, or does not.

DTAC, one of the leading network operators in Thailand, recognized the need to address and optimize user app experience. With the aim of becoming the number one digital brand in Thailand by 2020, DTAC approached Ericsson and Thailand’s most popular social media platform, Facebook, to focus on how the Facebook user experience could be optimized on mobile devices.

ENHANCING FACEBOOK APP EXPERIENCE • EXECUTIVE SUMMARY 3

Executive summary

DTAC in focus

As of Q4 2015, the company has 25 million mobile network subscribers in Thailand

2G, 3G and 4G services are available nationwide (as of Q4 2015)

15 million subscribers use smartphones and are active data users

Facebook is the most-accessed social media platform on the network

Using a methodology developed by Ericsson and Facebook, typical Facebook interactions were triggered from Facebook test accounts and measured on different mobile test devices in central Bangkok; an area chosen for its very high traffic load, due to a high number of data subscribers.

The partners correlated user-centric KPIs from the Facebook application with network performance KPIs, in a way that made it possible to understand the perceived user experience in various network conditions.

After adjustments to the DTAC network, results showed improvements to the Facebook app experience during ‘busy hour’. Within that scope, time-to-content improved up to 54 percent and time-to-share up to 63 percent respectively. This means that Facebook users were not only able to get content from their friends faster, but could also share their experiences within a quicker time period.

Many of the changes have systemic implications that can improve performance across the entire DTAC network, and for all Facebook mobile users. As a direct result of the optimization, the experience of using Facebook services for DTAC’s users was significantly improved.

This model can be replicated to monitor, analyze, measure and improve the app experience across any mobile network. Consequently, this will help operators cost-effectively target end-to-end network improvements for the greatest impact on user satisfaction, loyalty and retention.

After adjustments to the DTAC network Facebook users were able to get their friend’s content and share experiences faster

ENHANCING FACEBOOK APP EXPERIENCE • EXECUTIVE SUMMARY 4

DTAC network adjustments

A user’s experience with mobile apps, together with their ability to make voice calls, is the most important aspect to consider when determining the performance of a mobile network.

There is a huge difference between how an operator should optimize a voice-only mobile network, designed fifteen years ago, compared to now, where there are many different types of data and services. A voice call is one service – measuring and planning what targets to have is fairly simple – but in today’s data networks, with many services and needs, there are different and more complex requirements on the network.

To address this, Ericsson, Facebook and DTAC joined forces to optimize app experience, particularly on Facebook – the most popular social network for DTAC subscribers. The first kick-off meeting was held in February 2015 in Bangkok, and the objectives of the project were aligned. In May, Ericsson started to measure the Facebook app experience on the DTAC network. This was carried out by using a test agent developed by Facebook, which was then deployed on different devices that triggered ‘typical’ Facebook interactions. Hundreds of thousands of measurement samples were collected by the test agent based on the key metrics used to define app experience.

Ericsson and DTAC also collected network metrics in the areas where the test agents were deployed. Using this data, engineers correlated both the network data and the end-to-end metrics to identify the key bottlenecks that impacted the Facebook app experience. Based on these findings, actions were then taken to remove these bottlenecks and this resulted in substantial improvements for the Facebook app experience.

This highlights that it is possible to increase app coverage and improve app experience using existing network resources.

In today’s data networks there are different and more complex requirements on the network

App experience is everything

ENHANCING FACEBOOK APP EXPERIENCE • APP EXPERIENCE IS EVERYTHING 5

15 YEARS AGO TODAY

Measuring quality engagement

ENHANCING FACEBOOK APP EXPERIENCE • MEASURING QUALITY ENGAGEMENT 6

The usability of an app can have a very direct effect on user satisfaction.

If something goes wrong, subscribers are likely to blame the network operator – even if there isn’t a network issue. This, in turn, influences everything from brand reputation and customer loyalty, to the ability to generate sufficient return on investment.

For both the content provider and the mobile operator, it is crucial to have an experience that is responsive and fast, and to have services that work well throughout the chain. Research shows that time-to-content is a crucial metric and a has direct correlation with satisfaction and loyalty to mobile operators. If it takes too long, it could tarnish the user’s perception of the network coverage, causing stress, negative opinions and even complaints against the operator.

Added to this, most apps (including WhatsApp, Instagram and YouTube) are encrypted; this means that information required to assess the quality of user experience is going directly to the app provider, and not necessarily to the operator.

Apps, smart devices and mobile networks – as well as the companies and developers that make them – together form a single ecosystem that drives most of the growth. Within this ecosystem, apps and smart devices are continuously evolving and being upgraded, and this places high demands on the operator’s network coverage, capacity and load.

Data in the network originates from a wide spectrum of fast-evolving apps, both local and global, and this requires operators to make continuously updated decisions about service levels. At the other end, app developers face a similar uncertainty, as they must design their applications with a myriad of devices, platforms and network types – as well as performance levels – in mind. This can challenge application developers like Facebook, given that they often provide a visually-appealing experience with real-time content and large-scale photos.

Such instances demand an end-to-end network view, as traditional network statistics alone may not reflect the actual user experience. It’s tricky to understand if a user is happy or not, as millions of other processes are happening within the network simultaneously. Expanding app coverage is also central to helping billions of people get connected for the first time.

Source: Ericsson Mobility Report MWC edition (February 2016)

Heart rate increases 38% with mobile content delays

Internet.org

Internet.org is a Facebook-led initiative with the goal of bringing internet access and the benefits of connectivity to the two-thirds of the world that do not have it. Both independently and as members of Internet.org, Facebook and Ericsson have made it a priority to help operators tune and optimize their networks. This covers improvements in app experience and more efficient data delivery, as well as providing scaled-down apps that work well in network-constrained and price-sensitive environments. Examples of their common efforts include founding Internet.org with other partners in 2013, a related Innovation Lab launched in 2015, as well as a series of hackathons aimed at improving app performance in network conditions found in many emerging markets.

Yet only so much can be achieved under laboratory conditions. Measuring app experience effectively requires seeing how apps work – on real devices, in real networks, and in a variety of real-time conditions. With this in mind, Facebook, DTAC and Ericsson agreed on a joint project to discover the real user app experience across the mobile subscriber base, as well as any associated network limitations, wherever and whenever these occurred.

ENHANCING FACEBOOK APP EXPERIENCE • MEASURING QUALITY ENGAGEMENT 7

Applying the methodology to the DTAC networkTo better understand and ultimately optimize the Facebook experience in DTAC’s live commercial network, a series of objectives, methodologies, measurements and subsequent improvements were introduced.

Within the scope of the project, Ericsson and Facebook also aimed improve the quality of the user’s app experience by quantifying network optimizations made.

For this framework, the primary practical activities involved:

Placing high-end and low-end devices across different geographic areas and throughout different times of the day to judge positive and negative user experiences

Applying the methodology to monitor Facebook user KPIs, and then correlating this with network statistics to identify bottlenecks

Analyzing and improving the Facebook experience in central Bangkok

Starting in July 2015, Ericsson applied a series of metrics that considered the end-to-end network experience impacting app experience, or, in other words, the key measurable aspects that could go right or wrong in the network when using an app.

The methodology framework

ENHANCING FACEBOOK APP EXPERIENCE • APPLYING THE METHODOLOGY TO THE DTAC NETWORK 8

Define KPIs, test cases and

test tools

Execute test cases

Execute improvement

plan

Analyze improvements

Preparations for measurements

Analysis and definition of

improvement plan

Repeat test cases

Prepare final report

Putting the test into practice

ENHANCING FACEBOOK APP EXPERIENCE • PUTTING THE TEST INTO PRACTICE 9

The methodology applied by Ericsson and Facebook

1. Defining Facebook service KPIs, including time-to-content, time-to-share and Facebook app experience

2. Preparing the measurement set-up, including test tools, network traces etc.

3. Executing the test case

4. Analyzing and defining an improvement plan, including analysis of end-to-end bottlenecks and proposed areas to optimize

5. Carrying out the improvement plan in a small area of the operator’s network

6. Repeating test cases

7. Analyzing improvements and implementing them where possible

Specifically for this project, the Android-based test agent, which had the same networking stack as the commercial Facebook app, was used to simulate app traffic. It triggered typical Facebook use cases (test cases) and reported user-centric KPIs for a better understanding of the Facebook user experience. The test agent also generated additional data that could be correlated with DTAC network statistics, in order to determine the link between user experience and network performance.

Standard use cases included feed download, single and multi-picture download and single picture upload – all actions that are performed on the commercial app hundreds of millions of times a day.

The test agent was then loaded onto different types of smartphones – high-end models and low-end ones. Both were powerful enough to eliminate device limitations as factors in the testing, while still enabling clear identification of any device-driven differences in user experience.

Facebook test agentAgent Facebook app Autonomously emulates Facebook user activity 24/7

Real Facebook app

Facebook networking stack

QoE reporting functionality

PREPARATION

Using the developed test agent, Ericsson and DTAC prepared the network by tracing in the radio access network (RAN) nodes, along with network probes for analysis of user plane behavior, among other actions. In addition, for each network measurement, the test agent data was tagged with a number of parameters including time, device and which cell it used.

With two dozen high and low-end handsets placed in different strategic environments (such as loaded sites, less-loaded sites and attached to macro sites), Ericsson was then able to access this information and post-process it with data from the mobile network to spot patterns and instances. For example, within the eight locations, if the service was poor in one location but the others were OK, then it was noted that the problem was not with the Facebook server or the core network. Potential issues could be included or excluded from the study based on observing correlations.

A number of different locations were chosen throughout the central Bangkok area, including sites close to the base station. Mobility tests – involving physically driving the devices around Bangkok – were also conducted to cover a larger geographical area. Each location was carefully considered with DTAC to maximize the effectiveness of the study on both 3G and 4G networks.

Problem is not with the Facebook server or the core network


PERFORMANCE

When the test agent showed a poor user experience, it was then possible to analyze the bad sample and identify where in the network the issues seemed to occur (such as in the RAN, core or servers) and then dig deeper based on that information.

With this in place, Ericsson and Facebook were able to correlate user-centric KPIs reported by the test agent with network KPIs. This allowed them to develop an outside-in troubleshooting approach – working from actual user experience, not just network data.


The approach can be summarized as follows: Understand the user experience – compare Facebook user

performance data from different devices, geographical locations and time of day in order to understand positive and negative user experiences

Drill down on negative samples – analyze probe data to understand where in the network problems occur and analyze agent data to see which types of transactions go wrong; for example, transport layer security setup or object transfers

Focus on the network elements – detail statistics and configurations of network nodes and elements to pinpoint bottlenecks and issues

Identify key bottlenecks – define which key bottlenecks and improvement areas have the highest impact on user experience.

Overall, the key focuses were the total time taken to download a feed, image or multiple images, and the time taken to upload an image.

The analysis phase took a couple of months, but by November 2015 the team had a clear picture of what the issues were and what actions needed to be taken. After executing the network optimization activities, the same tests were carried out again to compare the ‘before’ and ‘after’ results. This highlighted that the project had improved user app experience significantly.

Results from the tests illustrated the impact of device selection and traffic patterns, as well as the correlation between Facebook experience and mobile network traffic load. System-wide bottlenecks and other observations in the RAN and Facebook servers were also identified.

There were some locations where app experiences were consistently poor because of network capacity, coverage and configuration constraints. For instance, if the time-to-content exceeded three seconds during ’busy hour’, it was often because the RAN was working close to maximum capacity.

Results

ENHANCING FACEBOOK APP EXPERIENCE • RESULTS 12

13-34 percent improvement in app experience during ‘busy hour’

54 percent improvement in time-to-content

63 percent improvement in time-to-share

60 percent improvement in Facebook upload and download time

This was particularly the case on the 3G network and on the low-end handsets.

By tuning the network more efficiently and maximizing resource on the uplink and downlink, user performance was improved across the radio network – not just for Facebook users.

After the adjustments were carried out, there were dramatic enhancements in performance on the Facebook test agent.

In order to have the greatest possible impact on user experience, many improvements were focused on fixing the worst performing scenarios. For instance, for the worst 10 percent of test cases, time-to-content was lowered from 32 seconds to 11.4 seconds.

It’s vital that the user has the same experience on their mobile as they would on their desktop via Wi-Fi, so improving the network operations helped deliver that goal. In fact, the success of the study meant that more Facebook users could be served from the same mobile network node, without having to build additional infrastructure and hardware.

For the worse 10 percent of test cases, time to content was lowered from 32 to 11.4 seconds

ENHANCING FACEBOOK APP EXPERIENCE • RESULTS 13

Average Worse 10 percent0

Sec

on

ds

32 seconds

11.4 seconds

5

10

15

20

25

30

35

Conclusion

ENHANCING FACEBOOK APP EXPERIENCE • CONCLUSION 14

The Bangkok region has some of the most active mobile internet users and data subscribers in Southeast Asia. As such, the network is expected to deliver quality user experience – particularly with the high penetration of smartphones.

However, to date, there has not been an adequate methodology – including KPIs, tools and correlation methods – for measuring quality user app experience on data-centric applications; plus, networks in fast-growing emerging markets are often particularly challenged when it comes to data performance.

Ericsson, Facebook and DTAC came together to analyze the real user experience and its relation to network capabilities. Facebook and Ericsson applied a unique, innovative methodology that helped to identify key issues and bottlenecks that affect user metrics; such as time-to-content within the Facebook app.

The bottlenecks in the network were identified by correlating user-centric KPIs with network KPIs. This kind of intelligent correlation and end-to-end understanding of the network then made it possible to understand the user experience in various network conditions.

By addressing these issues, the user experience of the Facebook app in the DTAC network was improved substantially, with quality app experience during ‘busy hour’ rising by 13 to 34 percent. Many of the changes have systemic implications that could improve performance across the entire DTAC network, and also for all Facebook’s mobile users.

MOBILE NETWORK METRICS

APP METRICS

IMPROVED APP COVERAGE

Innovative methodology

As part of Internet.org’s mission to bring affordable internet access to the two-thirds of the world not yet connected, Facebook and Ericsson, in conjunction with DTAC, applied a methodology to correlate user experience with network performance in order to do this.

DTAC now has the ability to focus network improvements on the areas that impact its users most, and this cost-effective approach to enhancing network performance positions the company as a data leader within its market. Facebook was able to better understand the many network variables that can impact user adoption, which is particularly critical as Facebook and Ericsson work to connect hundreds of millions, if not billions, of unconnected people to the internet.

Finally, Ericsson has enhanced a model that can now be applied to any mobile network. This will help operators cost-effectively target network improvements in the areas that will have the most impact on user satisfaction, loyalty and retention.

With up to a 60 percent improvement on both upload and download time, Facebook users on the DTAC network can now benefit from an optimized user experience.

ENHANCING FACEBOOK APP EXPERIENCE • CONCLUSION 15

Up to a 60 percent improvement on both upload and download time

GLOSSARY

App coverage The geographical area in which an app

works as the network operator intends

App experience The perception a user has when using

an application on their mobile device

Bottleneck A situation that causes delay in a process or system

End-to-end Including all stages of a process

RAN Radio access networks

Time-to-content Time taken for content to load on a user’s device

Time-to-share Time taken for users to share content with other users

ENHANCING FACEBOOK APP EXPERIENCE • GLOSSARY 16

DTAC

Established in 1989, DTAC is the second-largest mobile operator in Thailand.

The company offers mobile services to its customers via contract and pre-paid tariff plans, with over 25 million subscribers registered as of Q4 2015. Currently, dtac offers customers a comprehensive range of 2G, 3G and 4G services on the 850 MHz, 1800 MHz, and 2100 MHz bands.

Since 2001, DTAC has been partly owned by the leading Norwegian telecommunications company, Telecom Group.


Facebook

Facebook is one of the world’s most popular social media platforms, with over 1.7 billion active monthly users and counting. It’s widely used to connect with friends and family, share experiences and to discover what’s happening around the world.

Launched in 2004 by Mark Zuckerberg, Facebook was originally designed as a way for college students to connect with each other online. Today, it is a publicly-listed company with more than 14,000 employees. Facebook is available across the globe to anyone aged 13 years or older.

Facebook Inc. has also acquired a number of other companies, including Instagram, Oculus and WhatsApp.


Ericsson

Ericsson is the driving force behind the Networked Society – a world leader in communications technology and services. Its long-term relationships with every major telecom operator in the world allow people, business and society to fulfill their potential and create a more sustainable future.

The company’s services, software and infrastructure – especially in mobility, broadband and the cloud – enable the telecom industry and other sectors to do better business, increase efficiency, improve the user experience and capture new opportunities.

With approximately 115,000 professionals and customers in 180 countries, Ericsson combines global scale with technology and services leadership. Its support networks connect more than 2.5 billion subscribers; 40 percent of the world’s mobile traffic is carried over Ericsson networks, and investments in research and development ensure that its solutions – and customers – stay in front.


Internet.org

Internet.org is a Facebook-led initiative with the goal of bringing the internet to the two-thirds of the world’s population that do not have it.

This effort will require two key innovations:

Bringing down the underlying costs of delivering data

Using less data by building more efficient apps

If the industry can achieve a tenfold improvement in each of these areas – delivering data and building more efficient apps – then it becomes economically reasonable to offer free basic services to those who cannot afford them, and to begin sustainable delivery on the promise of connectivity as a human right.

To make this a reality, Internet.org and the rest of the industry, need to work together to drive efficiency gains across platforms, devices and operating systems. Creating more efficient technologies will speed up the rollout of more sophisticated technologies, which provide higher-quality experiences to more people in developing countries, while also enabling the industry to continue growing and investing in infrastructure development. Working together toward this common goal, the aim is to achieve shared learnings and advances that move the industry and society forward.

1/213 31-FGC 101 0619

© Ericsson AB 2016

Ericsson

SE-164 80 Stockholm, Sweden

Telephone +46 10 719 0000

www.ericsson.com

enhancing facebook app experience

Documents