- 1 -© FTW 2014

Assessing and quantifying the impact of network bandwidth fluctuations on Web-

QoEAndreas Sackl1, Pedro Casas1, Raimund

Schatz1, Werner Wiedermann2, Ralf Irmer3

1) FTW Telecommunications Research Center Vienna,

Austria

ETSI TC STQ Workshop on Telecommunication Quality beyond 2015 21-22 October 2015, Vienna, Austria

3) Vodafone Group Research &

Development, UK

2) Telekom Austria Group, Austria

- 2 -© FTW 2014

Introduction

t

Throughput

t

Throughput

constant fluctuations

Resulting Web QoE(well explored)

Resulting Web QoE?

Several (open) research questions: Influence of various fluctuation patterns on Web QoE (Sackl 2014 et al.) Identify KPIs regarding fluctuations Influence of single bandwidth outages on Web QoE How to model throughput fluctuations

Research trend

- 3 -© FTW 2014

Web QoE User Study

Task 3: Browsing a Photo Gallery

Task 1: Browsing a News Site

Task 2: Browsing Google Maps

Web QoE User Study (N=29)

- 4 -© FTW 2014

Single Bandwidth Outages – Results (1)

MOS (Outage Duration) fitting curve

MO

S O

utag

e A

nnoy

ance

notannoying

veryannoying

Photo Gallery News Site

MO

S O

utag

e A

nnoy

ance

e.g. 2 sec. outage

Thro

ughp

ut

30 seconds

Web Usage

- 5 -© FTW 2014

Single Bandwidth Outages – Results (2)

High acceptance rate

50% detection rate

Detection rate is content-dependent

- 6 -© FTW 2014

Conclusions for Single Outages

For Web browsing, outages of 2 or more seconds already impact perceived user quality, whilst outages of up to 4 seconds are still considered “acceptable” by most users

Outages in Web Scenarios are not always detected by the users

The impact of single outages are content-dependent

- 7 -© FTW 2014

Goal & Evaluated Patterns

Pattern P1 Pattern P2 Pattern P3

DBW = Downlink BandwidthADBW = Average Downlink Bandwidth

Progressive outage/recovery e.g. tunnels

Fast changes e.g. heavy loaded cells or interference sources

like trees

High/low profile with fast short-scale

variations

Goal: Identify KPIs of throughput fluctuations to predict resulting MOS values

- 8 -© FTW 2014

Approach (1)

ADT = Average Download Throughput (simple approach), CF = Correcting FactorEADT = Effective Average Download Throughput (novel approach)

MOS = 0.45∗log (DBW)+2.48

Common, non-fluctuations approach i.e. creating simple model from user ratings and constant bandwidth conditions

MOS = 0.45∗log (ADT)+2.48

MOS = 0.45∗log (EADT)+2.48 ✔

✖

(constant throughput)

fluctuation pattern

- 9 -© FTW 2014

Approach (2)

How to calculate the Correcting Factor CF? => 5 models possible

LTD model SLTD model JT model AREA model DOUBLE model

How to get EADT? => EADT=CF x ADT

CF = 1-(t1+t2)/t0

Threshold = ADT/2

- 10 -© FTW 2014

Results of Model Verification (for Google Maps)

Pattern MOS ADT

P1 2.9 5.9 Mbps

P2 3.4 5.1 Mbps

P3 3.1 4.3 Mbps

MOSADT

3.3

3.2

3.1

MOSLTD MOSSLTD MOSJT MOSAREA MOSDOUBLE

3 3.2 3.3 3.2 3

2.9 3.2 2.9 3.1 2.6

2.5 3.1 2.8 3.1 2.3

Directly related to Patterns i.e. most

accurate MOS

Calculated from the pattern

Utilization of fitting curve: MOS = 0.45∗log (ADT)+2.48(Note: fitting curve is extracted from constant BW conditions)

Simplest model (still quite good!)

EADT=CF x ADT

Depends on the model

With the resulting EADT, the MOS value is calculated via the fitting curve MOS = 0.45∗log (EATD)+2.48

- 11 -© FTW 2014

Conclusions for Fluctuation Models

We presented first approach to model throughput fluctuations 5 Models to calculate Correcting Factor CF

- LTD: Considering time below certain throughput threshold- SLTD: Like LTD, but short drops are ignored- JT: Sliding Window- AREA: how long below threshold and how deep- Double: Considering two thresholds

Best fitting Model depends on specific pattern In our study, MOSADT leads to good results Evaluated patterns

might not be optimal for evaluated services More evaluation is needed

- We examined three patterns more types and/or more realistic patterns e.g. from real network traces

- We examined Google Maps more services- 5 ways to calculate CF find more appropriate ways

ReasonableKPIs

- 12 -© FTW 2014

Thank you for your attentionsackl@ftw.at

A. Sackl, P. Casas, R. Schatz, J. Lucjan, and R. Irmer. Quantifying the Impact of Network Bandwidth Fluctuations and Outages on Web QoE. In Proc. International

Workshop on Quality of Multimedia Experience, Messinia, Greece, 2015.

- 13 -© FTW 2014

Backup

- 14 -© FTW 2014

Single Bandwidth Outages – Study Design

Evaluated outage durations: 2, 4, 8, 16 seconds Duration of each conditions: 30 seconds (in contrast to

common 2 minute duration) Each outage duration was evaluated three times per User

(reason: short, single outages might be missed)

e.g. 2 sec. outage

Thro

ughp

ut

30 seconds

Web Usage +Questionnaire

about subjective

quality perception

3x per user for each outage duration

- 15 -© FTW 2014

Single Bandwidth Outages – Results (3)News Site

Photo Gallery

Three (short) iterations are feasible and reasonable in the context of Web QoEstudies

- 16 -© FTW 2014

Results of Model Verification

- 17 -© FTW 2014

Approach

ADT = Average Download Throughput (simple approach)EADT = Effective Average Download Throughput (novel approach)CF = Correcting Factor

How to get EADT? => EADT=CF x ADT

- 18 -© FTW 2014

Study Design – Overview & Tasks

Empirical Web QoE user study at FTW´s iLab 29 users (13f/16m) Mean age: 34.7y Median age: 31y 70% students, 43% employees

70% of participants completeduniversity/bachelor studies

- 19 -© FTW 2014

Results of Model Verification

MOS = 0.45∗log (DBW)+2.48Pattern MOS ADTP1 2.9 5.9 MbpsP2 3.4 5.1 MbpsP3 3.1 4.3 Mbps

ADT is calculated via the pattern

- 20 -© FTW 2014

Conclusions for Single Outages

From a methodological point of view, it is reasonable to have- 30 seconds web sessions- Single outage events (instead of outage patterns)- 3 Iterations per condition

- 21 -© FTW 2014

1) Single Bandwidth Outages

- 22 -© FTW 2014

2) Fluctuation Model Verification