www3.informatik.uni-wuerzburg.de

Institute of Computer ScienceChair of Communication Networks

Prof. Dr.-Ing. P. Tran-Gia

Assessing Effect Sizes of Influence Factors Towards a QoE Model for

HTTP Adaptive Streaming

Tobias Hoßfeld, Michael Seufert, Christian Sieber, Thomas Zinner

Michael Seufert2

2

Agenda

1. HTTP Adaptive Streaming

2. QoE Influence Factors of HAS

3. Goal of the Study

4. Study Description

5. Key Influence Factors of HAS QoE

6. Simple QoE Model and Applications

7. Conclusion

Michael Seufert3

3

HTTP Adaptive Streaming

HTTP Adaptive Streaming (HAS) was introduced by MOVE Networks in 2007

Many proprietary streaming technologies exist Apple HTTP Live Streaming Adobe HTTP Dynamic Streaming Microsoft Silverlight Smooth Streaming

Recently also standardized in MPEG: Dynamic Streaming over HTTP (DASH)

Video is available in several encodings and split into small segments/chunks

Description file provides information about video chunks

Michael Seufert4

4

HTTP Adaptive Streaming

Client requests small chunks via HTTP Based on how fast the current (and previous) segments are

downloaded, the bit rate of the next segment is selected. Dimensions for adaptation:

Image quality (codec, quantization) Spatial (resolution) Temporal (frame rate, initial delay, stalling)

Bit rate 1

Bit rate 2

Bit rate 3

Bits

Time

Bit rate 1

Bit rate 2

Bit rate 3

Bits

Time

Bit rate

Time

TCP throughput

Requested chunks

1

2

3

Time

Quality Level

Michael Seufert5

HTTP Adaptive Streaming

Quality of Experience model for HAS is missing

Multiple dimensions have to be taken into account Network Parameters

– Initial Delay– Stalling

Video Parameters– Content– Codec– Resolution– Frame Rate– Image Quality

Key influence parameters have to be identified

Adaptation Parameters– Available representations– Frequency (segment length, number of

switches)– Amplitude (quality representation levels)

Context Psychological Effects

– Recency– Noticeable difference

Michael Seufert6

Survey of Subjective Studies

Many aspects are already researched in different works Subjective studies with varying video quality have been conducted

in related fields More than 150 works were surveyed

Survey paper currently under submission, technical report with preliminary version is available online (University of Würzburg, Institute of Computer Science, Tech. Rep. Nr. 490, online)

HTTP Adaptive Streamin

g

Video Quality

Human Computer Interaction

Networking

etc.

Michael Seufert7

7

Goal of the Study

Subjective investigation of adaptation parameters Switch amplitude (i.e., quality level difference) Switching frequency Recency effects

FP7 Project SmartenIT 10 European partners (academia, network

and cloud operators) Goals:

– Incentive-compatible cross-layer traffic management in the Internet

– Incorporate social awareness and energy efficiency

– Considered cloud applications: Video streaming, file sharing

HAS QoE model can facilitate monitoring and traffic management decisions

http://www.smartenit.eu

Michael Seufert8

8

Study Description

Crowdsourced video quality survey based on QualityCrowd2 framework

14 sec sequence from „Tears of Steel“ in 3 quality layers (resolution reduction, 640x360, 320x180, 160x90)

710 unique participants, 7-9 ratings per user Filtering of unreliable user ratings

Test subjects have to watch whole video sequence User have to rate and answer video content question 11% of users had to be filtered out

Different quality profiles (2 layers) to identify influence of 1 parameter 40 test conditions, minimum 82 (avg. 106) ratings per condition All study details are available in a technical report (University of

Würzburg, Institute of Computer Science, Tech. Rep. Nr. 491, online)

Michael Seufert9

9

Switching Frequency vs Time on Layer

In several works, switching frequency is reported to influence QoE

Often parameters „number/frequency of switches“ and „time on layer“ are correlated and change simultaneously

Keeping „time on layer“ constant, no influence of switching frequency could be found

Michael Seufert10

10

Key Influence Factors of HAS QoE

Determining key influence factors by: Spearman rank-order correlation coefficient One-way analysis of variance (ANOVA) Effect sizes (partial Eta-squared and Cohen‘s f²)

No effect can be observed for last quality level, recency time, and number of switches

Amplitude and time on highest layer are key influence factors

Michael Seufert11

11

Simple QoE Model and Applications

Simple QoE model (2 layers) based on two key influence factors Exponential model provides a very good fit to the data points

(R²=0.98)

Applications of simple QoE model: Monitoring becomes easier New HAS algorithms can be developed QoE-based performance evaluation of HAS algorithms is possible

(e.g., comparison to theoretical QoE-optimum for given conditions)

Michael Seufert12

12

Conclusion

Adaptation strategy related parameters of HAS were investigated and evaluated using subjective tests conducted in a crowdsourcing platform

Main results: Amplitude: high impact of the switching amplitude Frequency: time on highest video quality layer has a significant

impact on the QoE, number of quality switches can be neglected Recency effects: can be neglected

Based on these results, a simplified QoE model for adaptive streaming was derived

Future work: Fortification of findings and investigation of other HAS parameters Extension of simple QoE model Creation of a generally accepted QoE model for HAS Development of new HAS algorithms

Michael Seufert13

Thank you for your

attention!

Bit rate

Time

TCP throughput

Requested chunks

http://www.smartenit.eu