analyzing the dynamic adaptive streaming over http (dash) protocol

1

Analyzing the Dynamic Adaptive Streaming over HTTP (DASH) Protocol

Dan Rice, Terry Shaw

CableLabs

Louisville, CO

d.rice, [email protected]

Jim Martin

School of Computing

Clemson University

Clemson, SC

[email protected]

August 21, 2012

2

Executive Summary

The widespread deployment and adoption of the Dynamic Adaptive Streaming over

HTTP (DASH) standard is making Internet video-on-demand an Internet application

analogous to email and web browsing. The contribution of the research presented in this

paper is twofold. First we present the results of a measurement study that we have

conducted that characterizes the bandwidth consumption and behaviors of a range of

popular Netflix client devices. Second, based on the results of our measurement study

and supplemented with results from simulation, we provide preliminary results of a

performance analysis of DASH. The methodology that was used in both the measurement

and simulation studies was similar: we established a Netflix session and observed the

subsequent impacts when the data stream was subject to periods of either controlled

packet loss or congestion caused by competing TCP streams. The highlights of our results

include the following:

A Netflix session operates either in a ‘buffering state’ or a ‘steady state’. While

buffering, we observed download rates up to 37 Mbps. During steady state, we

observed a maximum TCP throughput of 4.31 Mbps.

Netflix clients operating on Roku, Windows Desktops, and Android Smartphone

devices used a single TCP connection at a time. Only the Xbox client used two

TCP connections concurrently (but the aggregate TCP throughput was always less

than 4.3 Mbps).

The time it takes a session to converge to a new steady state throughput depends

on many factors. We observed reaction times in the range of 50 to 100 seconds.

We also observed that Netflix is more conservative as it reacts to increases in

available bandwidth.

The range of throughputs the sessions reach once the loss process is disabled

ranges from 2.73 Mbps to 3.86 Mbps. None of the devices restored the steady

state to the level observed before the emulated network congestion occurred.

The playback buffer sizes ranged from 30 seconds (on the Android device) to 4

minutes (on the Windows wired device). The average size of the client request

ranged from 1.8 Mbytes to 3.57 Mbytes.

Measurement experiments over an 802.11n with competing TCP flows were

limited, however one noteworthy result showed a Netflix flow reduced its

throughput by only 5% when a competing downstream always-on TCP flow was

activated.

Simulation results suggest that the playback buffer capacity at the client (which is

not a configurable option in any of the client devices we studied) impacts the time

the session requires to transition to new steady states. Further simulation results

confirmed that Netflix sessions and competing always-on TCP flows achieve fair

allocation of bandwidth over a 10 Mbps point-to-point bottleneck link. We saw a

similar result when the bottleneck link was a single downstream DOCSIS

channel.

All simulation results and measurement results suggest that Netflix application-

level adaptation disengages during periods of network congestion allowing

Netflix flows to compete fairly with other TCP traffic flows..

3

Table of Contents

EXECUTIVE SUMMARY ...........................................................................................................2

INTRODUCTION..........................................................................................................................4

DASH PROTOCOL.......................................................................................................................4

METHODOLOGY ........................................................................................................................6

MEASUREMENT RESULTS ....................................................................................................10

SIMULATION ANALYSIS RESULTS .................................................................................2419

RELATED WORK ..................................................................................................................2924

CONCLUSIONS ......................................................................................................................3025

REFERENCES .........................................................................................................................3126

4

Introduction Video streaming has been widely deployed and studied for decades. In the early years,

Internet streaming typically involved UDP and RTP/RTSP protocols. The application

was likely a real-time video session which, due to the sensitivity to packet latency and

jitter, never became widely popular. Internet-oriented Video-on-Demand (VoD) began as

simple file downloads. As the popularity of watching video over the Internet increased,

VoD methods evolved. Progressive downloading, which allows video playback to begin

before all the data arrives, has been replaced recently by dynamic adaptive VoD

techniques which allow the choice of video content to be changed on-the-fly. Apple,

Microsoft, Netflix, and the 3GPP wireless communities have all developed proprietary

methods for adaptive TCP-based streaming. In 2009 the ISO solicited proposals for a

standards-based framework to support Dynamic Adaptive Streaming over HTTP

(DASH). The standard was ratified in December of 2011 [ISO11, STO11]. The 3GPP

community recently finalized its DASH variant that is optimized for wireless

environments [3GPP12].

Ongoing academic research is providing foundations for understanding how DASH

applications behave and how they might be improved. As a part of our ongoing research

in resource allocation issues in broadband access networks, we seek to understand the

impact of adaptive applications on congestion and bandwidth control mechanisms within

the Internet or within a broadband access network. The work presented in this paper

provides foundations for achieving this goal. The contribution of the research presented

in this paper is twofold. First, we present the results of a measurement study that was

conducted characterizing the bandwidth consumption and behaviors of a range of popular

Netflix client devices. Second, based on the results of our measurement study and

supplemented with results from simulation, we provide preliminary results of a

performance analysis of DASH. Our measurement study identifies a range of client

implementation decisions. Our performance study of DASH provides insight towards the

impacts of client design decisions, such as client playback buffer management or

adaptation sensitivity to performance and fairness issues both at the local and at the

global perspectives.

This paper is organized as follows. The next section provides an overview of the DASH

protocol. The following section summarizes the methodology used in our study. We

then present the results of the measurement and simulation analysis. The final section

identifies conclusions and future work.

DASH Protocol Figure 1a illustrates the main components of a DASH-based video distribution system.

Multimedia content is encoded to one or more representations (different bit rates),

allowing the client (i.e., the media player) to request any part of a representation in units

of variable size blocks of data. The data content is made available to client devices by

5

standard HTTP requests. The DASH server-side can be as simple as a standard HTTP

web server. The available representations are maintained in a summary file referred to as

the Media Presentation Description (MPD). MPD describes media types, resolutions, a

manifest of the alternate stream and its URLs, minimum and maximum bandwidths, and

required digital rights management (DRM) information.

Figure 1. DASH System Components and Client Behavior

As illustrated in Figure 1a, the DASH client contains three main functional areas: the

HTTP access, the media engine that decodes and renders the video stream, and the

control engine. The latter component monitors the arriving video stream and determines

0 10 20 30 40 50 60 70 800

10

20

30

40

50

60

70

Time (seconds)

Thro

ughput

(Mbps)

TCP Cx 1

1.0 second samples

10 second samples

Buffering state

(37.02 Mbps between

10-30 seconds)

Steady state

(4.39 Mbps)

b. DASH Client Behavior

a. DASH System Components

6

when a lower or higher quality stream should be requested. The client will maintain a

playback buffer that serves to smooth variable content arrival rates to support the video

playback. The client requests a new segment (tagged with the desired bitrate) once the

playback buffer drops below a certain threshold. When the playback buffer is almost

empty, the client is likely to go into a ‘buffering’ state where it requests segments at a

high rate. The session therefore operates in one of two states: buffering or steady state.

While in a buffering state, the client requests data at a rate up to the available bandwidth

over the path between the server and the client. If conditions permit, the session will

attempt to find a ‘steady state’ where it requests segments at a rate necessary to playback

the content at a given encoded bitrate. In the analysis we present in this paper, we define

steady state as a time interval of at least 50 seconds where the session exhibits a stable

mean (i.e., observed mean over at least two 50 second intervals is within a tolerance of

10% variation). This assessment of ‘steady state’ is a network-oriented measure and is

potentially different from an assessment of ‘steady state’ at the application level. In future

work we will explore the interaction between network conditions, playback buffer

management, video encoder/decoder behaviors and the subsequent effects on perceived

quality.

Figure 1b illustrates the observed behavior of a Netflix client operating on a Windows

Desktop PC located at Clemson University. The results illustrate the first 80 seconds of a

Netflix streaming session over a high speed network (i.e., the available bandwidth

significantly exceeds the requirements of a Netflix session). The two curves plot the

evolution of the observed throughput of a single TCP connection that transfers video

content from the Netflix system to the client. The solid curve formed by the square data

points is based on 10 second throughput sampling intervals and the lighter dashed curve

is based on 1 second samples. The figure shows the initial buffering state and, once the

playback buffer is full, that the session moves into a steady state within about 35 seconds.

While this basic ‘waveform’ associated with DASH is similar across different client

devices that we have studied, we will show in our results that the detailed behavior of

each client device varies.

Methodology The research reported in this paper involves a measurement component and a simulation

component. We describe the methodology of each activity separately.

Measurement methodology

Figure 2 illustrates the measurement testbed that was used to monitor live Netflix

sessions. We created a private network that connected four different client devices: an

Xbox, a Roku device, a Windows desktop, and an Android Smartphone. The devices

were connected to the campus network either through a private gigabit Ethernet or

through an 802.11n network that was in turn directly connected to the campus network

through our private gigabit Ethernet. A high-definition monitor (capable of 1080p format)

was connected to the Xbox and Roku’s HDMI port. We monitored Netflix traffic by

obtaining network traces using tcpdump at the Linux Router. The router connects the

measurement testbed with Clemson University’s Network which provides a 10 Gbps

7

connection to the commodity (public) Internet. We installed several additional Linux

boxes in the testbed to serve as background traffic generators or to provide additional

performance metric data. Linux Traffic Generators 1 and 2 were desktop PCs running

Linux. Traffic Generator 3 was a laptop running Linux.

We collected traces for a period of six months from January 2012 through June 2012. Our

client devices interacted with different Netflix servers with an IP address in the range of

108.175.34.66 through 108.175.34.71. We would occasionally see servers from other

network prefixes, however, the 108.175.32.0/20 network was favored by all devices. The

client devices exhibited different server selection policies. The Xbox tended to stick with

the same server even if it decides to reestablish a connection. The Android device would

switch to a server located on a different network when it tried to establish a new

connection. To ensure that the external network (Clemson’s network, the Internet) and

the Netflix content distribution system were not contributing unwanted network

congestion we performed several 24 hour ping assessments with the server

108.175.34.71. One result over a 24 hour period (consistent with other tests run

throughout our data collection period) indicated a loss rate of 0% and an RTT of 14.55

milliseconds (ms) with a standard deviation of 0.64 ms.

Figure 3 identifies five experimental scenarios that were considered. For the results

reported in this paper, each scenario consists of a set of experiments. For a given scenario

the experiments varied the choice of client device or access network. In each experiment,

we collected a trace of all packets associated with the Netflix session that was under

observation. We refer to specific experiment results through the naming convention

tracex-y where the x indicates the scenario and the y indicates the choice of device or

access network. The value of y follows the table labeled Client Devices in Figure 2. As

an example, the results associated with trace2-2 reflect an experiment based on scenario 2

using the Netflix client operating on a Windows device connected to the testbed by a

gigabit Ethernet connection. A Netflix session typically involves many short TCP

connections in addition to one or more long-lived connection. The short TCP connections

consume an insignificant amount of bandwidth, therefore we focus our analysis on the

larger connections which transfer the majority of video and audio content. A Netflix

session typically involves multiple (sometimes many) TCP connections. Some might be

short-lived and others might be long-lived. Our analysis examines the behavior of

specific TCP flows used by Netflix Sessions. In particular we monitor the throughput

achieved by the TCP connection under observation. The assessment is based on the rate

at which application data arrives at the client device from a Netflix server. We also

consider the aggregate bandwidth consumed by all TCP flows associated with Netflix

sessions. In this case, the assessment monitors the rate at which all TCP data sent by any

Netflix server arrives at the Netflix client. The aggregate bandwidth assessment includes

TCP and IP header overhead.

8

Figure 2. Measurement Testbed

Scenario 1 depicts a network that experiences a temporary increase in packet loss.

Scenario 2 depicts a temporary network outage event. Scenarios 3 and 4 provide insight

in how the algorithm adapts to more complex periods of network congestion episodes

that involve stochastic processes that replicate non-stationary network behavior. Scenario

5 addresses the interaction between a Netflix flow and a competing greedy TCP flow

(i.e., perhaps due to a large file transfer).

Figure 3. Measurement Scenarios

Scenario ID Description

1 Ideal with 3% artificial loss (5 minutes no impairment, 5 minutes 3% loss, 5 minutes no impairment)


3 Stepped loss (200 seconds at 0% loss, 100 seconds at 5% loss, 100 seconds at 10% loss, 100 seconds at 0% loss)

4 Chaotic loss (300 seconds at 0% loss, 300 seconds at variable loss, 300 seconds at 0% loss)

5 Competing flows (5 minutes no competing TCP, 5 minutes with competing flow, last 5 minutes with no competing TCP)5-1 : Xbox Wireless device, Downstream competing TCP5-2 : Roku Wireless device, Downstream competing TCP5-3 : Xbox Wireless device, Upstream competing TCP5-4 : Roku Wireless device, Upstream competing TCP

9

Simulation methodology

Figures 4 and 5 illustrate the simulation networks that were studied. Figure 4 models the

wired portion of the measurement testbed. The node labeled GW serves as the access

router for all client devices. The link between the middle router and the GW node was set

to either 1 Gbps or 10 Mbps depending on the simulation. The model is tuned such that

the uncongested end-to-end round trip time is 14 ms, which is what we observed in the

testbed. The access link rate is 1 Gbps. As in the measurement study, we applied an

artificial packet loss process to packets that are forwarded by the GW (access router) to

the Netflix client devices. Figure 5 models a DOCSIS-based cable network involving N

cable modems (CMs) that share one 256 QAM downstream channel and one 64 QAM

upstream channel. The model assumes lossless channels implying that frames are never

dropped as a result of RF impairment.

Figure 4. Simulation Model – Wired

Figure 5. Simulation Model - Cable

Figure 6 identifies the six simulation scenarios that were studied. Simulation Scenarios 1

through 4 directly match the Measurement Scenarios 1 through 4. Scenarios 5 and 6

allow us to explore how DASH behaves when subject to competing DASH session traffic

or to competing always-on TCP flows.

1000Mbps,

.5ms prop delay

Data rates:

1 Gbps (US and DS)

Node 1

Node n

GWrouter router

1 Gbps or 10Mbps,

6.5ms prop delayNetflix Servers

Competing Traffic

Generators/Sinks

1000Mbps,

.5ms prop delay

Data rates:

30.72Mbps upstream,

42.88Mbps downstream

Cable Modem 1

(CM-1)

CM-n

CMTSrouter router

1000Mbps,

Variable prop

delay

Netflix Servers

Competing Traffic

Generators/Sinks

10

Figure 6. Simulation Scenarios

Measurement Results

Scenario 1 results

Figures 7 and 8 illustrate the results from Measurement Scenario 1. Figure 7 illustrates

the Windows Wired client device behavior in more detail, focusing on the evolution of

the single TCP connection that is used by the Netflix session under observation. Two

curves are plotted in Figure 7. The curve formed by the light dashed line is based on

TCP throughput samples obtained using 2 second intervals. The curve formed by the

solid line (with the square points) is based on samples obtained using 50 second

intervals. The initial buffer fill rate is roughly 40 Mbps (this data point cannot be seen in

Figure 7 because we set the y-axis limit to 20 Mbps). After 50 seconds, the session

moves to a steady state throughput of 4.12 Mbps. Once the loss process begins at time

300 seconds, the session reaches converges to its next steady state value of 2.01 Mbps in

roughly approximately 100 seconds. Finally, once the loss process is disabled, the session

converges to a steady state throughput of 3.18 Mbps. It takes roughly approximately 150

seconds for the session to move to the third steady state.

Figure 8 illustrates results from five experiments, each involving a different client device

(or access link).). While Figure 7 visualized the throughput achieved by the single TCP

connection used by the Netflix session, Figure 8 plots the aggregate bandwidth consumed

by the Netflix session which might involve multiple TCP connections. As in Figure 7,

two curves are plotted in each sub-figure. The curve formed by the light dashed line is

based on bandwidth samples obtained using 2 second intervals. The curve formed by the

solid line (with the square points) is based on samples obtained using 50 second sample

intervals. Figures 8a and 8c plot the bandwidth consumed by the Xbox device when

operated over a wired and wireless network respectively. The Xbox client behaves almost

identically in both scenarios. In both cases, the client uses two TCP connections in

parallel. The aggregate bandwidth consumed is evenly split between the two connections.

Scenario ID Description



3 Stepped loss (200 seconds at 0% loss, 100 seconds at 5% loss, 100 seconds at 10% loss, 100 seconds at 0% loss

4 Chaotic loss (300 seconds at 0% loss, 300 seconds at variable loss, 300 seconds at 0% loss)

5 A mix of netflix sessions and competing TCP flows over a wired network (no cable)

6 A mix of netflix sessions and competing TCP flows over a cable network

11

Figure 8 identifies the mean bandwidth consumption during three time periods that

represent the steady state periods that are observed. We refer to these intervals as

follows: (steady state) SS1: the intervals from 50-300 seconds; SS2: the interval from

400 to 550 seconds; SS3: the interval from 750 to 900 seconds. The wired Xbox Netflix

client consumes approximately 4.36 Mbps, 2.97 Mbps, and 2.93 Mbps in each of the

three intervals. The wireless device consumes roughly the same bandwidth.

The Windows wired results, illustrated in Figure 8b, is essentially the same as Figure 7

(i.e., the vast majority of audio/video data is transferred over the single long-lived TCP

connection). The bandwidth consumption observed in each steady state interval is

approximately 5% greater than the throughput observed in the intervals illustrated in

Figure 7. This is because the bandwidth consumption assessment includes TCP and IP

header overhead as well as additional TCP packets from a small number of ‘tiny’ (i.e.,

sent or received a very small amount of data) TCP connections that were active during

the experiment. Figure 8d illustrates the Roku device converges to steady state

throughputs of 4.37 Mbps, 3.31 Mbps and 3.55 Mbps. Further analysis (not presented in

this paper) indicates that in the time period 550 to 600 seconds, there were actually a

sequence of 5 short-lived TCP connections that were started. Each persisted for roughly

20 seconds. Once the loss process was stopped, a single connection was established that

transferred all subsequent content. Finally, Figure 8e visualizes the behavior of the

Netflix client on the Android device. Once loss started at time 300 seconds, multiple TCP

connections were observed. As with the Roku, the connections were started sequentially

and lasted about 20 seconds. The aggregate bandwidth consumed during this time period

was 2.61 Mbps. There are one, two or three figures that visualize the results from each

experiment. For example, Figures 8a and 8c each plot two graphs which together

visualize the throughput achieved by two parallel TCP connections that are used by the

Netflix client session running on an Xbox device connected by a wired or wireless

network respectively. Figure 8b plots the throughput achieved by the single TCP

connection used by a Windows Netflix client session. Two curves are plotted in each

graph. The light dashed line is based on TCP throughput samples obtained using 2

second intervals and the solid line is based on samples obtained using 30 second sample

intervals. Figures 8d and 8e each plots sequential TCP flows used by a Netflix client

running on a Roku and an Android device respectively. In all of our traces, the maximum

steady state throughput consumed by any session was 4.31 Mbps (observed with an Xbox

client session in Measurement Scenario 3-2). Table 1 provides the mean and standard

deviation of the observed throughput in each steady state period. For example, the first

row of Table 1 summarizes the results associated with Trace 1.1 which is the scenario1

experiment using the Xbox Wired device. The second column identifies the time period

during the trace that is being summarized. The third time identifies the setting of the

artificial loss process (i.e., it was either disabled or set to provide a random loss with an

packet loss rate of 3.0% during the time interval 300 to 600 seconds. The last three

columns provides information about the steady state throughput statistic (which is

considered during the time period given in column 2). The sample is based on 50 second

throughput samples. The fourth column identifies how many samples are considered. The

fifth and sixth columns provide the mean and standard variation of the steady state

12

throughput. For the Xbox experiments, the mean and standard deviation values in the

table are based on the samples from the two concurrent connections.

There clearly are differences across client devices were significant. It is difficult to

determine if observed differences in client behaviors is due to Netflix implementation

details or to the device TCP/IP stack. Based on the results from Measurement Scenario

1, we identify the following differences across client devices:

Netflix clients operating on Roku, Windows Desktops, and Android Smartphone

devices used a single TCP connection at a time. Only the Xbox client used two

TCP connections concurrently. All devices consumed steady state bandwidth in

the range of 4.35 to 4.40 Mbps during the first steady state interval. Based on

observation of the quality of the rendered video, we believe this reflects the

highest video quality level provided by Netflix.

(but the aggregate TCP throughput was always less than 4.3 Mbps).

The range of steady state throughputs bandwidth consumption achieved under

various controlled network scenarios varied significantly. Our results suggest that

Netflix application-level adaptation is conservative and that during periods of

network congestion appears to disengage.

The range of steady state throughputs bandwidth consumption achieved when loss

was active varied from 2.01 to 3.3120 Mbps. It is difficult to determine if

observed differences were due to Netflix implementation details or to the device

TCP/IP stack

Interestingly the Xbox converged to a throughput of 2.51 Mbps when connected

via gigabit Ethernet but only to 2.15 Mbps over WiFi. The Android device was

not able to maintain a TCP connection during this time period.

The range of times it takes the session to converge to new steady state

throughputs depends on the timescale that is used to define steady state. We

observed that Netflix reacts to congestion quickly (the example in Figure 7 shows

100 ms) but reacts more slowly to sudden increase in available bandwidth. Again

looking at Figure 7, it takes over 150 seconds for Netflix to increase the

application throughput change in throughput.

The range of steady state bandwidth consumption once the loss process is

disabled throughputs the sessions reach once the loss process is disabled ranges

from 2.973 Mbps to 3.7986 Mbps. This is less than the initial steady state

suggesting that Netflix reacts slowly to sudden increases in available bandwidth

after a period of sustained congestion. None of the devices restored the steady

state to the level observed before the emulated network congestion occurred.

14

Figure 7. Trace1-2 (Wired Windows) Detailed ResultsTCP Throughput

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

TCP Cx 1

2.0 second samples

50 second samples

Steady State 1

(Mean: 4.12 Mbps

std: 0.13 Mbps)

Steady State 2

(Mean: 2.01 Mbps

Std: 0.32 Mbps)

Steady State 3

(Mean: 3.18 Mbps

Std: 0.50 Mbps)

0 % Artificial Loss 0 % Artificial Loss

3% Artificial Loss

15

Figure 8. Measurement Scenario 1 Results (Consumed Bandwidth)

Scenario 2 results

Figure 9 illustrates the behavior of the devices when subject to a temporary network

outage. All devices are unable to receive content once the loss process is enabled. We

measured the time from when the network outage began until when we observed that the

video rendering had stopped. The time, which represents the size of the client device’s

playback buffer, ranged from 30 seconds (on the Android device) to 4 minutes (on the

b. Trace 1-2 (Windows wired) d. Trace 1-4 (Roku Wireless)

e. Trace 1-5 (Android Wireless)

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

a. Trace 1-1 (Xbox wired)c. Trace 1-3 (Xbox wifi)

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

16

Windows wired device). Once the outage ends (at time 600), the devices move into

buffering state to refill the playback buffer. As examples, we observed fill rates with

peaks of up 40 Mbps (the Windows wired device) to 15 Mbps (the Xbox device). Each

device used a new TCP connection once the outage was complete. Table 2 identifies the

throughput for each device at time 700 through 900 seconds. The results indicate that the

devices converge to different steady state throughputs. The Xbox quickly moves back to

its highest quality level, although using a single TCP connection (rather than two

concurrent connections). The Windows device appears to converge to a lower quality

level by 700 seconds (the mean throughput between 700 and 800 seconds is 2.33 Mbps)

but then moves to a higher bit rate setting after time 800 seconds (the mean throughput

between 800 and 900 seconds is 3.44 Mbps). The Roku client initiated connection 2 at

time 600 seconds. It stalled and was replaced by a new connection at time 680 seconds.

The wired Xbox client converges quickly to a steady state of 4.25 Mbps (the wireless

Xbox client converged to 4.34 Mbps). This recovery is quite different from that observed

in the previous scenario where the client converged to a state of 3.95 Mbps. We

conjecture that starting a new TCP connection. The Windows wired device appears to

enter a steady state of 2.80 Mbps from time 650 to 800 seconds and then switch to a

steady state of 3.8 Mbps after time 800 seconds. The Roku device (Figure 9d) converges

to a steady state of 4.03 Mbps, however it suffered an impairment at time 675 seconds

when the consumed bandwidth dropped to less than 1 Mbps. The Android device

converges to the highest observed steady state bandwidth consumption level of 4.63

Mbps observed in all of the experiments. We conjecture that the device refills its buffer

during this time period.

17

Figure 9. Measurement Scenario 2 Results

a. Trace 2-1 (Xbox wired) c. Trace 2-3 (Xbox wifi)



0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

18

Figure 9. Measurement Scenario 2 Results (Consumed Bandwidth)




0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

19

Scenario 3 results

Figure 10 illustrates the throughput achieved by the single TCP connection uses by the

Windows wired device (Trace 3-2) subject to two different settings of artificial packet

loss. The loss process moves from settings of 0% loss to 5% loss at time 200 seconds,

from 5% loss to 10% loss at 300 seconds, and finally from 10% loss back to 0% loss at

400 seconds. We observe the Netflix session evolve over four steady state periods

involving throughputs of 4.31 Mbps, 1.16 Mbps, 0.53 Mbps, and 4.04 Mbps respectively.

Appendix Figure A1 illustrates the bandwidth consumed by all behavior of all the client

devices in this scenario. The Windows device is able to continue to transfer data even

when the loss rate is 10%. As illustrated in Figure A1, the other devices were unable to

transfer a significant amount of data while packet loss was active. Further analysis is

required to determine if this result is an artifact of the Netflix implementation or of the

TCP/IP stack on the device.

Figure 10. Measurement Scenario 3 Trace 3-2 (Windows Wired) TCP Throughput

50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

TCP Cx 1

2.0 second samples

50 second samples

Steady State 1

(Mean: 4.31)

Steady State 2

(Mean: 1.16 Mbps)Steady State 3

(Mean: 0.53 Mbps)

Steady State 4

(Mean: 4.04 Mbps)

0% Loss

5% Loss

10% Loss 0% Loss

20

Figure 10. Measurement Scenario 3 Trace 3-2 (Windows Wired) Detailed Results

Scenario 4 results

Figure 11 illustrates the throughput achieved by the single TCP connection used by the

Windows device subject to the variable (random) loss process. We varied the controlled

packet loss rate in the time period between 300 through 600 seconds. Every 20 seconds

the loss process moved to a new state (with a new average packet loss rate). The specific

transition times and loss rate settings are shown in Figure 11. The average loss rate of the

17 settings was 4.4%. This scenario was used to explore the stability of the Netflix

adaptation algorithm when subject to ‘chaotic’ (i.e., non-stationary) packet loss behavior.

Figure 11 illustrates that the Windows device does adapt to changing conditions. Based

on this data it is not possible to know if the adaption is driven by TCP or by the

application. Appendix Figure A2 illustrates the results for the other devices. As observed

in Scenario 3, the other devices appear not to deal with the varying packet loss as

effectively as the Windows device.



0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

21

Figure 11. Measurement Scenario 4 Trace 4-2 Results (TCP Throughput)

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

TCP Cx 1

2.0 second samples

50 second samples

loss rate transitions

0-300 : 0%

300-320 : 4%

320- 340 3%

340-360 0%

360-380 0%

380-400 8%

400-420 10%

420-440 9%

440-460 3%

460-480 1%

480-500 5%

500-520 10%

520-540 1%

540-560 4%

560-580 8%

580-600 9%

600-900 0%

Steady State 1

Steady State 2

Steady State 3

Steady State 4



0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

22

Figure 11. Measurement Scenario 4 Trace 4-2 Results

Scenario 5 results

Figure 12 illustrates the results from Scenario 5. We defined four experiments, identified

as Trace 5-1 through Trace 5-4. Trace 5-1 involved the Xbox wireless device and Trace

5-2 involved the Roku wireless device. These traces demonstrate the effect of the

competing TCP flow downstream while Trace 5-3 and 5-4 involved an upstream TCP

flow. In all four experiments, a Netflix session is established at time 0 seconds. At time

300 seconds, the competing TCP flow is initiated between Traffic Generators 1 and 3.

We used iperf to generate competing TCP traffic. To generate competing downstream

traffic, iperf generates application traffic at the Traffic Generator located on Clemson’s

network destined for iperf set to sink arriving data at the Traffic Generator located on the

wireless network. Upstream traffic is generated simply by reversing the

sending/receiving roles of the Traffic Generators. The wireless AP was a Linksys

EA3500 configured to support 802.11n using 20 or 40 MHz channels in the 2.4 GHz ISM

band and in the 5.0 GHz band. Due to the campus WiFi network, our ‘rogue’ AP was

subject to significant interference. We observed significant variation in the competing

TCP flow’s throughput. We conjecture that interference was causing the AP to switch

back and forth between 20 MHz and 40 MHz channels frequently. In the best conditions

we observed a maximum sustained upstream TCP throughput of 63 Mbps. In the

downstream case, the results were more unpredictable. In some cases, we observed

throughput up to 63 Mbps. Most of the time however, we observe throughput in the 15-

25 Mbps range.

Figure 12a illustrates the results from Experiment 5-1. The graphs labeled TCP Cx1 and

TCP Cx2 visualize the two concurrent TCP connections used by the Xbox Netflix client.

The graph labeled TCP Cx3 visualizes the throughput achieved by the competing TCP

flow. Figure 12a identifies the three steady state periods the Xbox client experiences.

Table’s 3 and 4 shows the average throughput (and standard deviation) of the steady state

1 and steady state 2 periods of each trace for the scenarios that involve a downstream and

then an upstream competing TCP. The results suggest that the Netflix session is impacted

more by a competing flow that is oriented downstream rather than upstream. For the

Xbox, prior to when the competing flow begins (at time 300 seconds), the aggregate TCP

throughput achieved by the Netflix session is 4.19 Mbps. Once the competing flow

begins, the aggregate Xbox TCP throughput drops by 11.9% to 3.70 Mbps. Once the loss

process terminates however, the Xbox TCP throughput converges to an even lower rate

of 3.06 Mbps. This is consistent with the Roku device which evolves from 3.93Mbps to

3.72 Mbps and finally to 3.21 Mbps. We conclude that Netflix adapts quickly to drops in

available bandwidth but reacts very cautiously as available bandwidth is restored.

23

Figure 12 Measurement Scenario 5 Results

0 200 400 600 800 10000

10

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

0 200 400 600 800 10000

10

20 TCP Cx 2

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

0 200 400 600 800 10000

50

TCP Cx 3

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

a. Trace 5-1 (Xbox Wireless, Downstream TCP) c. Trace 5-2 (Roku Wireless, Downstream TCP)

b. Trace 5-3 (Xbox Wireless, Upstream TCP) d. Trace 5-4 (Roku Wireless, Upstream TCP)

0 100 200 300 400 500 600 700 8000

5

10

15

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

0 100 200 300 400 500 600 700 8000

20

40

60

TCP Cx 2

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

0 100 200 300 400 500 600 700 8000

10

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

0 100 200 300 400 500 600 700 8000

10

20 TCP Cx 2

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

0 100 200 300 400 500 600 700 8000

50

TCP Cx 3

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

0 100 200 300 400 500 600 700 8000

5

10

15

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

0 100 200 300 400 500 600 700 8000

20

40

60

TCP Cx 2

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

24

Table 3 Measurement Scenario 5 Steady State Throughput Statistics (downstream TCP)

Table 4 Measurement Scenario 5 Steady State Throughput Statistics (upstream TCP)

Simulation Analysis Results Figure 13 illustrates the design of the simulation model. The client periodically issues

requests to the server for a data segment. In an actual implementation, the request

specifies a ‘slice’ of video, including the desired bitrate encoding. In our model, the client

requests a specific amount of data to be sent. While the request message conforms to the

DASH specification, the contents have no meaning (aside from the byte request) to the

server. Similarly, the data sent by the server has no meaning to the client. When the

server receives a client request, it transmits the request number of bytes at the rate

allowed by TCP. The client ensures the proper session sending rate by controlling the

rate at which client is requested from the server. The simulation model uses a single TCP

connection. The main simulation model parameters that affect the behavior are

summarized as follows:

Trace ID and Cx #

Time Period (seconds)

Number of Samples

Mean (Mbps)

Standard Deviation (Mbps)

Trace 5-1 Cx1 400-600 4 1.89 0.28

Trace 5-1 Cx2 400-600 4 1.81 0.21

Trace 5-1 Cx3 400-600 4 26.99 0.52

Trace 5-2 Cx1 400-600 4 3.72 0.34

Trace 5-2 Cx2 400-600 4 15.93 0.23

Trace ID and Cx #

Time Period (seconds)

Number of Samples

Mean (Mbps)

Standard Deviation (Mbps)

Trace 5-3 Cx1 300-450 4 2.13 0.34

Trace 5-3 Cx2 300-450 4 2.12 0.15

Trace 5-3 Cx3 300-450 4 56.10 3.38

Trace 5-4 Cx1 300-450 4 3.93 0.42

Trace 5-4 Cx2 300-450 4 55.25 2.92

25

Playback buffer capacity: This is configured in units of seconds. The client

issues requests to maintain the playback buffer within an operating range defined

by two internal parameters, Highwatermark and Lowwatermark. The default

setting of the playback buffer capacity is 90 seconds.

Number of outstanding client requests: This determines the maximum number

of requests that can be outstanding at any given time. The default setting is 2

segments.

Segment size: This determines the granularity of the data exchanges between the

DASH server and the client. The default is 2 seconds.

Adaptation Threshold: This tunes the client’s sensitivity to changes in observed

throughput.

Discrete bitrate encoder options: The range of possible bitrate encoder values is

set as follows (in units of bps)

o Bitrate Encoder Value Options: 768000, 1500000, 2200000, 2600000,

3200000, 3800000, 4200000, 4800000

Figure 13. Simulation Logical Model

Figure 14 illustrates Simulation Scenarios 1 through 4. The results serve to demonstrate

and validate basic functionality of the simulation model. Figures 14a through 14d can be

compared with the results from Measurement Scenarios 1 through 4 (e.g., Figures 8, 9,

Appendix A1 and A2). In each of the four figures in Figure 14, we observe an initial

playback buffer filling operation and then the sessions evolve through at least three

steady states. Figure 15 as well as all figures in Figure 14 plot two curves: the lighter

25

HTTP Server

DASH Client

Playback buffer queues segments

• clientbuffersize: Capacity in time

• maxSegments: max queue size (segs)

• maxSize: max buffer size (bytes)

• Highwatermark, Lowwatermark: used by the

client to determine when to request new

segments from the server

Representations

Player

Controller

HTTP Get Requests

{SegmentNumber,BitRate}

Segment data

Controller parameters

• Throughput Monitor Timescale

• Segment size in seconds

• Number of outstanding requests

Monitor Arrival Process

Rendering

Performance

updates

… …

26

curve is based on 2.0 second throughput samples and the darker curve (formed with

square data points) identifies the steady state throughputs based on 50 second samples.

Based on further examination, the TCP connection moves to steady state 2 within about

10 seconds which is faster than what we observe in Measurement Scenario 1.2 (Figure 7).

We have confirmed that we can increase the convergence time by modifying the DASH

model parameters such as the playback buffer capacity and adaptation threshold. Figure

15 also identifies the throughput of steady state 1 is 4.52 seconds. This does reflect the

bitrate encoding of the highest quality level (4.8 Mbps). We observe a throughput of less

than 4.8 Mbps due to overhead.

Figure 14. Simulation Scenarios 1 through 4 Results

b. Simulation Scenario 2 d. Simulation Scenario 4

a. Simulation Scenario 1 c. Simulation Scenario 3

0 100 200 300 400 500 600 700 8000

2

4

6

8

10

12

14

16

18

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

0 100 200 300 400 500 600 700 8000

2

4

6

8

10

12

14

16

18

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

0 100 200 300 400 500 600 700 8000

2

4

6

8

10

12

14

16

18

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

0 100 200 300 400 500 600 700 8000

2

4

6

8

10

12

14

16

18

20 TCP Cx 1

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

2.0 second samples

30 second samples

27

Figure 15. Simulation Scenario 1 Detailed Results

Figure 16. Simulation Scenarios 5

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Thro

ug

hp

ut

(Mb

ps)

TCP Cx 1

2.0 second samples

50 second samples

Steady State 1

(Mean: 4.52 Mbps

std: 0.13 Mbps)

Steady State 2

(Mean: 1.43 Mbps

Std: 0.15Mbps)

Steady State 3

(Mean: 4.44 Mbps

Std: 0.12 Mbps)

0 % Artificial Loss0 % Artificial Loss

3% Artificial Loss

a. Simulation Scenario 5-1 b. Simulation Scenario 5-2

1 2 3 4 5 6 7 8 9 10 110

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Number of Competing Flows

Thro

ug

hp

ut

(Mb

ps)

TCP stats

Average Throughput of Single Flow

Average Throughput of Competing Flows

1 2 3 4 5 6 7 8 9 10 110

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5


Thro

ug

hp

ut

(Mb

ps)

TCP stats



28

Figure 17. Simulation Scenarios 6

Figures 16 and 17 illustrate the results of Simulation Scenarios 5 and 6 respectively.

Scenario 5 involves two experiments that use the simulation model illustrated in Figure 4.

Each experiment consists of six simulations. At the end of each 1000 second simulation,

the average throughput of the Netflix session as well as the average of all the competing

TCP flows is obtained. The first experiment, Experiment 5-1, establishes a Netflix

session at the beginning of the simulation. Between 1 and 11 always-on TCP flows are

added between the nodes labeled ‘Competing Traffic Generators/Sinks’ in Figure 4) and

the nodes that represent end users. In Figure 4, Node #1 is the Netflix session and each

additional competing TCP traffic flow has the receiving side attached to a new Node that

connects with the GW node through a 1 Gbps point-to-point access link.

Figure 16a plots two curves that visualize the results of Experiment 5-1. The curve

formed by the left-facing triangle points plots the Netflix session throughput and the

curve formed by the diamonds plots the average of all the competing TCP flows. As

illustrated in Figure 15, a single Netflix session that is not subject to congestion or

channel impairment consumes roughly 4.5 Mbps. Figure 16a shows this is reduced to

about 4 Mbps when it competes with one TCP flow (which achieves 5 Mbps of

throughput) for resources over a 10 Mbps bottleneck link. The difference in bandwidth

allocation between always-on TCP flows and the Netflix session dissipates as the level of

network congestion increases. Therefore, in this experiment, we see that Netflix flows

and always-on TCP flows compete fairly when the network is congested.

The second experiment in scenario 5, Experiment 5-2, establishes a single always-on

downstream TCP flow at the beginning of the simulation and adds a varying number of

additional Netflix sessions (one per Node that attaches to the GW node). The results

illustrated in Figure 16b are similar to those observed in Experiment 5-1. We conclude

that the significance of Netflix adaptation diminishes as the intensity of network

congestion increases.

The experiment in scenario 6, Experiment 6, repeats Experiment 5 using the cable

network model illustrated in Figure 5. The results are visualized in Figure 17. The cable

a. Simulation Scenario 6-1 b. Simulation Scenario 6-2

1 2 3 4 5 6 7 8 9 10 110

5

10

15

20

25

30

35

40

45

50


Thro

ug

hp

ut

(Mb

ps)

TCP stats



1 2 3 4 5 6 7 8 9 10 110

5

10

15

20

25

30

35

40

45

50


Thro

ug

hp

ut

(Mb

ps)

TCP stats



29

model, which is based on DOCSIS® 3.0, has been studied and validated in prior work

[MMW10, MMF10, MW07]. For the simulations, we did not engage service rate control.

In both Trace 6-1 and 6-2, the first data point reflects the single competing TCP flow’s

achieved throughput of 33 Mbps (while the Netflix Session obtained 4.9 Mbps). Figure

17 suggests that Netflix flows and the competing TCP flows receive similar allocation as

the network becomes congested.

Related Work In 2009 the ISO solicited proposals for a standards-based framework to support Dynamic

Adaptive Streaming over HTTP (DASH). The standard was ratified in December of 2011

[ISO11, STO11]. The 3GPP community recently finalized its DASH variant that is

optimized for wireless environments [3GPP12]. As summarized in [ST011, SOD11], the

standards focus primarily on the format of data by defining the syntax required to

describe content manifests and segments containing actual chunks of encoded video

streams. The standards do not identify specific encoding methods, specific protocol

behaviors such as how frequently the client should request data, or system guidelines

such as optimal choices of segment size.

Academic research on DASH is just emerging. We summarize the related work by

identifying three distinct categories of published research: measurement-based

performance studies, algorithm enhancements, and performance or quality metric

development.

The work in [ABD11,AN11, LMT12, RLB11, KKH11] have established the basic

behaviors of DASH applications. The work in [ABD11] is the most similar to our

measurement study. They used ‘dummynet’ to provide a controlled network testbed.

While we primarily controlled the loss rate observed by the Netflix sessions, the authors

in [ABD11] controlled the emulated access link bandwidth (referred to as ‘avail-bw’) to

the clients under observation. They evaluated the three players when the avail-bw was

unconstrained, constrained to 5 Mbps, or subject to periodic positive or negative spikes in

avail-bw. They results show differences in the details of each client’s response to change

(positive or negative) in available bandwidth. The work in [KKH10,KKH11] discusses

the benefits of parallel TCP connections although they authors point out the potential

unfairness. The work in [LMT12] shows performance drops (in terms of resulting PSNR

of the rendered video) if HTTP persistent connections are not used. The work in

[RLB11] studied Netflix and YouTube streaming using a variety of client devices. They

show that YouTube and Netflix each can exhibit widely differing ON/OFF cycle

behaviors ranging from always sending (no ON/OFF cycles), to frequent bursts of small

amounts of data (e.g., 64 Kbytes), to infrequent bursts of large amounts of data (>2.5

Mbytes). The exact behavior depends on the network properties (bottleneck link rates)

and the capabilities of the client device. The work in [AN11] provides more insight in

the ON/OFF behavior of YouTube and suggests the relatively bursty nature of the

YouTube server might contribute to higher than necessary packet loss rates.

30

The work in [ACH12, CMP11, EKG11, LBG11, SSH11] consider possible

improvements to DASH-based streaming. [ACH12] considers the benefits of using

Scalable Video Coding (SVC) outweigh the additional overhead. The authors identify

optimal approaches for determining which block should be selected next. The block

might be data that will improve the video quality of a frame whose base layer has already

arrived or it might be a block corresponding to the base layer of a frame further out in

time). The work in [SSH11] shows that DASH applications extended to support SVC can

improve cache hit rates in typical CDN distribution networks. The work in [CMP11,

LBG11, EKG11] focus on the adaptation algorithm. All approaches seek to react to

persistent impairment by dropping to lower bitrate content and then restore to higher

quality content as conditions improve. The challenge is to do this so as perceived quality

is maximized.

All of the related work involves assessing performance in some manner. As summarized

in [OS12], the standards have defined various metrics including throughput,

request/response time, buffer levels, adaptation event counts. The work in [DAJ11]

provides a more comprehensive framework by identifying metrics such as join time (the

time from when the session is initiated by the user until when video rendering begins),

percentage of time the viewing is halted due to buffering (i.e., when the video rendering

stops and the user is shown a message indicating “Buffering in Progress”), and the rate

of buffering events. Finally, the work in [NEE11A, NEE11B, XST09] describes the

subtle flicker effects that are especially evident in low compute power devices such as

smartphones.

Conclusions Video streaming has been widely deployed and studied for decades. However DASH-

based streaming has not been widely studied simply because it is a new protocol. Our

main research objective is to understand the impact of adaptive applications on

bandwidth control mechanisms within a broadband access network.

In our measurement results, we have observed a wide range of behaviors across different

client devices and network behaviors. The highlights of our results include the following:

The range of steady state throughputs achieved under controlled packet loss

settings varied significantly.

The range of times it takes the session to converge to new steady state

throughputs also varied. One challenge that we experienced was determining

when the session achieved steady state. Based on our results, we find that the

Netflix implementations that we studied appeared to behave conservatively and

tended to react over timescales in the 50 to 100 second range.

The range of throughputs the sessions reach once the loss process is disabled

varied by device. This aspect of the protocol represents a challenging design

issue: if the client attempts to use increased levels of available bandwidth quickly

it runs the risk of having to reduce bandwidth in the near future if congestion is

intermittent. On the other hand, the client should not give up available bandwidth

unnecessarily.

31

The playback buffer sizes ranged from 30 seconds (on the Android device) to 4

minutes (on the Windows wired device). The average size of the client request

ranged from 1.8 Mbytes to 3.57 Mbytes (we did not show these results in the

paper).

The measurement experiments that involved wireless provides further insight in

the behavior of Netflix. However additional analysis is required to better

characterize and assess Netflix in realistic wireless networks.

Our simulation results help us better understand the measurement results. For example,

by changing the simulation model’s playback buffer capacity and adaption threshold

values we can observe behaviors that move the model closer or further from the

behaviors we observed in the measurement analysis. The results from simulation

scenarios 5 and 6 suggest that the significance of application adaptation diminishes as the

intensity of network congestion increases. Based on this insight, we conjecture that the

observed throughput variability during the period 300 to 700 seconds of Measurement

scenario 4 (Trace 4-2 illustrated in Figure 11) is driven by TCP congestion control rather

than application adaptation. In future work we plan to explore in more detail the

parameter space associated with DASH applications. In particular we want to explore

how certain parameter settings, such as playback buffer capacity and adaptation

responsiveness, impact system performance. We would also like to better understand the

behavior of large numbers of competing Netflix sessions. Specific questions are:

Does Netflix give up too much bandwidth when competing with greedy, always-

on TCP flows?

Can aggregate Netflix traffic be accurately modeled using tractable analytic

methods?

Can the optimal buffer capacity or the most appropriate AQM strategy be

identified for a given router or CMTS subject to an expected number of Netflix

sessions?

The obvious next step must address the issue of how network impairment (which leads to

subsequent changes to bandwidth consumption) impact the perceived quality. The

performance of video streaming is challenging to assess since the details of the video

encoding and decoding process significantly impact the perceived quality., especially

when the streams are subject to network impairment. In future work we will correlate

network measurable results with perceived quality.

References [3GPP12] 3GPP TS 26.247 version 10.1.0 Release 10: “Transparent End-to-end Packet

Switched Streaming Service (PSS); Progressive Download and Dynamic Adaptive

Service over HTTP”, 3GPP, January 2012, available online at

http://www.etsi.org/deliver/etsi_ts/126200_126299/126247/10.01.00_60/ts_126247v1001

00p.pdf

Formatted: Bulleted + Level: 1 + Aligned at: 0.29" + Indent at: 0.54"

32

[ABD11] S. Akhshabi, A. Begen, C. Dovrolis, “An Experimental Evaluation of Rate-

Adaptation Algorithms in Adaptive Streaming over HTTP”, Proceedings of ACM

MMSys, February 2011.

[ACH12] T. Andelin, V. Chetty, D. Harbaugh, S. Warnick, D. Zappala, “Quality

Selection for Dynamic Adaptive Streaming over HTTP with Scalable Video Coding”,

Proceedings of ACM MMSys, February 2012.

[AN11] S. Alcock, R. Nelson, “Application Flow Control in YouTube Video Streams”,

ACM SIGCOMM Computer Communication Review, April 2011.

[CMP11] L. Cicco, S. Mascolo, V. Palmisano, “Feedback Control for Adaptive Live

Video Streaming”, Proceedings of MMSys’11, February 2011.

[DAJ11] F. Dobrian, A. Awan, D. Joseph, A. Ganjamm J. Zhan, V. Sekar, I. Stoica, H.

Zhang, “Understanding the Impact of Video Quality on User Engagement”, Proceedings

of SIGCOMM’11, August 2011.

[EKG11] K. Evensen, D. Kaspar, C. Griwodz, P. Halvorsen, A. Hansen, P. Engelstad,

“Improving the Performance of Quality-Adaptive Video Streaming over Multiple

Heterogeneous Access Networks”, Proceedings of MMSys’11, February 2011.

[ISO11] ISO/IEC, “Information technology — MPEG systems technologies — Part 6:

Dynamic adaptive streaming over HTTP (DASH)”, Jan 2011, available online at

http://mpeg.chiariglione.org/working_documents/mpeg-dash/dash-dis.zip

[KKH10] R. Kuschnig, I. Kofler, H. Hellwagner, “Improving Internet Video Streaming

Performance by Parallel TCP-based Request-Response Streams”, Proceedings of IEEE

CCNC 2010, January 2010.

[KKH11] R. Kuschnig, I. Kofler, H. Hellwagner, “Evaluation of HTTP-based Request-

Response Streams for Internet Video Streaming”, Proceedings of MMSys’11, February

2011.

[LBG11] C. Liu, I. Bouazizi, M. Gabbouj, “Rate Adaptation for Adaptive HTTP

Streaming”, Proceedings of ACM MMSys, February 2011.

[LMT12] S. Lederer, C. Muller, C. Timmerer, “Dynamic Adaptive Streaming over HTTP

Dataset”, Proceedings of ACM MMSys, February 2012.

[MW07] J. Martin, J. Westall, “A Simulation Model of the DOCSIS Protocol”,

Simulation: Transactions of the Society for Modeling and Simulation International, 83(2),

139-155 (2007).

33

[MWF10] J. Martin, J. Westall, J. Finkelstein, G. Hart, T. Shaw, G. White, R.

Woundy, "Cable Modem Buffer Management in DOCSIS Networks", Proceedings of

the IEEE Sarnoff 2010 Symposium, (Princeton University, Princeton NJ, April 2010).

[MMW10] S. Moser, J. Martin, J. Westall, B. Dean, "The Role of Max-min Fairness in

DOCSIS 3.0 Downstream Channel Bonding", Proceedings of the IEEE Sarnoff 2010

Symposium, (Princeton University, Princeton NJ, April 2010).

[MT11] C. Muller, C. Timmerer, “A Testbed for the Dynamic Adaptive Streaming over

Http Featuring Session Mobility”, Proceedings of ACM MMSys, February 2011.

[NEE11A] P. Ni, R. Eg., A. Eichhorn, C. Griwodz, P. Halvorsen, “Flicker Effects in

Adaptive Video Streaming to Handheld Devices”, Proceedings of MM’11, November

2011.

[NEE11B] P. Ni, R. Eg., A. Eichhorn, C. Griwodz, P. Halvorsen, “Spatial Flicker Effect

in Video Scaling”, Proceedings of Quality of Multimedia Experience (QoMEX)

Workshop, September, 2011.

[OS12] O. Oyman, S. Singh, “Quality of Experience for HTTP Adaptive Streaming

Services”, IEEE Communications Magazine, April 2012.

[RLB11] A. Roa, Y. Lim, C. Barakat, A. Legout, D. Towsley, W. Dabbous, “Network

Characteristics of Video Streaming Traffic”, Proceedings of ACM CoNEXT Conference,

December 2011.

[SOD11] I. Sodagar, “The MPEG-DASH Standard for Multimedia Streaming Over the

Internet”, IEEE Multimedia magazine, October/December 2011.

[SOP12] S. Singh, O. Oyman, A. Papathanassiou, D. Chatterjee, J. Andrews, “Video

Capacity and QoE Enhancements in LTE”, submitted.

[SSH11] Y. Sanchez, T. Schierl, C. Hellge, T. Wiegand, D. Vleeshauwer, W. Keekwijck,

Y. Louedec, “iDash: Improved Dynamic Adaptive Streaming over HTTP using Scalable

Video Coding”, Proceedings of ACM MMSys, February 2011.

[STO11] T. Stockhammer, “Dynamic Adaptive Streaming over HTTP – Standards and

Design Principles”, Proceedings of the MMSys’11, February 2011.

[XST09] J. Xia, Y. Shi, K. Tuenissen, I. Heynderick, “Perceivable Artifacts in

Compressed Video and their Relation to Video Quality”, Elsevier Signal Processing:

Image Communication, vol 24, 2009

34

Appendix

Figure A1. Measurement Scenario 3 Results



0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 50 100 150 200 250 300 350 400 450 5000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

35

Figure A2. Measurement Scenario 4



0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

0 100 200 300 400 500 600 700 800 9000

2

4

6

8

10

12

14

16

18

20

Time (seconds)

Bandw

idth

Consum

ption (

Mbps)

2.0 second samples

50 second samples

analyzing the dynamic adaptive streaming over http (dash) protocol

Documents