france télécom r&d engineering for qos and the limits of service differentiation jim roberts...

France Télécom R&D

Engineering for QoS and the limits of service differentiation

Jim Roberts

([email protected])

IWQoSJune 2000


quality of service• transparency• response time• accessibility

service model• resource sharing• priorities,...

network engineering• provisioning• routing,...

feasible technology

a viable business model

The central role of QoS


Engineering for QoS: a probabilistic point of view

statistical characterization of trafficnotions of expected demand and random processesfor packets, bursts, flows, aggregates

QoS in statistical termstransparency: Pr [packet loss], mean delay, Pr [delay > x],...response time: E [response time],...accessibility: Pr [blocking],...

QoS engineering, based on a three-way relationship:

performance

capacity

demand


Outline

traffic characteristics

QoS engineering for streaming flows

QoS engineering for elastic traffic

service differentiation


Internet traffic is self-similar

a self-similar processvariability at all time scales

due to:infinite variance of flow sizeTCP induced burstiness

a practical consequencedifficult to characterize a traffic

aggregate

Ethernet traffic, Bellcore 1989


Traffic on a US backbone link (Thomson et al, 1997)

traffic intensity is predictable ... ... and stationary in the busy hour


Traffic on a French backbone link

traffic intensity is predictable ... ... and stationary in the busy hour

12h 18h 00h 06h

tue wed thu fri sat sun mon


IP flows

a flow = one instance of a given applicationa "continuous flow" of packets basically two kinds of flow, streaming and elastic


IP flows


streaming flowsaudio and video, real time and playbackrate and duration are intrinsic characteristicsnot rate adaptive (an assumption)QoS negligible loss, delay, jitter


IP flows


streaming flowsaudio and video, real time and playbackrate and duration are intrinsic characteristicsnot rate adaptive (an assumption)QoS negligible loss, delay, jitter

elastic flowsdigital documents ( Web pages, files, ...)rate and duration are measures of performanceQoS adequate throughput (response time)


Flow traffic characteristics

streaming flowsconstant or variable rate

compressed audio (O[103 bps]) and video (O[106 bps])highly variable durationa Poisson flow arrival process (?)

variable rate video


Flow traffic characteristics

streaming flowsconstant or variable rate

compressed audio (O[103 bps]) and video (O[106 bps])highly variable durationa Poisson flow arrival process (?)

elastic flowsinfinite variance size distributionrate adaptivea Poisson flow arrival process (??)

variable rate video


Modelling traffic demand

stream traffic demandarrival rate x bit rate x duration

elastic traffic demand arrival rate x size

a stationary process in the "busy hour"eg, Poisson flow arrivals, independent flow size

busy hour

trafficdemand

Mbit/s

time of day


Outline






Open loop control for streaming traffic

a "traffic contract"QoS guarantees rely ontraffic descriptors + admission control + policing

time scale decomposition for performance analysispacket scaleburst scaleflow scale

user-networkinterface

network-networkinterface

user-networkinterface


Packet scale: a superposition of constant rate flows

constant rate flowspacket size/inter-packet interval = flow ratemaximum packet size = MTU




buffer size for negligible overflow?over all phase alignments......assuming independence between flows

buffer size

log Pr [saturation]





worst case assumptions:many low rate flowsMTU-sized packets

buffer size

log Pr [saturation]

increasing number,increasing pkt size





worst case assumptions:many low rate flowsMTU-sized packets

buffer sizing for M/DMTU/1 queuePr [queue > x] ~ C e -r x

buffer size

log Pr [saturation]

M/DMTU/1

increasing number,increasing pkt size


The "negligible jitter conjecture"

constant rate flows acquire jitternotably in multiplexer queues




conjecture: if all flows are initially CBR and in all queues: flow rates < service ratethey never acquire sufficient jitter to become worse for performance than a

Poisson stream of MTU packets




conjecture: if all flows are initially CBR and in all queues: flow rates < service ratethey never acquire sufficient jitter to become worse for performance than a

Poisson stream of MTU packets

M/DMTU/1 buffer sizing remains conservative


Burst scale: fluid queueing models

assume flows have an intantaneous rateeg, rate of on/off sources

packets

bursts

arrival rate


Burst scale: fluid queueing models

assume flows have an intantaneous rateeg, rate of on/off sources

bufferless or buffered multiplexing?Pr [arrival rate < service rate] < E [arrival rate] < service rate

packets

bursts

arrival rate


Pr [rate overload]

log Pr [saturation]

buffer size

0

0

Buffered multiplexing performance: impact of burst parameters



log Pr [saturation]

buffer size

0

0

burst length

shorter

longer



burst length

less variable

more variable

log Pr [saturation]

buffer size

0

0



burst length

short range dependence

long range dependence

log Pr [saturation]

buffer size

0

0


Choice of token bucket parameters?

r

b

the token bucket is a virtual queueservice rate rbuffer size b



r

b


non-conformance depends onburst size and variabilityand long range dependence

b'

non-conformance

probability

b



r

b


non-conformance depends onburst size and variabilityand long range dependence

a difficult choice for conformancer >> mean rate......or b very large

b'

non-conformance

probability

b


outputrate C

combinedinput

rate t

time

Bufferless multiplexing: alias "rate envelope multiplexing"

provisioning and/or admission control to ensure Pr [t>C] < performance depends only on stationary rate distribution

loss rate E [(t -C)+] / E [t]

insensitivity to self-similarity


Efficiency of bufferless multiplexing

small amplitude of rate variations ...peak rate << link rate (eg, 1%)




... or low utilisationoverall mean rate << link rate




... or low utilisationoverall mean rate << link rate

we may have both in an integrated networkpriority to streaming trafficresidue shared by elastic flows


Flow scale: admission control

accept new flow only if transparency preservedgiven flow traffic descriptorcurrent link status

no satisfactory solution for buffered multiplexing(we do not consider deterministic guarantees)unpredictable statistical performance

measurement-based control for bufferless multiplexinggiven flow peak ratecurrent measured rate (instantaneous rate, mean, variance,...)


Flow scale: admission control

accept new flow only if transparency preservedgiven flow traffic descriptorcurrent link status

no satisfactory solution for buffered multiplexing(we do not consider deterministic guarantees)unpredictable statistical performance

measurement-based control for bufferless multiplexinggiven flow peak ratecurrent measured rate (instantaneous rate, mean, variance,...)

uncritical decision threshold if streaming traffic is lightin an integrated network


Provisioning for negligible blocking

"classical" teletraffic theory; assume Poisson arrivals, rate constant rate per flow rmean duration 1/ mean demand, A = r bits/s

blocking probability for capacity C B = E(C/r,A/r)E(m,a) is Erlang's formula:

E(m,a)= scale economies

mi ia

ma im

!! /

0 20 40 60 80 100

0.2

0.4

0.6

0.8

utilization (=a/m) for E(m,a) = 0.01

m


Provisioning for negligible blocking

"classical" teletraffic theory; assume Poisson arrivals, rate constant rate per flow rmean duration 1/ mean demand, A = r bits/s

blocking probability for capacity C B = E(C/r,A/r)E(m,a) is Erlang's formula:

E(m,a)= scale economies

generalizations exist: for different ratesfor variable rates

mi ia

ma im

!! /

0 20 40 60 80 100

0.2

0.4

0.6

0.8

utilization (=a/m) for E(m,a) = 0.01

m


Outline






Closed loop control for elastic traffic

reactive controlend-to-end protocols (eg, TCP)queue management

time scale decomposition for performance analysispacket scaleflow scale


a multi-fractal arrival process

Packet scale: bandwidth and loss rate


a multi-fractal arrival processbut loss and bandwidth related by TCP (cf. Padhye et al.)

B(p)

loss ratep

congestionavoidance




B(p)

loss ratep

congestionavoidance

1

64

1

0321833132

1

0

0

20

pT

ppppTpRTT

pRTT

W

pB

for

> small for)(/,min/

= for

)(

max




thus, p = B-1(p): ie, loss rate depends on bandwidth share

B(p)

loss ratep

congestionavoidance

1

64

1

0321833132

1

0

0

20

pT

ppppTpRTT

pRTT

W

pB

for

> small for)(/,min/

= for

)(

max



Packet scale: bandwidth sharing

reactive control (TCP, scheduling) shares bottleneck bandwidth unequally

depending on RTT, protocol implementation, etc.and differentiated services parameters





optimal sharing in a network: objectives and algorithms...max-min fairness, proportional fairness, maximal utility,...

route 0

route 1 route L

Example: a linear network





optimal sharing in a network: objectives and algorithms...max-min fairness, proportional fairness, maximal utility,...

... but response time depends more on traffic process than the static sharing algorithm!

route 0

route 1 route L

Example: a linear network


Flow scale: performance of a bottleneck link

assume perfect fair shareslink rate C, n elastic flows each flow served at rate C/n fair shares

link capacity C



assume perfect fair shareslink rate C, n elastic flows each flow served at rate C/n

assume Poisson flow arrivals an M/G/1 processor sharing queueload, = arrival rate x size / C a processor sharing queue

fair shares

link capacity C




assume Poisson flow arrivals an M/G/1 processor sharing queueload, = arrival rate x size / C

performance insensitive to size distributionPr [n transfers] = n(1-)E [response time] = size / C(1-)

100

throughputC

a processor sharing queue

fair shares

link capacity C




assume Poisson flow arrivals an M/G/1 processor sharing queueload, = arrival rate x size / C

performance insensitive to size distributionPr [n transfers] = n(1-)E [response time] = size / C(1-)

instability if > 1i.e., unbounded response timestabilized by aborted transfers...... or by admission control

100

throughputC

a processor sharing queue

fair shares

link capacity C


Generalizations of PS model

non-Poisson arrivalsPoisson sessionsBernoulli feedback

Poissonsessionarrivals

p

1-pflows

think time

transfer

processor sharing

infiniteserver




discriminatory processor sharingweight i for class i flows

service rate i


p

1-pflows

think time

transfer

processor sharing

infiniteserver




discriminatory processor sharingweight i for class i flows

service rate i

rate limitations (same for all flows)maximum rate per flow (eg, access rate)minimum rate per flow (by admission control)


p

1-pflows

think time

transfer

processor sharing

infiniteserver


Admission control can be useful


Admission control can be useful ...

... to prevent disasters at sea !


Admission control can also be useful for IP flows

improve efficiency of TCPreduce retransmissions overhead ...... by maintaining throughput

prevent instabilitydue to overload ( > 1)......and retransmissions

avoid aborted transfersuser impatience"broken connections"

a means for service differentiation...


Choosing an admission control threshold

N = the maximum number of flows admitted negligible blocking when <maintain quality when >

0 100 200 N

300

200

100

0

E [Response time]/size

0 100 200 N

1

.8

.6

.4

.2

0

Blocking probability




M/G/1/N processor sharing system min bandwidth = C/N Pr [blocking] = N(1 - )/(1 - N+1) (1 - 1/for>

0 100 200 N

300

200

100

0


= 0.9

= 1.5

0 100 200 N

1

.8

.6

.4

.2

0


= 0.9

= 1.5




M/G/1/N processor sharing system min bandwidth = C/N Pr [blocking] = N(1 - )/(1 - N+1) (1 - 1/for>

uncritical choice of threshold eg, 1% of link capacity (N=100)

0 100 200 N

300

200

100

0


= 0.9

= 1.5

0 100 200 N

1

.8

.6

.4

.2

0


= 0.9

= 1.5


backbone link(rate C)

access links(rate<<C)

100

throughput C

accessrate

Impact of access rate on backbone sharing

TCP throughput is limited by access rate...modem, DSL, cable

... and by server performance


backbone link(rate C)

access links(rate<<C)

100

throughput C

accessrate

Impact of access rate on backbone sharing

TCP throughput is limited by access rate...modem, DSL, cable

... and by server performance

backbone link is a bottleneck only if saturated!ie, if > 1


Provisioning for negligible blocking for elastic flows

"elastic" teletraffic theory; assume Poisson arrivals, rate mean size s

blocking probability for capacity Cutilization = s/Cm = admission control limit B(,m) = m(1-)/(1-m+1)

0 20 40 60 80 100

0.2

0.4

0.6

0.8

utilization () for B = 0.01

m


Provisioning for negligible blocking for elastic flows

"elastic" teletraffic theory; assume Poisson arrivals, rate mean size s

blocking probability for capacity Cutilization = s/Cm = admission control limit B(,m) = m(1-)/(1-m+1)

impact of access rateC/access rate = mB(,m) E(m,m)

0 20 40 60 80 100

0.2

0.4

0.6

0.8

utilization () for B = 0.01

m

E(m,m)


Outline






Service differentiation

discriminating between stream and elastic flowstransparency for streaming flowsresponse time for elastic flows

discriminating between stream flowsdifferent delay and loss requirements... or the best quality for all?

discriminating between elastic flowsdifferent response time requirements... but how?


Integrating streaming and elastic traffic

priority to packets of streaming flowslow utilization negligible loss and delay




elastic flows use all remaining capacitybetter response timesper flow fair queueing (?)





to prevent overload flow based admission control......and adaptive routing





to prevent overload flow based admission control......and adaptive routing

an identical admission criterion for streaming and elastic flowsavailable rate > R


Differentiation for stream traffic

different delays? priority queues, WFQ, ... but what guarantees?

delay

delay




different loss? different utilization (CBQ, ...) "spatial queue priority"

partial buffer sharing, push out

delay

delay

loss loss




different loss? different utilization (CBQ, ...) "spatial queue priority"

partial buffer sharing, push out

or negligible loss and delay for all elastic-stream integration ... ... and low stream utilization

lossdelay

delay

delay

loss loss


Differentiation for elastic traffic

different utilizationseparate pipesclass based queuing

100

throughputC

access

rate1 st class

3 rd class

2 nd class




different per flow sharesWFQimpact of RTT,... 10

0

throughputC

accessrate

1 st class

3 rd class

2 nd class

100

throughputC

access

rate




different per flow sharesWFQimpact of RTT,...

discrimination in overloadimpact of aborts (?)or by admission control

100

throughputC

accessrate

1 st class

3 rd class

2 nd class

100

throughputC

accessrate


Different accessibility

block class 1 when 100 flows in progress - block class 2 when N2 flows in progress

0 100 200 N

1

0


= 0.9

= 1.5




0

1 = 2 = 0.4

0 N2

1

100




in underload: both classes have negligible blocking (B1 B2 0)

0

1 = 2 = 0.4

0 N2

1

100

B2B10


.33

0

1 = 2 = 0.6

0 100N2

1




in overload: discrimination is effective if 1 < 1 < 1 + 2, B1 0, B2 (1+2-1)/2

B1

B2

B2B100

1 = 2 = 0.4

0 N2

1





in overload: discrimination is effective if 1 < 1 < 1 + 2, B1 0, B2 (1+2-1)/2

if 1 < 1, B1 (1-1)/1, B2 1

B1

B21

.17

1 = 2 = 1.2

0 100N2

B2

B1

.33

0

1 = 2 = 0.6

0 100N2

1

B2B100

1 = 2 = 0.4

0 N2

1


Service differentiation and pricing

different QoS requires different prices...or users will always choose the best

...but streaming and elastic applications are qualitatively different choose streaming class for transparencychoose elastic class for throughput

no need for streaming/elastic price differentiation

different prices exploit different "willingness to pay"... bringing greater economic efficiency

...but QoS is not stable or predictable depends on route, time of day,.. and on factors outside network control: access, server, other networks,...

network QoS is not a sound basis for price discrimination


Pricing to pay for the network

fix a price per byteto cover the cost of infrastructure and operation

estimate demandat that price

provision network to handle that demandwith excellent quality of service

demand

time of day

capacity


Pricing to pay for the network

fix a price per byteto cover the cost of infrastructure and operation

estimate demandat that price

provision network to handle that demandwith excellent quality of service

demand

time of day

$$$

capacity

$$$

demand

time of day

capacity

optimal price revenue = cost


Outline





conclusions


Conclusions

a statistical characterization of demanda stationary random process in the busy perioda flow level characterization (streaming and elastic flows)


Conclusions


transparency for streaming flowsrate envelope ("bufferless") multiplexingthe "negligible jitter conjecture"


Conclusions



response time for elastic flowsa "processor sharing" flow scale modelinstability in overload (i.e., E [demand]> capacity)

10

0

C


Conclusions



response time for elastic flowsa "processor sharing" flow scale modelinstability in overload (i.e., E [demand]> capacity)

service differentiationdistinguish streaming and elastic classeslimited scope for within-class differentiationflow admission control in case of overload

10

0

C

france télécom r&d engineering for qos and the limits of service differentiation jim roberts...

Documents

france tlcom rd traffic

france tlcom rd engineering

france tlcom rd ip

elastic slide

variable rate video

kinds of flow

jitter slide

playback rate