Statistical Modeling for Per-Hop Statistical Modeling for Per-Hop QoSQoS

Statistical Modeling for Per-Hop Statistical Modeling for Per-Hop QoSQoS

Mohamed El-Gendy(mgendy@eecs.umich.edu)

In collaboration withAbhijit Bose, Haining Wang, and Prof. Kang G.

Shin

Real-Time Computing Laboratory EECS Department

The University of Michigan@Ann Arbor

June 4th, 2003

OutlineOutlineOutlineOutline

•Intro of DiffServ, PHB, and Per-Hop QoS•Motivations•Related Work•Approach to Statistical Characterization•Experimental Framework•Results and Analysis•A Control Example•Conclusions and Future Work

DiffServ and PHBDiffServ and PHBDiffServ and PHBDiffServ and PHB

•Scalable network-level QoS based on marked “traffic aggregates”

•Traffic is conditioned and marked with DSCP at edge

•Per-Hop Behaviors (PHBs) are applied to traffic aggregates at core

•QoS is achieved through different PHBs:– Expedited Forwarding (EF) for delay assurance– Assured Forwarding (AF) for bandwidth

assurance

Per-Hop QoSPer-Hop QoSPer-Hop QoSPer-Hop QoS

•Throughput (BW), delay (D), jitter (J), and loss (L) experienced by traffic crossing a PHB

DiffServ node

PHB

Input traffic c/c’sI

Output traffic c/c’sO (BW, D, J, L)

Configuration parametersC

MotivationsMotivationsMotivationsMotivations

• Why modeling Per-Hop QoS?– PHB is the key building block of DiffServ– Wide variety of PHB realizations– PHB control and configuration– Necessary for end-to-end QoS calculation

• Benefits of PHB modeling:– Facilitates the control and optimization of PHB

performance– Enables contribution of per-hop admission control to e2e

admission control decisions

Related WorkRelated WorkRelated WorkRelated Work

•Study of TCP ACK marking in DiffServ [IWQoS’01] :– Used full factorial design and ANOVA– Compared many marking schemes for TCP acks– Suggested an optimal strategy for marking the

acks for both assured and premium flows– Used ns simulation for analysis

•AF performance using ANOVA [IETF draft]:– Compared different bandwidth and buffer

management schemes for their effect on AF performance

Related WorkRelated WorkRelated WorkRelated Work

•Performance of TCP Vegas [Infocom’00] :– Used ANOVA to test the effect of ten congestion

and flow control algorithms– Clustered the ten factors into three groups

according to the the three phases of the TCP Vegas operation

Approach to Statistical Approach to Statistical CharacterizationCharacterization

Approach to Statistical Approach to Statistical CharacterizationCharacterization

•Identify the factors in I and C that affect output per-hop QoS most

•Construct statistical models of the per-hop QoS in terms of these important factors

Fullfactorialdesign

Run Exp. &collect

dataANOVA Tests Regression

Scenariofile

Adjust scenario parameters

Input Traffic Factors - IInput Traffic Factors - IInput Traffic Factors - IInput Traffic Factors - I

Dual Leaky Bucket (DLB) representation for I:– Average rate, peak rate, burst size, packet size,

number of flows per aggregate, and traffic type

– Ia : assured traffic, Ib: background traffic

– Used the ratio between assured to best-effort traffic, instead of absolute value

– Number of input interfaces to PHB node

Alternative PHB RealizationsAlternative PHB RealizationsAlternative PHB RealizationsAlternative PHB Realizations

Priority

EF

BE

RED

FIFO

CBQ/WFQ

EF

BE

RED

FIFO

EF TBF

Priority

BE

RED

EF-EDGE EF-CORE EF-CBQ

•Different PHB realizations have different functional relationships between inputs and outputs

Configuration Parameters- CConfiguration Parameters- CConfiguration Parameters- CConfiguration Parameters- C

Configuration parameters depend on PHB realization:– EF-EDGE: token rate, bucket size, and MTU– EF-CORE: queue length– EF-CBQ: service rate, burst size, and avg packet

size– AF: min. threshold, max. threshold, and drop

probability

Statistical AnalysisStatistical AnalysisStatistical AnalysisStatistical Analysis

Analysis of Variance (ANOVA)– Models

Output response as a linear combination of the main effects and their interactions

– Allocation of variationCalculate the percentage of variation in the output

response due to factors at each level, their interactions, and the errors in the experiments

– ANOVAStatistically compare the significance of each factor as

well as the experimental error

ANOVAANOVAANOVAANOVA

•For any three factors (k = 3), A, B, and C with levels a, b, and c, and with r repetitions, the response variable y can be written as:

rlckbjai

ey ijklABCijkBCjkACikABijkjiijkl

,,1 ,,1 ,,1 ,,1

ijkijklijkl

kjiBCjkACikABijijkABCijk

jiijABij

jjii

yye

y

y

yy

.

..

......

,

, ,

ANOVAANOVAANOVAANOVA

•Squaring both sides we get:

ijklijkl

ijkijk

jkjk

ikik

ijij

kk

jj

ii

ijklijkl

erarbrcr

abracrbcrabcry

22222

22222

SSESSABCSSBCSSACSSABSSCSSBSSASSSSY 0

0)( 2 SSSSYySSTijkl

ijkl

SSE: sum of squared errors

ANOVAANOVAANOVAANOVA

Effect SS %age DF MS F

SSY abcr

SS0 1

SST abcr-1

A SSA a-1

AB SSAB (a-1)(b-1)

ABC SSABC(a-1)(b-1)(c-

1)

e SSE abc(r-1)

....y

y

....yy

SST

SSA100

SST

SSAB100

SST

SSABC100

SST

SSE100

1aSSA

)1)(1( ba

SSAB

)1)(1)(1( cba

SSABC

)1( rabc

SSE

MSE

MSA

MSE

MSAB

MSE

MSABC

100

ANOVA Model Assumptions ANOVA Model Assumptions ANOVA Model Assumptions ANOVA Model Assumptions

• Assumptions:• Effects of input factors and errors are additive• Errors are identical, independent, and normally

distributed random variables• Errors have a constant standard deviation

• Visual tests:• No trend in the scatter plot of residuals vs. predicted

response• Linear normal quantile-quantile (Q-Q) plot of residuals

No assumptions about the nature of the statistical relationship between input factors and response variables

Statistical Analysis - Statistical Analysis - RegressionRegression

Statistical Analysis - Statistical Analysis - RegressionRegression

Polynomial regression– A variant of multiple linear regression– Any complex function can be expanded into

piecewise polynomials with enough number of terms

– Transformations to deal with nonlinear dependency

– Coefficient of determination (R2) as a measure of the regression goodness

RegressionRegressionRegressionRegression

ni

exbxbxbby ikikiioi

12211

iiiiiiioi exxbxbxbxbxbby 21522423

21211

Linear model for one dependent variable y and k independent variables x :

Polynomial model for two independent variables x1, x2:

iiiiiiiiiii xxzxzxzxzxz 21522423

21211

Transformation to fit into linear model:

Experimental FrameworkExperimental FrameworkExperimental FrameworkExperimental Framework

Framework components:– Traffic Generation Agent

•Generates both TCP and UDP traffic•Policed with a built-in leaky bucket for profiled traffic•BW, D, J, L are measured within the agent itself

– Controller and Remote Agents•Control the flow of the experiments according to a

distributed scenario file•Executes and keep track of the experiments steps and

other components

– Network and Router Configuration Agents•Configure traffic control blocks on router according to

experiment scenarios•Receive scenario commands from the Controller agent•Current implementation works on Linux traffic control

– Analysis Module•Performs ANOVA, model validation tests, and

polynomial regression on output data

Experimental Framework, Experimental Framework, cont’dcont’d

Experimental Framework, Experimental Framework, cont’dcont’d

Experimental Framework, Experimental Framework, cont’dcont’d

Experimental Framework, Experimental Framework, cont’dcont’d

Network Setup– Using ring topology for one-way delay

measurements

M

switch

Router ConfigurationAgent

Network ConfigurationAgent

NI

S

H

1 2 3 4 5 6

TrafficAgent

TrafficAgent

TrafficAgent

TrafficAgent

TrafficAgent

TrafficAgent

TrafficAgent

ControllerAgent

AnalysisModule

TrafficAgent

RemoteAgent

Full Factorial Design of Full Factorial Design of ExperimentsExperiments

Full Factorial Design of Full Factorial Design of ExperimentsExperiments

•If we have k factors, with ni levels for the i-th factor, and repeat r times

Total number of experiments =

k

i inr1

LARGE!!

•Use factor clustering and automated experimental framework

Scenarios of ExperimentsScenarios of ExperimentsScenarios of ExperimentsScenarios of Experiments

EF PHB– Factor sets: Ia, Ib, C

– PHB configurations: EF-EDGE, EF-CORE, EF-CBQ– Operating mode: over-provisioned (OP), under-

provisioned (UP), fully-provisioned (FP)EF

EF-EDGE EF-CORE EF-CBQ

Ia+C Ib

OP1

UP2

FP3

OP4

Ia+C5

Ib6

Ia+C Ib

OP7

UP8

FP9

OP10

experimentnumber

Scenarios of Experiments, Scenarios of Experiments, cont’dcont’d

Scenarios of Experiments, Scenarios of Experiments, cont’dcont’d

AF PHB– Use AF11 as assured traffic

– Use AF12 and AF13 as background traffic

– Change max. threshold, min. threshold, and drop probability for AF11 only

EF PHB – OP, EF-EDGE w/o BG trafficEF PHB – OP, EF-EDGE w/o BG trafficEF PHB – OP, EF-EDGE w/o BG trafficEF PHB – OP, EF-EDGE w/o BG traffic

BW surface response: significant factors are assured rate ( ar ) and number of assured flows ( an ), R2 = 96%

nrnrnr aahahahahahhBW 52

42

3210

EF PHB – OP, EF-EDGE w/o BG trafficEF PHB – OP, EF-EDGE w/o BG trafficEF PHB – OP, EF-EDGE w/o BG trafficEF PHB – OP, EF-EDGE w/o BG traffic

J model and visual tests

ar apkt an (ar, apkt) Error

1/J 4.12%

12.52%

46.58%

2.88% 25.34%

Significant factors are: assured rate ( ar ), number of assured flows ( an ), and assured packet size (apkt)

EF PHB – UP, EF-EDGE w/o BG trafficEF PHB – UP, EF-EDGE w/o BG trafficEF PHB – UP, EF-EDGE w/o BG trafficEF PHB – UP, EF-EDGE w/o BG traffic

•ANOVA results for LSignificant factors are: assured rate ( ar ), number of

assured flows ( an ), and the token bucket rate ( efr )

ar an efr

(ar,an

)(ar, efr) (an, efr) (ar, an ,efr)

1/L 6.06%20.81

%6.63% 15.1% 8.9% 16.21% 25%

•Regression model for L

318

217

216

315

21413

212

211

210

310

298

27

652

43210/1

rrn

rnnrrrnrnr

rrnrrrrnn

rrnrrrnr

efhefah

efahahefahefaahaah

efahaahahefhefahah

efahaahahefhahahhL

•ANOVA results for BW, D, and JSignificant factors are: BG packet size ( bpkt ), number of BG

flows ( bn ), and ratio of assured to BG traffic ( Rab )

EF PHB – OP, EF-EDGE w/ BG trafficEF PHB – OP, EF-EDGE w/ BG trafficEF PHB – OP, EF-EDGE w/ BG trafficEF PHB – OP, EF-EDGE w/ BG traffic

bpkt bn Rab (bpkt,bn) (bpkt, Rab) (bn, Rab)(bpkt,

bn ,Rab)

BW 0% 36.6% 2.65% 0% 0% 2.87% 4.4%

Log(D) 3.28% 8.98% 36.3% 3.54% 10.01% 27.07% 10.69%

J 3.36% 7.5% 39.14% 3.7% 9.72% 12.13% 22.77%

•J surface response:Significant factors are assured packet size ( apkt ) and

number of assured flows ( an ), R2 = 64%

EF PHB – EF-CORE w/o BG trafficEF PHB – EF-CORE w/o BG trafficEF PHB – EF-CORE w/o BG trafficEF PHB – EF-CORE w/o BG traffic

npktpktnpktn aahahahahahhJ 52

42

3210/1

•J visual tests

EF PHB – EF-CORE w/o BG trafficEF PHB – EF-CORE w/o BG trafficEF PHB – EF-CORE w/o BG trafficEF PHB – EF-CORE w/o BG traffic

•ANOVA results for BW, D, and JSignificant factors are assured rate ( ar ), assured packet

size ( apkt ) and number of assured flows ( an )

EF PHB – OP, EF-CBQ w/o BG trafficEF PHB – OP, EF-CBQ w/o BG trafficEF PHB – OP, EF-CBQ w/o BG trafficEF PHB – OP, EF-CBQ w/o BG traffic

ar apkt an efr

(apkt, an)

(an, efr) Error

BW 96.73%

0% 0% 0% 0% 0% 0%

1/D 0%94.55

%0% 0% 0% 0% 2.51%

1/J 0% 14.1% 37.5% 6.8% 2.0% 2.0% 22.55%

•ANOVA results for LSignificant factors are BG packet size ( bpkt ), number of BG

flows ( bn ), and ratio of assured to BG traffic ( Rab )

EF PHB – OP, EF-CBQ w/ BG trafficEF PHB – OP, EF-CBQ w/ BG trafficEF PHB – OP, EF-CBQ w/ BG trafficEF PHB – OP, EF-CBQ w/ BG traffic

bpkt bn Rab (bpkt,bn) (bpkt, Rab) (bn, Rab)(bpkt,

bn ,Rab)

Log(D) 0% 0% 83.23% 2.34% 3.53% 2.04% 7%

L 8.87% 2.38% 36.89% 5.14% 25.84% 4.8% 15.45%

•D regression modelR2 = 89%

EF PHB – OP, EF-CBQ w/ BG trafficEF PHB – OP, EF-CBQ w/ BG trafficEF PHB – OP, EF-CBQ w/ BG trafficEF PHB – OP, EF-CBQ w/ BG traffic

3232

22223

22

2

90.196.189.144.388.1

99.179.051.034.124.78

61.819.125.574.289.1

86.12103.633.265.4691.1)log(

npktabpktnabpktpktn

pktnnabpktabnabab

pktpktnnpktabnab

abpktnab

bbRbbRbbb

bbbRbRbRR

bbbbbRbR

RbbRD

2

2223

2

96.1

89.179.051.034.124.78

74.289.186.12165.4691.1)log(

pktab

pktnabnabpktabnabab

pktabnababab

bR

bbRbRbRbRR

bRbRRRD

•D approximate model

AF PHBAF PHBAF PHBAF PHB

•ANOVA results for BW, D, J, and LSignificant factors are: assured rate ( ar ), assured peak rate

( ap ), assured packet size ( apkt ), max. threshold ( maxth ) , and min. threshold ( minth )

ar ap apkt maxth minth (ar, ap)

BW 51.38%

12.51% 0% 2.67% 2.38% 13.27%

D 38.2% 25.42% 2.85% 0% 3.1% 25.06%

1/J 14.75%

17.35% 34.82% 0% 0% 18.94%

L 29.32%

17.12% 0% 7.14% 4.17% 16.98%

DiscussionDiscussionDiscussionDiscussion

• BW shows a square root relationship with factors in Ia in EF-CBQ only, and direct relation in the other EF realizations

• D shows a direct relation with Ia in EF-EDGE, and EF-CORE, and inverse relation in EF-CBQ

• D shows a logarithmic (multiplicative) relation with Ib

• J shows inverse relation with Ia and a direct relation with Ib

• J depends on the number of flows in the aggregate as well as the difference in packet size with other flows/aggregates

ErrorsErrorsErrorsErrors

1. Experimental errors: due to experimental methods; captured in ANOVA

2. Model errors: due to factor truncation3. Statistical and fitting errors: due to

regression; captured in coefficient of determination (R2 )

A PHB Control ExampleA PHB Control ExampleA PHB Control ExampleA PHB Control Example

•For OP, EF-CBQ w/ BG traffic: – For bpkt = 600 B, bn = 1, Rab = 2 D = 0.4136

msec

– For bpkt = 1470 B, bn = 3, D = 0.4136 msec Rab = ??

•Use the delay model to find Rab = 0.494 with accuracy of (1-R2) = 11%

Co

nfi

gu

rati

on

C(t

)

QoSSubSystem

Model-basedController

|O - O'|

InputI(t)

OutputO(t)

RequiredOutput (O')

ConclusionsConclusionsConclusionsConclusions

• Simple statistical models are derived for per-hop QoS using ANOVA and polynomial regression

• Statistical full factorial design of experiments is an effective tool for characterizing QoS systems

• Using automated experimental framework is shown to be effective in such studies

• Different PHB realizations show differences in dependency of per-hop QoS on input factors

Extensions and Future WorkExtensions and Future WorkExtensions and Future WorkExtensions and Future Work

•The framework presented is general to be applied for studying edge-to-edge (Per-Domain Behavior or PDB) in DiffServ

•More rigorous control analysis and study of suitable control algorithms

•Validate the models derived with analytical methods such as network calculus

•Use real-time measurements to update models and control criterion while operation

Multi-Hop CaseMulti-Hop CaseMulti-Hop CaseMulti-Hop Case

•First approach

Input(I)

Output(O)

Configuration(C)

EgressIngress

Multi-Hop CaseMulti-Hop CaseMulti-Hop CaseMulti-Hop Case

•Second approach

Input(I)

Output(O)

EgressIngress

I1 Ii InO1 Oi On

CnCiC1

QuestionsQuestions ? ?QuestionsQuestions ? ?