fundamental characteristics of queues with fluctuating load (appeared in sigmetrics 2006) varun...

Fundamental Characteristics of Queues with Fluctuating Load

(appeared in SIGMETRICS 2006)

VARUN GUPTA

Joint with:

Mor Harchol-Balter

Carnegie Mellon Univ.

Alan Scheller-Wolf

Carnegie Mellon Univ.

Uri Yechiali

Tel Aviv Univ.

2

Motivation

ClientsServer Farm

Requests

3

Motivation

ClientsServer Farm

Requests

4

Motivation

ClientsServer Farm

Requests

5

Motivation

ClientsServer Farm

Requests

6

Motivation

ClientsServer Farm

Requests

7

Motivation

ClientsServer Farm

Requests

8

Motivation

ClientsServer Farm

Requests

9

Motivation

ClientsServer Farm

RequestsReal

World Fluctuating arrival

and service intensities

10

A Simple Model

HL

exp(H)

exp(L)

HighLoad

LowLoad

11

• Poisson Arrivals• Exponential Job Size Distribution• H/H > L/L

• H>H possible, only need stability

A Simple Model

HighLoad

LowLoad

H,H

L,L

exp()

exp()

HH

LL

12

The Markov ChainP

has

e

Number of jobs

L

H

H

H

0 1

0 1

2

2

L

L

H

H

L

L

. . .

. . .

Solving the Markov chain provides no behavioral insight

13

HH

LL

• N = Number of jobs in the fluctuating load system

• Lets try approximating N using (simpler) non-fluctuating systems

14

HH

LL

Method 1

Nmix

15

HH

LL

Q: Is Nmix ≈ N?

A: Only when 0

Method 1

Nmix

½

½

+

,

16

HH

LL

Method 2

17

avg(H,L)avg(H,L)

Method 2

≡ Navg

Q: Is Navg ≈ N?

A: When ,

18

Example

H=1, H=0.99

L=1, L=0.01

E[Nmix] ≈ 49.5 E[Navg] = 1

0

19

Observations

• Fluctuating system can be worse than non-fluctuating

0 and asymptotes can be very far apart

E[Nmix] > E[Navg]

E[Nmix] E[Navg]

20

Questions

• Is fluctuation always bad?

• Is E[N] monotonic in ?

• Is there a simple closed form approximation for E[N] for intermediate ’s?

• How do queue lengths during High Load and Low Load phase compare? How do they compare with Navg?

More than 40 years of research has not

addressed such fundamental questions!

21

Outline

Is E[Nmix] ≥ E[Navg], always?

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

22

Prior Work

Fluid/DiffusionApproximations

Transforms Matrix Analytic& Spectral Analysis

- P. Harrison- Adan and Kulkarni

Numerical ApproachesInvolves solution of cubic

- Clarke- Neuts- Yechiali and Naor

Involves solution of cubic

- Massey- Newell- Abate, Choudhary, Whitt

Limiting Behavior

But cubic equations have a close form solution…

?

23

Good luck understanding this!

24

Asymptotics for E[N] (H<H)

E[Navg]

E[Nmix]

E[N]

(switching rate)Highfluctuation

H=1, H=0.99

L=1, L=0.01

E[Nmix] > E[Navg]

Lowfluctuation

25

Asymptotics for E[N] (H<H)

E[N]

E[Nmix]

E[Navg]

• Agrees with our example (H = L)

• Ross’s conjecture for systems with constant service rate:

“Fluctuation increases mean delay”

Q: Is this behavior possible?

A: Yes

E[N]

E[Navg]

E[Nmix]

26

Our Results

E[N]

(H-H) > (L-L)(H-H) = (L-L)(H-H) < (L-L)

• Define the slacks during L and H as• sL = L - L

• sH = H - H

E[N]

E[N]

27

Our Results

• Define the slacks during L and H as• sL = L - L

• sH = H - H

• Not load but slacks determine the response times!

sH > sLsH = sLsH < sL

KEY IDEA

E[N]

E[N]

E[N]

28

Outline

Is E[Nmix] ≥ E[Navg], always?

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

29

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

30

Monotonicity of E[N]

:

MeanQueueLength

H L

’ :

MeanQueueLength

H L

31

Monotonicity of E[N]

We show : E[N] is monotonic in

:

MeanQueueLength

H L

’ :

MeanQueueLength

H L

Not obvious that true for all ,’ with ’< !

32

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

33

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

34

Approximating E[N]

• Express the first moment as *

E[N] = E[Nmix]r+E[Navg](1-r)

• Approximate r by the root of a quadratic

KEY IDEA

KEY IDEA

* True for H < H; a similar expression exists for case of transient overload

35

Approximating E[N]

1

3

5

7

9

1.E-05 1.E-04 1.E-03 1.E-02 1.E-01 1.E+00 1.E+0110-5 10-4 10-3 10-2 10-1 100 101

3

5

7

9

E[N]

ExactApprox.

H=L=1, H=0.95, L=0.2

36

Approximating E[N]

1

3

5

7

9

1.E-05 1.E-04 1.E-03 1.E-02 1.E-01 1.E+00 1.E+0110-5 10-4 10-3 10-2 10-1 100 101

3

5

7

9

E[N]

ExactApprox.

H=L=1, H=0.95, L=0.2

37

Approximating E[N]

2

6

10

14

18

1.E-02 1.E-01 1.E+00 1.E+0110-2 10-1 100 10

ExactApprox.

H=L=1, H=1.2, L=0.2

2

6

10

14

18

E[N]

38

Approximating E[N]

2

6

10

14

18

1.E-02 1.E-01 1.E+00 1.E+0110-2 10-1 100 10

ExactApprox.

H=L=1, H=1.2, L=0.2

2

6

10

14

18

E[N]

39

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

40

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

41

Stochastic Ordering refresher

• For random variables X and Y

X st Y Pr{Xi} Pr{Yi}

for all i.

• X stY E[f(X)] E[f(Y)] for all increasing f

– E[Xk] E[Yk] for all k 0.

42

Notation

• NH: Number of jobs in system during H phase

• NL: Number of jobs in system during L phase

• N = (NH+NL)/2

H,H

L,L

exp()

exp()NH NL

43

Stochastic Orderings for NL, NH

• NL ≥st NM/M/1/L

• NH ≤st NM/M/1/H

• NH ≥st NL

• NH ≥st Navg

• NL st Navg

?

?

?

?

?

H,H

L,L

exp()

exp()NH NL

NH increasesstochastically as ↓

Conjecture:

44

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

45

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

Application: Capacity Planning

46

Scenario

Application: Capacity Provisioning

HH

LL

2HH

2LL

Aim: To keep the mean response times same

47

Scenario

Application: Capacity Provisioning

HH

LL

2H2H

2L2L

Question: What is the effect of doubling the arrival and service rates on the mean response time?

48

What happens to the mean response time when , are doubled in the fluctuating load queue?

Halves

Remains almost the sameReduces by less than half

Reduces by more than halfA:

D:C:

B:

49

What happens to the mean response time when , are doubled in the fluctuating load queue?

Halves

Remains almost the sameReduces by less than half

Reduces by more than halfA:

D:C:

B:

50

What happens to the mean response time when , are doubled in the fluctuating load queue?

Halves

Remains almost the sameReduces by less than half

Reduces by more than halfA:

D:C:

B:

Look at slacks!

A: sH = sL

B: sH > sL

C: sH < sL

D: sH < 0, 0

reduces by half more than half less than half remains same

51

Our Contributions

• Give a simple characterization of the behavior of E[N] vs.

• Provide simple (and tight) quadratic approximations for E[N]

• Prove the first stochastic ordering results for the fluctuating load model

52

Thank you

53

Analysis of E[N]

First steps:

– Note that it suffices to look at switching points

– Express

• NL = f(NH)

• NH = g(NL)

– The problem reduces to finding Pr{NH=0} and Pr{NL=0}

H,H

L,L

NH NL

NL=f(g(NL))

fg

54

– Find the root of a cubic (the characteristic matrix polynomial in the Spectral Expansion method)

– Express E[N] in terms of

E[N] =

The simple way forward…

H,H

L,L

fg

A

A-A

H(L -L)0H+ L(H-H)0

L - (L -L)(H-H)

2 (A -A)+

Where 0L = 0

H = (A-A)

L(-1)(H-H)

(A-A)

H(-1)(L-L)

NH NL Difficult to even prove the monotonicity of E[N] wrt

using this!

55

Our approach (contd.)

• Express the first moment as

E[N] = f1()r+f0()(1-r)

– r is the root of a (different) cubic– r1 as 0 and r0 as

KEY IDEA

56

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

57

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

Need at least 3 roots for when r=c1

but has at most 2 roots

c1

58

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

Need at least 2 positive roots for when r=c2

but for r>1 product of roots is negative

c2

59

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

E[N] is monotonic in !

60

Why do slacks matter?

• Fact: The mean response time in an M/M/1 queue is (-)-1

– Higher slacks Lower mean response times

• What is the fraction of customers departing during H

H,H

L,L

exp()

exp()

when ?

H,H

L,L

exp()

exp()

when 0?

H

H + L

H

H + L?

61

Why do slacks matter?

when ?

H,H

L,L

exp()

exp()

when 0?

H

H + L

H

H + L?

• Fact: The mean response time in an M/M/1 queue is (-)-1

– Higher slacks Lower mean response times

• What is the fraction of customers departing during H

H,H

L,L

exp()

exp()

62

Why do slacks matter?

when ?

H,H

L,L

exp()

exp()

when 0?

A H?

• Fact: The mean response time in an M/M/1 queue is (-)-1

– Higher slacks Lower mean response times

• What is the fraction of customers departing during H

As switching rates decrease, larger fraction of customers experience lower mean response times when sH>sL

H,H

L,L

exp()

exp()

63

Q: What happens to E[N] when we double ’s and ’s?

A:System A: , ,

System B: 2, 2,

?

64

Q: What happens to E[N] when we double ’s and ’s?

A:System A: , ,

System B: 2, 2,

System C: 2, 2, 2

E[N] remains same in going from A to C

A) sL = sH : remains same

B) sL > sH : increases, but by less than twice

C) sL < sH : decreases

D) 0, H>1 : queue lengths become twice as switching rates halve, E[N] doubles