module 7: introduction to queueing theory (notation...

ECE/CS 441: Computer System AnalysisModule 6, Slide 1

Module 7: Introduction to Queueing Theory(Notation, Single Queues, Little’s Result)

(Slides based on Daniel A. Reed, ECE/CS 441 Notes, Fall 1995, used with permission)


Outline of Section on Queueing Theory

1. Notation2. Little’s Result3. Single Queues4. Solutions for networks of queues - Product Form Results (on blackboard, not

slides)5. Mean value analysis (if time permits)


Queueing Theory Notation• Queuing characteristics

– Arrival process– Service time distribution– Number of servers– System capacity– Population size– Service discipline

• Each of these is described mathematically– Descriptions determine tractability of (efficient) analytic solution– Only a small set of possibilities are solvable using standard queueing theory


Arrival Processes

• Suppose jobs arrive at times t1 , t2 , ... ,tj

– Random variables τj = tj - tj-1 are inter-arrival times– There are many possible assumptions for the distribution of the τj

– Typical assumptions for the τj:• Independent• Identically distributed

– Many other possible assumptions:• Bulk arrivals• Balking• Correlated arrivals

• For Poisson arrival, the inter-arrival times are:– IID (independent and identically distributed)– exponentially distributed (i.e., CDF F(x) = 1 - e -x/a)

• Other common arrival time distributions include– Erlang, Hyper-exponential, Deterministic, General (with a specified mean and

variance)


Other Queue Features

• Service time– Interval spent actually receiving service (exclusive of waiting time)– As with arrival processes, there are many possible assumptions– Most common assumptions are

• IID random variables• exponential service time distribution

• Number of servers– Servers may or may not be identical– Service discipline determines allocation of customers to servers

• System capacity– Maximum number of customers in the system (including those in service)– May be finite or infinite

• Population size– Total number of potential customers– May be finite or infinite


Other Queue Features (Continued)• Service discipline

– The order waiting customers are serviced– Many possibilities, including

• First-come-first-serve (FCFS), the most common• Last-come-first-serve (LCFS)• Last-come-first-serve preempt resume (LCFS-PR)• Round robin (RR) with finite quantum size• Processor sharing (PS) --- RR with infinitesimal quantum size• Infinite server (IS)

– Almost anything might be used, depending on the the total state of the queue– As expected, service discipline affects the nature of the stochastic process

that represents the behavior of the queueing system


Queueing Discipline Specification• Queueing discipline is typically specified using Kendall’s notation (A/S/m/B/K/SD), where

– Letters correspond to six queue attributes• A: interarrival time distribution• S: service time distribution• m: number of servers• B: number of buffers (system capacity)• K: population size• SD: service discipline

• Interarrival and service time specifiers– M exponential– Ek Erlang with parameter k– Hk hyperexponential with parameter k– D deterministic– G general (any distribution, mean and variance used in the solution)

• Bulk arrivals or service are denoted by a superscript– M[x] denotes exponential arrivals with group size x– x is generally a random variable with separately specified distribution

• Omitted specifiers assume certain defaults– infinite buffer capacity– infinite population size– FCFS service discipline


Example Queueing Discipline Specifications

• M/D/5/40/200/FCFS– Exponentially distributed interarrival times– Deterministic service times– Five servers– Forty buffers (35 for waiting)– Total population of 200 customers– First-come-first-serve service discipline

• M/M/1– Exponentially distributed interarrival times– Exponentially distributed service times– One server– Infinite number of buffers– Infinite population size– First-come-first-serve service discipline


An Introductory Example• Given these descriptions, what are examples of their application?• Consider a typical bank

– 5 tellers– Customers form a single line and are serviced FCFS– Excluding a run on the bank, the waiting room is effectively infinite– For a large bank, the population is effectively infinite– Bulk arrivals are possible if friends arrive together for service

• What about service time and inter-arrival time distributions?– We can go measure them with a watch at the bank– Or, we can make mathematically simplifying assumptions– Latter is most common and exponential distribution is typical

• Combining these facts and assumptions– M/M/1 queue– As we shall see, the mean queue length (including one in service) for an M/M/1 queue is

– Where• λ is the mean inter-arrival time• µ is the mean service time

!µ!"


Notation and Basic “Facts”• Standard variable names

– τ is job interarrival time– λ = 1/E[τ] mean job arrival rate– s is service time per customer (job)– m is number of servers– µ = 1/E[s] is mean service rate per server– n = nq + ns is number of jobs in the system– nq is number of jobs waiting to receive service– ns is number of jobs in service– r is response time (service time plus queueing delay)– w is waiting time (queueing delay only)

• System must be “stable” to have an interesting steady state solution– Number of jobs in the system is finite– Requires the relation λ < mµ hold unless

• the population is finite (queue length is bounded)• the buffer capacity is finite (arrivals are lost when queue is full)• (in these cases, system is always stable)


Notation and Basic “Facts”• Number of jobs in the system

– n = nq + ns (jobs are either waiting or in service)–– and, if the service rates are independent of queue length

• Cov(nq,ns) = 0• Var[n] = Var[nq] + Var[ns]

• Number and time– r = w + s (response time is the sum of queueing delay and service)–– and, if the service rates are independent of queue length

• Cov(w,s) = 0• Var[r] = Var[w] + Var[s]

E[n] = E[nq ] + E[ns] (or n = n q + n s)

swrswr += so , variablesrandom are and , , but,


Little’s Law

• Very important result -- Part of the queueing folk literature for the past century• Formal proof due to J. D. C. Little (1961)• Relates mean queue length to arrival rate and mean response time• Mathematically (in seady state),

• Applies to any “black box” queue under the following assumptions– System is work conserving– Number of jobs entering is same as number leaving (system is stable)

• Also applies to any transient interval, without requirement that system be stable.• Note that these are very general conditions, and can apply for any system

(“black box”) in which customers leave and enter subject to the aboveconstraints.

• An intuitive proof...

n = !r


Little’s Law (Continued)• Sketch of proof (of steady-state case):

– During a long interval, arrivals ≈ departures (else no stability)– Area under the curve is total job time units– Mean queue length is average curve height (area/time)– Mean time in system is area/arrivals– Mean arrival rate is arrivals/time

• A very general result:– No assumptions about arrival or service processes– Holds for any queueing discipline (simply charge the area differently)

n r jobs x time jobs x time

time jobs

Avg number in system Avg time in system

jobs

timex

arrivalrate

=

(jobs x time)


Analysis of Single Queues• Plan:

– Start with one of the simplest queues, an M/M/1– Model as a “birth-death” process– Generalize result to other types of queues

• A birth-death process is a Markov process in which states are numbered a integers, andtransitions are only permitted between “neighboring” states.

• Steady state solution of a birth death process (Kleinrock, Queueing Systems, vol. 1):– (Theorem) steady state probability pn of being in state n is

– where p0 is the probability of being in state 0• Now for a proof ...

pn =!0!1 ...! n"1

µ1µ2 ...µnp0 n = 1, 2, ..., #


Birth-Death (Steady-State) State Occupancy Proof

!="=

=

•

=

=#$$%

&''(

) +=

++#=•

#

= +

#

#+

#

++

++##

,...,2,1

......

is...solution theAnd

and

,...3,2,1

or

)(0 earlier) describedsolution process Markov(by statesteady in the stable, If

1

0 10

021

110

01

01

11

1

11

1111

np

pp

pp

jppp

ppp

n

j j

j

n

nn

jj

jj

j

jjj

jjjjjjj

µ*µµµ***

µ*

µ*

µ*µ

µ*µ* Flow balance at state j


Birth-Death (Steady-State) State Occupancy Proof, cont.

! "+=

=!

•

#=

$=

+

#

=

110

1

0

0

1

1

have we

1

because Finally,

nnj

j

j

jj

p

p

µ%


M/M/1 Queue Analysis

jiji

/M/M

j

j

and allfor -

and allfor - processdeath -birth a of case special a is 1

i

i

µµ!!

=

=•

( ) !="=

•

"=++++

=

=

=

•

!=##$

%&&'

(=

•

!

,...,,np

...p

pp

,...,,npp

nn

nn

n

n

210 1

onsubstitutiBy

11

1

and

and intensity" traffic" thecalled is

ratio theon,By traditi

21

tionsimplificaBy

20

0

0

))

))))

)

µ*)

µ*


M/M/1 Queue Analysis (Continued)

[ ] ( )

( ) [ ]( )

( )

( )!!

!

!

!

"

=

"

=

"

=

"

=

"

=

=#=

•#

=

#$$%&

''() #=

#=

•#

=

#=

=

•=#•

nj

nj

njj

n

n

n

n

nn

p

n

nEn

]n[EnE]n[Var

)(n

npn

n]n[EpU

***

**

**

**

**

*

1

system in the jobs moreor ofy Probabilit 1

1

system in the jobs ofnumber of Variance 1

1

)(or length queueMean 1simply is on Utilizati

2

2

1

2

22

1

1

0

“almost” mean of a geometric random variable---factor out a rho first


M/M/1 Queue Analysis (Continued)

( )

[ ]

!!

µ""µ"!

!"

"

!µ

#=$ #=

•#=

•%&

#=

#==

=•

&

=

##

1)1(

)(or queue in the jobs ofnumber Mean 1)(

...)homework as (show is timeresponse of CDF as approaches timeresponse the where

11

yields

Law sLittle' via)(or timeresponseMean

2

1

)1(

nnq

qq

r

pnn

nnEerF

nr

rnRr


M/M/1 Queue Example

0520

11

timeresponsemean -

514060

1

system in the jobs ofnumber mean -

605030

n utilizatio - statistics following thecalculatecan We

0.5 = - 0.3 = -

queue following heConsider t

..

r

r

...n

n

...U

U

==!

=

==!

=

====

•

•

"µ

##

µ"#

µ"


M/M/1 Queue Example (Continued)• Consider changing λ

– hold µ fixed at 0.5– examine changes in performance metrics


M/M/m Queues• M/M/m queues

– m servers rather than one server– Reasonable model of

• a bank queue with multiple tellers• a shared memory multiprocessor

• Assumptions– m servers– All servers have the same service rate µ– Single queue for access to the servers– Arrival rate λ– Formally

• What are the state occupancy probabilities?

!n = ! n = 0,1,..., "

µn =nµmµ

# $ %

n = 0,1,...m !1n = m,m+1,..."


M/M/m Queues (Continued)

!!"

!!#

$

=

•

%==

•

&

&

0

0

021

110

!

!

have we, and theofon substituti simpleBy

,...,2,1 ...

...

earlier soccupanciey probabilit theof form general Recall - processdeath -birthanother Just - iesprobabilitoccupancy State

pmm

pnp

npp

nmn

n

n

n

n

jj

n

nn

µ'µ'

µ'µµµ'''

1,...,2,1 != mn

!+= ,...,1,mmn



or equivalently (with ! = " /(mµ))

pn =

m!( )nn!

p0

!nmm

m!p0

#

$ % %

& % %

• And, because

pnn= 0

'

( = 1

we have

p0 = 1+ (m!)m

m!(1) !)+ (m!)n

n!n=1

m)1

(*

+ ,

-

. / )1

!+= ,...,1,mmn

1,...,2,1 != mn



1

system in the jobs ofnumber Mean yourself sderivation thesedo shouldYou -

etc.) time,response n,utilizatio length, (queue measures standard"" thederivecan We-

queue, M/M/1 for the that manner tosimilar aIn

!!"!#

+=

+=•

•

mn

nnn sq



( ) ( )( )

!

"!

!"

µ#!

mmpnpn

n

pmmpjobsmP

m

mnn

m

nns

s

m

mnn

=$+$=

•

%=$=&=

=

'

=

%

=

'

=

1

1

0

servicein jobs ofnumber Expected formula C sErlang' asknown also -

queuemust job arrivingan y probabilit theis that observe

1!

where



!!"

#$$%

&'

=

•'

=

'=

=

•

!!"

#$$%

&'

+=

=

+=•

•

qw,r

qr)(m

nn

nw

w)(m

nr

swr

mm

q

q

s

q

100100

ln0max

time) waitingof percentile ( 1

again) law s(Little' timengMean waiti

11

1

law) sLittle'apply (just timeresponseMean bemust n utilizatioserver individual -

servicein jobsmean - servers -

servereach ofon Utilizati

((

)µ)

*

*

)(

µ

*

))


M/M/m Queue Example

• Consider changing m– hold λ and µ fixed– examine changes in performance metrics

• Observations– M/M/m queue has asymptote at– substantial performance gains with even two servers

!mµ


M/M/1 and M/M/m Queue Comparison

• Which is better? - m queues each with an arrival rate !/m - one queue with m servers and an arrival rate of !• Suppose we use mean response time as our metric... - m M / M /1 queues

r = 1µ " ! /m

- one M / M /m queue

r = 1µ

1+ #m 1" $( )

%

& '

(

) *

where

# =! /µ( )m

m! 1" ! / mµ( )( ) p0

and

p0 = 1+! /µ( )m

m! 1" ! / mµ( )( ) +! /µ( )n

n!n=1

m"1

+,

-

.

.

/

0

1 1

"1


Queueing Comparison

• Consider the following– service rate µ fixed at 4, divided evenly among m servers– fix λ =2– m M/M/1 queues (arrival rate to each is λ/m)– One M/M/m queue (total arrival rate is λ)– Increase m

• What happens to response time in both queues? Why?


Mean Response Time as function of m


Queueing Comparison

• Consider the following– service rate µ fixed at 2.1, divided evenly among m servers– varying λ (subject to stability constraint)– m M/M/1 queues (arrival rate to each is λ/m)– One M/M/m queue (total arrival rate is λ)

• What happens as λ approaches 2.1? Why?


Mean Response Time as a Function of Arrival Rate


Extrapolation Scenarios• Given queueing formulae, standard questions include

– Performance measures for different parameters– Parameters values needed to satisfy a particular performance constraint

• Examples:– What is the mean response time if arrival rate doubles?– What is the mean queue length if service rate decreases by one third?– What is the number of servers for mean response time less than five

minutes?• Approach:

– Plug and crank– Repeated solution with different parameter values


Extrapolation Scenarios (Continued)

( )

constraint thissatisfies 3 here, - 30such that of aluesmallest v find -

43for solve - minutes? 30 have toneed wedo processorsmany How

seconds) (24.6 minutes 4101

11

timeresponsemean have wecrank, and plugby and ejobs/minut 5.0 is rate arrivalmean -

processor)(per ejobs/minut 4.0 is rate servicemean - inspectionBy

seconds 12 is timealinterarriv jobmean - seconds 15 is timeservice jobmean -

)processors (two systemssor multiproce - example Concrete

=<

=<•

=!"#

$%&

'+=

•

•

m.rm

,...,m.r

.m

r

r

()

µ


M/M/m/B Queues• Finite buffers

– no more than B jobs in total can be• queued• and in service

(i.e., total number of jobs in the system must be less than B)– jobs arriving when B jobs are present are discarded

• More formally, this implies

and

• Observations– or servers are wasted– birth-death process– finite number of states

!n = ! n = 1,2,..., B "1

µn =nµmµ

! " #

n = 1,2,..., m !1n = m,m+1,..., B

B ! m


M/M/m/B Queues (Continued)

( )( ) ( )

rates arrival effective - lengths queuemean -

timeresponsemean - compute toiesprobabilitoccupancy state theusecan weNow,

11

1

is system in the jobs zero ofy probabilit theFinally,

because And,

formulaoccupancy state theApplying

11

1

1

00

0

0

0

0

•

!"

#$%

&+

''+==

•

(()

((*

+

=

=•

(()

((*

+

=

•

''

=

+'

=

'

,,m

n

nmmBB

nn

mn

n

n

nmn

n

n

n

n

!nm

)(!mmpp

p!mm

p!n)m(

p

m

pm!m

p!np

--

--

-

-

µ.-

µ.µ. n = 1,2,..., m !1

n = m,m+1,..., B

n = 1,2,..., m !1

n = m,m+1,..., B


M/M/m/B Queues (Continued)

( )

)p(m

~U

U

)p(n

~nr

~

pp~

~

p)mn(n

npn

n

B

B

B

B

nn

B

mnnq

B

nn

!==

!==

!==

•

!=

=

•

"

"

"

!

=

+=

=

1

isserver each of n utilizatio theFinally, - law sLittle'by

1

is timeresponsemean theentry,after lost not are jobs Because - rate loss theis - difference theand

1

available are buffersonly when system enter the jobs - than less is rate arrival effective -

space by waiting dconstraine are Arrivals

queue in thenumber mean and

service) plus (queue length queueMean

1

0

1

1

#µ$

$$

$$

$$$

$$


Other Queues• Other queues can be solved to varying degrees...• Exact solutions are possible for

– M/Er /1 (Erlangian service)– M/D/1 (special case of M/G/1)– M/M/1 with bulk arrivals (restricted cases)

• Analysis is more difficulty for:– G/M/1– M/G/1– G/G/1


M/G/1 Queues• M/G/1

– General service time distribution– Otherwise, similar to M/M/1 queues– The most complex, readily solvable single queue

• Solution approach– First, some additional mathematical machinery– Then, comparisons with M/M/1 queues

• Service time distribution is general– Service history matters– Denote service time already received by X0(t)

• Arrival distribution is negative exponential– Arrival history does not matter– But we do need to know the number of customers N(t) present– N(t) is non-Markovian because it depends on service time

• State-space description– States are [N(t), X0(t)]– Mixed discrete/continuous, two-dimensional description– Analysis via this method (supplementary variables) is ugly– Use the method of embedded Markov chains...


M/G/1 Queues (Continued)• What has changed from M/M/1?

– Two-dimensional state space– State space is now continuous (due to X0(t))

• Ideally– Convert [N(t), X0(t)] to one-dimensional N(t)– Implicitly specify remaining service duration X0(t)

• How do we do this?– Look only at selected points in time– Compute new metrics only at those points– Choose those points to implicitly carry X0(t)– departures instants make great choices

• Remaining (residual) service X0(t) is zero!• At that instant, we can treat the behavior like a Markov chain• N(t) is the number of customers left behind• This is an embedded Markov chain; for details (see Kleinrock, vol. 1) but we

haven’t specified the distribution of departure instants


M/G/1 Queues (Continued)

• A informal derivation follows (see Kleinrock vol. 1 for details)...• Notation

– Arrival rate λ (Poisson process)– General service time distribution

• mean• variance

• What is the expected time until a customer that arrives completes service?– Mean time needed to service customers already waiting

• Mean time is• Note that this is independent of the distribution of x

– plus the residual time for customer in service ...• Residual life requires yet another aside...

x

n q x


Residual Life• What is a “renewal”?

– Informally, a point where random variables which describe a model arememoryless given current state, with respect to past state.

• Renewal example– Consider a queue with general service distribution, and Poisson arrival

process– Most time points are not renewal points, since remaining service time

depends on service time completed.– However, times at which service completes are renewal points, since

arrival process is Poisson.• Need to determine the residual lifetime of a customer in service:

– Denote this random variable as R– Distribution of R depends on

• Distribution of original variable A (the service time distribution) atits renewal point and some time expended after the renewal point


Residual Life (Continued)

( ) ( )

ff

r

notf

f

)x(fr

not

ds)s(a

)(a)A(P

)(a

tattr

R)t(rt

A)t(a

et

e

eee

e

2

is lifetime residualmean - ) variance! the( moment second -

mean -

pdf original theof moments first two on theonly depends - proof) without (claim lifetime residual Average

domain discrete in the caseonly theison distributi geometric the- ondistributi lexponentia for the true was that thissaw we-

pdf thechanges knowledge short,in - helps timeexpended about the knowing general,in -

Intuition

1

is lifetime) residual (the of pdf the, then timeexpended has lifetime original the-

variable)(original of pdf theis - Suppose

2

2

0

=

•

•

!=

>=

>=!

•

"##

####


Residual Life (Continued)

67610233133

2

thatnotice -

otherwise 0150

5

finally and

7505201

11

and

otherwise 020

thenunits, time5for usein been haspart theif -

otherwise 0200

10 is mean value thesuppose and - uniform is timefailure theof pdf thesuppose -

part)(computer Example

2

151

5

0

201

201

..ffr

t)t(r

.ds)s(b

ttt)tt(b

)t(b

)t(b

ee

eee

=!

==

"#$ %+<

=&

=!&=&

"#$ %+<

=+

"#$ %<

=

•

'

((

(

Observe– pdf of residual time is not the same as the

original pdf– Knowledge of past behavior changes the pdf– There are only two exceptions

• negative exponential distribution(continuous)

• geometric distribution (discrete)



)(x

r

xx

xrr

,rxx

xnr

xx

t

plusx

xnx

q

qq

qq

q

!"

!"

"

!

!

#=

+=

•

+=

•$

=

$

$

$$

•

12

termsgrearranginby and 2

so is interval thisduring arrivals ofnumber Expected 2

is arrival new afor time waiting theitems, Combining is servicein customer a ofy probabilit the

servicein iscustomer a assuming 2

is that thisrecall servicein customer for timeresidual the -

ofon distributi theoft independen is that thisnote

is mean time timeservicemean thedenote let

itingalready wa customers service toneeded mean time - service?for wait tohave arrival new a does long How

2

2

2

2

Little’s Law again!



222

2

2

22

2

22

2

2

22

2

2

2

2

2

2

2

111 :n)formulatio original (yieldstion simplificaby and -

timeservicemean theis 1

-

timeservicemean theof variance theis -

and

variationoft coefficien theis where

121

and 12

11

math) e(verify th as expressed are and both Normally, 12

timeresponsemean theyields timeservicemean theAdding 12

is is service receive tomean time thesaw,just weAs

xxx

xxx

xC

xC

C

)()C(

r

)()C(

r

rr)(

xxr

)(x

r

s

s

s

sq

s

q

q

µ!µ

!

!

"µ#

"µ#

µ

"#

"#

==$+=+=+

=

$+

=

$+

+=

•$

+=

•$

=

•



( )

( )

s

s

s

s

s

s

s

CC

C)()(

)(n

/D/MzeroC

)()(

n

oneC

)(C

)(C

rn

ith linearly w grow values- timeresponse andlength queuemean increases larger -

nsimplicatio profound has of valueThe 2

2112

01

queue) 1(on distributi ticdeterminis for the is - before knew weas

111211

so on,distributi lexponentia negative for the is - it! treasureit,remember it,learn -

formula (PK)Khinchin -Pollaczek famous theis this- nsObservatio

121

121

is system in thenumber mean thelaw, sLittle' Via

2

22

22

2

22

•

!"#$

%& '

'=

'++=

'=

'+=

'++=

•'+

+=

'+

+=

=•

((

((

((

((

(((

(((

((

(

(µ)

µ)

)


Queueing Comparison• Consider the following

– M/D/1 queue (Cs = 0)– M/M/1 queue (Cs = 1)– M/G/1 queue (Cs > 1)


Queueing Example• Consider the following

– arrival rate λ = 0.6– service rate µ = 1.0– M/D/1, M/M/1, and M/G/1 queues

and compare mean response times• M/M/1

• M/D/1

• M/G/1 (Cs = 2.0)

r = 1µ ! "

=1

1.0 ! 0.6= 2.5

r = 1µ+! 1+Cs

2( )2µ 2 1" #( )

=11.0

+0.6(1+ 0)

2(1.0)(1" 0.6 /1.0)= 1.75

r = 1µ+! 1+Cs

2( )2µ 2 (1" #)

=11.0

+0.6(1+1)

2(1.0)(1" 0.6 /1.0)= 3.25


Queueing Example (Continued)• Consider M/M/1 and M/G/1 queues

– assume same arrival rates for both– desire same mean response times– must solve for ratio of service rates

• M/M/1

• M/G/1

• Equating, we have

• Let’s look at some numerical solutions...

r = 1µm ! "

r = 1µg

+! 1+Cs

2( )2µg

2 1" ! / µg( )

1µm ! "

=1µg

+" 1+Cs

2( )2µg

2 1! " / µg( )


M/G/1 via Embedded DTMC


Queueing Example (Continued)• Comparison Example (Continued)

– arrival rate λ = 0.6– M/M/1 queue (service rate µm = 1.0)– M/G/1 queue (service rate µg)

module 7: introduction to queueing theory (notation...

Documents