1

Queuing Delay and Queuing Delay and Queuing AnalysisQueuing Analysis

RECALL: Delays in Packet RECALL: Delays in Packet Switched (e.g. IP) NetworksSwitched (e.g. IP) Networks End-to-end delay (simplified) =End-to-end delay (simplified) =

– (d(dpropprop + d + dtranstrans + d + dqueuequeue + d + dprocproc)) … … on each linkon each link

Introduction2

BA Where:Where:

Propagation delay (dPropagation delay (dpropprop) = d/s (dependent on path)) = d/s (dependent on path) Transmission delay (dTransmission delay (dtranstrans) = L/R (dependent on path)) = L/R (dependent on path) Queuing delay (dQueuing delay (dqueuequeue) = (dependent on load)) = (dependent on load) Processing delay (dProcessing delay (dprocproc) = (minimal-insignificant/node)) = (minimal-insignificant/node) Number of links (Q) = (dependent on path)Number of links (Q) = (dependent on path)

Queuing Analysis3

Projected vs. Actual Response Projected vs. Actual Response TimeTime

Why??Why??

Queuing Analysis

R:R: link bandwidth link bandwidth (bps)(bps)

L:L: packet length (bits) packet length (bits) a: average packet a: average packet

arrival ratearrival ratetraffic intensity

= La/R

La/RLa/R ~ 0: avg. queueing delay small ~ 0: avg. queueing delay small La/R La/R -> 1: avg. queueing delay large-> 1: avg. queueing delay large La/R La/R > 1: more > 1: more ““workwork”” arriving arriving than can be serviced, average delay than can be serviced, average delay

infinite!infinite!

aver

age

que

uein

g de

lay

La/R ~ 0

Queueing delay (revisited)Queueing delay (revisited)

La/R -> 14

Queuing Analysis5

Introduction- MotivationIntroduction- Motivation Address how to analyze changes in Address how to analyze changes in

network workloads (i.e., a helpful network workloads (i.e., a helpful tooltool to use) to use)

Analysis of system (network) load Analysis of system (network) load and performance characteristicsand performance characteristics

– response timeresponse time– throughputthroughput

Performance tradeoffs are Performance tradeoffs are often not often not intuitiveintuitive

Queuing theory, although Queuing theory, although mathematically complex, often mathematically complex, often makes analysis very straightforwardmakes analysis very straightforward

Queuing Analysis6

Important NoteImportant Note Queuing theory is heavily dependent Queuing theory is heavily dependent

on basic probability theory (a pre-on basic probability theory (a pre-requisite for our graduate program)requisite for our graduate program)

If you need to refresh your If you need to refresh your knowledge in this area, please knowledge in this area, please review the Stallings textbook, review the Stallings textbook, Chapter 7: Overview of Probability Chapter 7: Overview of Probability and Stochastice Processesand Stochastice Processes..

I will not test you specifically on I will not test you specifically on probability theory, but will reference probability theory, but will reference it in coverage of the queuing topics it in coverage of the queuing topics addressed in this module. addressed in this module.

Queuing Analysis7

Single-Server Queuing SystemSingle-Server Queuing System

QueuingSystem

(Delay Box)

Items ArrivingItems Arriving(rate: (rate: )

(message, packet, cell)(message, packet, cell)

Items Lost Items Lost

Items DepartingItems Departing(rate: R)(rate: R)

Introduction8

Router output port functionsRouter output port functions

buffering/queuing buffering/queuing required when datagrams arrive from fabric faster than the required when datagrams arrive from fabric faster than the transmission ratetransmission rate

scheduling disciplinescheduling discipline chooses among queued datagrams for transmission chooses among queued datagrams for transmission sending discipline (servicing the queue) sending discipline (servicing the queue) on the output link as determined by link protocolon the output link as determined by link protocol

linetermination

link layer

protocol(send)

switchfabric

datagrambuffer(s)

queueing

Queue Queue server

Queuing Analysis9

The Fundamental Task of The Fundamental Task of Queuing AnalysisQueuing Analysis

Given:Given:• Arrival rate, Arrival rate, • Service time, Service time, TTss

• Number of servers, Number of servers, NN

Determine:Determine:• Items waiting, Items waiting, ww• Waiting time, Waiting time, TTww

• Items queued, Items queued, rr• Residence time, Residence time, TTrr

Queuing Analysis10

Parameters for Single-Server Parameters for Single-Server Queuing SystemQueuing System

Comments, assuming queue has infinite capacity:1. At = 1, server is working 100% of the time (saturated), so items are

queued (delayed) until they can be served. Departures remain constant (for same L).

2. Traffic intensity, u = L/R. Note that Ts = L/R, so:max = 1 / Ts = 1 / (L/R) is the theoretical maximum arrival rate,

and thatLmax/R = u = 1 at the theoretical maximum arrival rate

Queuing Analysis11

Queuing Process - ExampleQueuing Process - Example

General Expression:General Expression:TTRn+1Rn+1 = T = TSn+1Sn+1 + MAX[0, D + MAX[0, Dnn – A – An+1n+1]]

Depth of the Queue

Queuing Analysis12

General Characteristics of General Characteristics of Network Queuing ModelsNetwork Queuing Models

Item populationItem population– generally assumed to be generally assumed to be infiniteinfinite therefore, therefore,

arrival rate is persistent through timearrival rate is persistent through time Queue sizeQueue size

– infiniteinfinite, therefore no loss, therefore no loss– finite, more practical, but often immaterialfinite, more practical, but often immaterial

Dispatching discipline Dispatching discipline – FIFOFIFO, typical, typical– LIFO (when is this practical?)LIFO (when is this practical?)– Relative/Preferential, Relative/Preferential, based on QoSbased on QoS

Queuing Analysis13

Multiserver Queuing SystemMultiserver Queuing System

Comments:1. Assuming N identical servers, and is the utilization of each server. 2. Then, N is the utilization of the entire system, and the maximum

utilization is N x 100%.3. Therefore, the maximum supportable arrival rate that the system can

handle is: max = N / Ts = NR/L

Chapter 8 Overview of Queuing Analysis14

Multiple Single-Server Queuing Multiple Single-Server Queuing SystemsSystems

Queuing Analysis15

Basic Queuing RelationshipsBasic Queuing Relationships

GeneralGeneral Single Single ServerServer MultiserverMultiserver

rr = = TTrr Little’s Little’s FormulaFormula = = TTss

= =

ww = = TTww Little’s Little’s FormulaFormula rr = = ww + + uu = = TTss = = NN

TTrr = = TTww + + TTss r = w + Nr = w + N

TTs s

NN

Queuing Analysis16

Kendall’s notationKendall’s notation Notation is Notation is X/Y/NX/Y/N, where:, where:

X is distribution of interarrival X is distribution of interarrival timestimes

Y is distribution of service timesY is distribution of service timesN is the number of serversN is the number of servers

Common distributionsCommon distributions G = general distribution if interarrival times G = general distribution if interarrival times

or service timesor service times GI = general distribution of interarrival time GI = general distribution of interarrival time

with the restriction that they are independentwith the restriction that they are independent M = negative exponential distribution of M = negative exponential distribution of

interarrival times (Poisson arrivals – p. 167) interarrival times (Poisson arrivals – p. 167) and service timesand service times

D = deterministic arrivals or fixed length D = deterministic arrivals or fixed length serviceservice

M/M/1? M/D/1?M/M/1? M/D/1?

Queuing Analysis17

Important Formulas for Important Formulas for Single-Server Queuing Single-Server Queuing SystemsSystems

Note Coefficient of variation: if Ts = Ts => exponential if Ts = 0 => constant

Queuing Analysis18

Mean Number of Items in Mean Number of Items in System (System (rr)- Single-Server )- Single-Server QueuingQueuing

Ts/Ts = Coefficient of variation

M/M/1

M/D/1

Queuing Analysis19

Mean Residence Time – (Mean Residence Time – (TTrr) ) Single-Server QueuingSingle-Server Queuing

M/M/1

M/D/1

Queuing Analysis20

Network Queue Performance: Network Queue Performance: Key FactKey Fact

The higher the variability in arrival The higher the variability in arrival rate at the router, relative to the rate at the router, relative to the service time on the output link(s), i.e., service time on the output link(s), i.e., TTss/T/Tss , , the poorer the performance of the poorer the performance of the router, especially at high rates of the router, especially at high rates of utilization.utilization.

Queuing Analysis21

Multiple Server Queuing Multiple Server Queuing SystemsSystems

Multiple Multiple Single-Single-Server Server Queuing Queuing SystemSystem

Multiserver Multiserver Queuing Queuing SystemSystem

Queuing Analysis22

Important Formulas for Important Formulas for Multiserver QueuingMultiserver Queuing

Note:Note:Useful only inUseful only inM/M/N case,M/M/N case,with equal with equal service times service times at all N at all N servers.servers.

Queuing Analysis23

Multiple Server Queuing Multiple Server Queuing Example Example (p. 203)(p. 203)

Single serverM/M/1 (2nd Floor)

MultiserverM/M/? (2nd Floor)

Multiple Single server

M/M/1 (1st Floor)

M/M/1 (2nd Floor)

M/M/1 (3rd Floor)

Queuing Analysis24

MultiServer vs. Multiple Single-MultiServer vs. Multiple Single-Server Queuing System Server Queuing System Comparison Comparison (from example problem, pp. 203-(from example problem, pp. 203-204)204)

Single server case (M/M/1):Single server case (M/M/1):Single server utilization: Single server utilization: = 10 engineers x 0.5 hours each / 8 = 10 engineers x 0.5 hours each / 8 hour work dayhour work day

= 5/8 = .625= 5/8 = .625Average time waiting: TAverage time waiting: Tww = = TTss / 1 - / 1 - = 0.625 x 30 / .375 = 50 = 0.625 x 30 / .375 = 50 minutesminutesArrival rate: Arrival rate: = 10 engineers per 8 hours = 10/480 = 0.021 = 10 engineers per 8 hours = 10/480 = 0.021 engineers/minuteengineers/minute9090thth percentile waiting time: m percentile waiting time: mTTww(90) = T(90) = Tww// x ln(10 x ln(10) = 146.6 minutes) = 146.6 minutes

Average number of engineers waiting: w = Average number of engineers waiting: w = TTww = 0.021 x 50 = 1.0416 = 0.021 x 50 = 1.0416 engineersengineers

Queuing Analysis25

Example: Router QueuingExample: Router QueuingInternetInternet ……

96009600bpsbps

= 5 packets/sec= 5 packets/secL = 144 octetsL = 144 octets

From data provided:From data provided:• TTs s = L/R = (144x8)/9600 = .12sec= L/R = (144x8)/9600 = .12sec = = TTs s = 5 packets/sec x .12sec = = 5 packets/sec x .12sec =

.6.6

Determine:Determine:1.1. TTrr= T= Tss / (1- / (1-) = .12sec/.4 = .3 sec) = .12sec/.4 = .3 sec2.2. r = r = / (1- / (1-) = .6/.4 = 1.5 ) = .6/.4 = 1.5

packetspackets

3. m3. mrr(90) = - 1 = 3.5 (90) = - 1 = 3.5 packetspackets

4.4. mmrr(95) = - 1 = 4.8 (95) = - 1 = 4.8 packetspackets

ln(1-.90)ln(1-.90)ln (.6)ln (.6)

ln(1-.95)ln(1-.95)ln (.6)ln (.6)

For 3 & 4, use:For 3 & 4, use:

mmrr(y) = - (y) = - 1 1

ln(1 – ln(1 – y/100)y/100)ln ln

Queuing Analysis26

Priorities in Queues – Two Priorities in Queues – Two priority classespriority classes

r

Chapter 8 Overview of Queuing Analysis27

Priorities in Queues – Priorities in Queues – ExampleExample

Router queue services two packet Router queue services two packet sizes:sizes:• Long = 800 octetsLong = 800 octets• Short = 80 octetsShort = 80 octets• Lengths exponentially distributedLengths exponentially distributed• Arrival rates are equal, 8packets/secArrival rates are equal, 8packets/sec• Link transmission rate is 64KbpsLink transmission rate is 64Kbps• Short packets are priority 1,Short packets are priority 1,• Longer packets are priority 2Longer packets are priority 2From data above, calculate:From data above, calculate:TTs 1s 1 = L = Lshortshort/R = (80 x 8) / 64000 = .01 /R = (80 x 8) / 64000 = .01 secsecTTs 2s 2 = L = Llonglong/R = (800 x 8) / 64000 = .1 /R = (800 x 8) / 64000 = .1 secsec11 = = TTs 1 s 1 = 8 x 0.01 = 0.08= 8 x 0.01 = 0.0822 = = TTs 2 s 2 = 8 x 0.1 = 0.8= 8 x 0.1 = 0.8 = = 1 1 ++ 2 2 = 0.88= 0.88

Find the average Queuing Delay (Find the average Queuing Delay (TTrr) ) through the router:through the router:

TTr1r1 = T = Ts1 s1 + +

= .01 + = 0.098 = .01 + = 0.098 sec sec

TTr2r2 = T = Ts2 s2 + +

= .1 + = 0. 833 sec = .1 + = 0. 833 sec

TTrr = T = Tr1 r1 + + TTr2 r2

= .5 x .098 + .5 x .833 = 0.4655 = .5 x .098 + .5 x .833 = 0.4655 secsec

11 TTs 1 s 1 + + 2 2 TTs 2s 2

1 - 1 - 11.08 x .01 .08 x .01 + + .8 .8 x .1x .1

1-.081-.08TTr 1 r 1 -- TTs 1s 1

1 - 1 - .098.098 -- .01 .01

1 - .881 - .8811

22

64Kbps64Kbps

TTrr

Queuing Analysis28

Network of QueuesNetwork of Queues

Queuing Analysis29

Elements of Queuing NetworksElements of Queuing Networks

Queuing Analysis30

Queuing NetworksQueuing Networks

Queuing Analysis31

Jackson’s Theorem and Jackson’s Theorem and Queuing NetworksQueuing Networks Assumptions:Assumptions:

– the queuing network has m nodes, each providing the queuing network has m nodes, each providing exponential serviceexponential service

– items arriving from outside the system at any node items arriving from outside the system at any node arrive with a Poisson ratearrive with a Poisson rate

– once served at a node, an item moves immediately once served at a node, an item moves immediately to another with a fixed probability, or leaves the to another with a fixed probability, or leaves the networknetwork

Jackson’s Theorem states: Jackson’s Theorem states: – each node is an independent queuing system with each node is an independent queuing system with

Poisson inputs determined by partitioning, merging Poisson inputs determined by partitioning, merging and tandem queuing principlesand tandem queuing principles

– each node can be analyzed separately using the each node can be analyzed separately using the M/M/1 or M/M/N modelsM/M/1 or M/M/N models

– mean delays at each node can be added to mean delays at each node can be added to determine mean system (network) delaysdetermine mean system (network) delays

Queuing Analysis32

Jackson’s Theorem - Application Jackson’s Theorem - Application in Packet Switched Networksin Packet Switched Networks

Packet SwitchedPacket SwitchedNetworkNetwork

External load, offered to network:External load, offered to network: = = jkjk

where:where: = = total workload in total workload in packets/secpackets/sec jk jk = = workload between source j workload between source j and destination kand destination k N = total number of (external) N = total number of (external) sources and destinationssources and destinations

N NN N

j=1 j=1 k=2k=2

Internal load:Internal load:

= = iiwhere:where: = = total on all links in networktotal on all links in network i i = = load on link iload on link i L = total number of linksL = total number of links

L L

i=i=11

Note:Note:• Internal > offered loadInternal > offered load• Average length for all paths:Average length for all paths: E[number of links in path] = E[number of links in path] = //• Average number of item waiting Average number of item waiting and being served in link i: rand being served in link i: r ii = = i i TTriri• Average delay of packets sent Average delay of packets sent through the network is:through the network is: T = T =

where: M is average packet length where: M is average packet length andand RRi i is the data rate on link iis the data rate on link i

11

L L i=i=11

MMiiRRii - - MMii

Queuing Analysis33

Estimating Model Estimating Model ParametersParametersTo enable queuing analysis using To enable queuing analysis using

these models, we must these models, we must estimate estimate certain parameterscertain parameters::– Mean and standard deviation of Mean and standard deviation of

arrival ratearrival rate– Mean and standard deviation of Mean and standard deviation of

service timeservice time (or, packet size) (or, packet size)Typically, these estimates use Typically, these estimates use

sample measurementssample measurements taken from taken from an existing systeman existing system

Queuing Analysis34

Sample Means for Exponential Sample Means for Exponential DistributionDistribution Sampling:

• The mean is generally the most important quantity to estimate:

() = Xi

• Sample mean is itself a random variable

• Central Limit Theorem: the probability distribution tends to normal as sample size, N, increases for virtually all underlying distributions

• The mean and variance of X can be calculated as:

E[]= E[X] = Var[]= 2x/N

N

i = 1

1N