limiting probabilities

'

lim ( )

lim ( ) lim ( ) ( )

0

j ijt

ij kj ik j ijk jtt

kj k j jk j

kj k j j j j kj kk j k j

P P t

P t q P t v P t

q P v P

q P v P v P q P

Limiting probabilities

The limiting probabilities can be obtained by solving the following

system of equations:

0

1

kj k j jk j

jj

q P v P

P

rate at which the process leaves state

rate at which the process enters state

rate out of state rate into state

j j

kj kk j

j j kj kk j

v P j

q P j

v P q P

j j

The limiting probabilities Pj exist if

(a) all states of the Markov chain communicate (i.e., starting in state i, there is a positive probability of ever being in state j, for all i, j and

(b) the Markov is positive recurrent (i.e, starting in any state, the mean time to return to that state is finite).

When do the limiting probabilities exist?

The M/M/1 queue

10 2

3

0 1

1 2 0

State rate out of state rate into state

0

1 ( )

2

j j

P P

P P P

2 3 1

1 1

( )

1 ( ) n n n

P P P

n P P P

The M/M/1 queue

0 1

1 2

2 3

By adding to each equation the equation preceding it, we obtain

0

1

2

P P

P P

P P

n

11 n nP P

The M/M/1 queue

0

1 0

22 0

33 3

Solving in terms of yields

0 ( / )

1 ( / )

2 ( / )

1 ( /n

P

P P

P P

P P

n P

0)nP

The M/M/1 queue

0

0 0 1

0

1

Using the fact that 1,we obtain

( / ) 1

11

1 ( / )

/ (1 / )

Note that we must have / 1.

nn

n

n

n

n

n

n

P

P P

P

P

The expected number in the system

0 0

20

( ) / (1 / )

(use the fact )(1 )

n

nn n

n

n

E n nP n

xnx

x

The birth and death process

0 0 1 1

1 1 1 2 2 0 0


0

1 ( )

2

j j

P P

P P P

2 2 2 3 3 1 1

1 1 1 1

( )

1 ( )n n n n n n n

P P P

n P P P

10 2

3

0 0 1 1

1 1 2 2

2 2 3 3

By adding to each equation the equation preceding it, we obtain

0

1

2

P P

P P

P P

1 1 1 n n n nn P P

0

1 0 1 0

2 1 2 1 1 0 2 1 0

3 2 3 2 2 1 0 3 2 1 0

Solving in terms of yields

0 ( / )

1 ( / ) ( / )

2 ( / ) ( / )

P

P P

P P P

P P P

1 1 1 2 1 0 1 2 1 0

1 ( / ) ( ... / ... )n n n n n n n nn P P P

0

1 2 1 00 0 1

1 2 1

01 2 1 0

11 2 1

1 2 1 0

1 2 1 01 2 1

1 2 1

Using the fact that 1,we obtain

...1

...

1...

1...

...

...... 1

...

Note that

nn

n n

nn n

n nn

n n

n nn

n nn n n

n n

P

P P

P

P

1 2 1 0

11 2 1

... we must have .

...n n

nn n

011 2 1 0

1 11 2 1

The / / queue is a birth and death process with

if

if

1 1...

1 1... ! !

n

n

n nkn n

n n n k nn n kn n

M M k

n n k

k n k

P

n k k

The M/M/m queue

0

0

( / ),

!

( / )

!

The stability condition is / 1 .

n

n n

n k

P n knP

P n kk k

k

The M/M/m queue

020

The expected number of customers in a / / queue

( / )( ) / where / .

!(1 )

k

nn

M M k

E n nP P kk

The M/M/m queue

A machine repair model

A system with M machines and one repairman. The time between machine is exponentially distributed with mean 1/. Repair times are also exponentially distributed with mean 1/. What is the average number of working machines? What is the fraction of time each machine is in use?

The system is in state if there are failed machines

0,1,2,...,

n n

n M

The machine repairman problem


0,1,2,...,

The corresponding process is a birth and death process with

( ) if

0 if n

n n

n M

M n n M

n M

, 1n n



0,1,2,...,

The corresponding process is a birth and death process with

( ) if

0 if n

n n

n M

M n n M

n M

01 2 1 0

111 2 1

1

, 1

1 1... ( 1) ...( 1)

11...

1

1 ( / ) !/( )!

n

MM n nnnn

n n

M n

n

n

PM M M n

M M n


1 2 1 00

1 2 1

1

...

...

( / ) !/( )! = , 0,1,...,

1 ( / ) !/( )!

n nn

n n

n

M n

n

P P

M M nn M

M M n


1 2 1 00

1 2 1

1

1

1

1

...

...

( / ) !/( )! = , 0,1,...,

1 ( / ) !/( )!

( / ) !/( )!( )

1 ( / ) !/( )!

n nn

n n

n

M n

n

M nM n

n Mn n

n

P P

M M nn M

M M n

n M M nE n nP

M M n


0{machine is working} {machine is working | not working}

M

nnP P n P


0

0

0

{machine is working} {machine is working | not working}

=

1 1 ( ) /

M

nn

M

nn

M

nn

P P n P

M nP

M

nPE n M

M


The automated teller machine (ATM) problem

Customers arrive to an ATM according to a Poisson process with rate . If the customer finds more than N other customers at the machine, he/she does not wait and goes away. Machine transaction times are exponentially distributed with mean 1/. What is the probability that a customer goes away? What is the average number of customers at the ATM? If the machine earns $h per customer served, what is the average revenue the machine generates per unit time?

The M/M/1/N queue

0 0 1 1

1 1


0

1 -1 ( )

1 n n n

j j

P P

n N P P P

N

1 N NP P

The M/M/1/N queue

0

0 10

1

( / )

1 /1

1 ( / )

( / ) (1 / )

1 ( / )

nn

N

n Nn

n

n N

P P

P P

P

The M/M/1/N queue

0

0 10

1

10 0

1

1

( / )

1 /1

1 ( / )

( / ) (1 / )

1 ( / )

1 /( ) ( / )

1 ( / )

[1 ( / ) ( 1)( / ) ]

( )[1 ( / ) ]

nn

N

n Nn

n

n N

nn Nn n

N N

N

P P

P P

P

E n nP n

N N

The M/M/1/N queue

1

( / ) (1 / )The probability that a customer goes away is

1 ( / )

The average number of customers at the ATM is ( )

The average revenue per unit time (revenue rate) is (1 )

N

N N

N

P

E n

P h

The production inventory problem

Consider a production system that manufacturers a single product. Production times are exponentially distributed with mean 1/. The production facility can produce ahead of demand by holding finished goods inventory. Orders from customers arrives according to a Poisson process with rate . If there is inventory on-hand, the order is satisfied immediately. Otherwise, the order is backordered. The production system incurs a holding cost $h per unit of held inventory per unit time and a backorder cost $b per unit backordered per unit time. The production system manages its finished goods inventory using a base-stock policy with base-stock level s.

The production inventory problem

• What is the expected inventory level?

• What is expected backorder level?

• What is the expected total cost?

• What is the optimal base-stock level?

I: level of finished goods inventoryB: number of backorders (backorder level)IO: inventory on order.

Three basic processes

Under a base-stock policy, the arrival of each customer order triggers the placement of an order with the production system

s = I + IO – B

s = E[I] + E[IO] – E[B]


I and B cannot be positive at the same time

I = max(0, s - IO) = (s – IO)+

E[I] = E[(s – IO)+]

B = max(0, IO - s) = (IO - s)+

E[B] = E [(IO - s)+]


The production system behaves like an M/M/1 queue, with IO corresponding to the number of customers in the system.

Pr( ) (1 )

[ ]1

nIO n

E IO

0

0

1

[ ] [max(0, )]

( ) (1 )

(1 )

(1 )

1

n

n s

n s

n

s n

n

s

E B E IO s

n s

n

n

Expected backorder level

1

[ ] ( ) [ ]1 1

s

E I s E IO E B s

Expected inventory level

1 1

( ) : expected cost (holding cost + backorder cost)

( ) [ ] [ ] ( )1 1 1

s s

z s

z s hE I bE B h s b

Expected cost

Optimal base-stock level

2 2 1 1

2 1 2 1

1

1

1

( 1) ( ) ( 1 ) ( ) 01 1 1 1 1 1

(1 ) ( ) 01 1 1 1

(1 )( ) 01

( ) 0

s s s s

s s s s

s

s

s

r r r r r rz s z s h s b h s b

r r r r r r

r r r rh b

r r r r

rh r h b

r

h r h b

hr

h b

Optimal base-stock level

1

ln1

ln

ln ln* 1

ln ln

lnIf we ignore the integrality of *, then * .

ln

s hr

h bhh bsrh hh b h bsr r

hh bs sr

limiting probabilities

Documents