flows and networks plan for today (lecture 2): questions? birth-death process example: pure birth...
TRANSCRIPT
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Last time on Flows and Networks:Highlights: continuous time Markov chain
• stochastic process X(t) countable or finite state space S
Markov property
transition rates
independent t
irreducible: each state in S reachable from any other state in S
Assume ergodic and regular global balance equations (equilibrium
eqns)
π is stationary distribution
solution that can be normalised is equilibrium distributionif equilibrium distribution exists, then it is unique and is limiting distribution
)],()(),()([0 kjqjjkqkjk
)())0(|)((lim kjXktXPt
))(|)((
))(...,)(,)(|)(( 11
itXjstXP
jtXjtXitXjstXP nn
jih
jiPjiq h
h
),(lim),(
0
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Birth-death process• State space
• Markov chain, transition rates
• Bounded state space:
q(J,J+1)=0 then states space bounded above at J
q(I,I-1)=0 then state space bounded below at I
• Kolmogorov forward equations
• Global balance equations
otherwise
jkjj
ratedeathjkj
ratebirthjkj
kjq
0
)()(
1)(
1)(
),(
ZS
)1()1()]()()[()1()1(0
)1()1,()]()()[,()1()1,(),(
jjjjjjj
jjiPjjjiPjjiPdt
jidPttt
t
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Example: pure birth process• Exponential interarrival times, mean 1/
• Arrival process is Poisson process• Markov chain? • Transition rates : let t0<t1<…<tn<t
• Kolmogorov forward equations for P(X(0)=0)=1
• Solution for P(X(0)=0)=1
jk
jkkjq
hohjtXjhtXP
hojtXjhtXP
hohjtXjhtXP
jntnXjtXjtXjhtXP
1),(
)(1))(|)((
)())(|2)((
)())(|1)((
))(,...,0)0(,)(|1)((
),0()0,0(
),0()1,0(),0(
jPdt
dP
jPjPdt
jdP
tt
ttt
0,...,2,1,0,!
)(),0( tje
j
tjP t
j
t
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Example: pure death process• Exponential holding times, mean 1/
• P(X(0)=N)=1, S={0,1,…,N}
• Markov chain? • Transition rates : let t0<t1<…<tn<t
• Kolmogorov forward equations for P(X(0)=N)=1
• Solution for P(X(0)=N)=1
jkj
jkjkjq
hohjjtXjhtXP
hojtXjhtXP
hohjjtXjhtXP
jntnXjtXjtXjhtXP
1),(
)(1))(|)((
)())(|21)((
)())(|1)((
))(,...,0)0(,)(|1)((
),0(),(
),0()1,0()1(),(
NPNdt
NNdP
jPjjPjdt
jNdP
tt
ttt
0,,...,2,1,0,1),(
tNjeej
NjNP
jNtjtt
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Simple queue• Poisson arrival proces rate , single server
exponential service times, mean 1/
• Assume initially empty:
P(X(0)=0)=1, S={0,1,2,…,}
• Markov chain? • Transition rates :
0,
0,][
0,1
1
),(
)(][1))(|)((
)())(|1)((
)())(|1)((
jjk
jjk
jjk
jk
kjq
hohhjtXjhtXP
hohjtXjhtXP
hohjtXjhtXP
Simple queue• Poisson arrival proces rate , single server
exponential service times, mean 1/
• Kolmogorov forward equations, j>0
• Global balance equations, j>0
0,
0,][
0,1
1
),(
jjk
jjk
jjk
jk
kjq
)1()0(0
)1(])[()1(0
)1,()0,()0,(
)1,(])[,()1,(),(
jjj
iPiPdt
idP
jiPjiPjiPdt
jidP
ttt
tttt
Simple queue (ctd)
j j+1
Equilibrium distribution: <
Stationary measure; summable eq. distrib.
Proof: Insert into global balance
Detailed balance!
j
jj
)/)(/1(
)/)(0()(
).1()1()1,()( jjqjjjqj
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Birth-death process• State space
• Markov chain, transition rates
• Definition: Detailed balance equations
• Theorem: A distribution that satisfied detailed balance is a stationary distribution
• Theorem: Assume that
then
is the equilibrium distrubution of the birth-death prcess X.
0,)0(
0,)()(
0,1)(
1)(
),(
jjk
jjkjj
ratedeathjjkj
ratebirthjkj
kjq
,...}2,1,0{ NS
).1()1()1,()( jjqjjjqj
1
1 )1,(
),1()0(
rrq
rrqj
rSj
Sjrrq
rrqj
j
r
,)1,(
),1()0()(
1
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Reversibility; stationarity• Stationary process: A stochastic process is
stationary if for all t1,…,tn,
• Theorem: If the initial distribution is a stationary distribution, then the process is stationary
• Reversible process: A stochastic process is reversible if for all t1,…,tn,
NOTE: labelling of states only gives suggestion of one dimensional state space; this is not required
))(),...,(),((~))(),...,(),(( 2121 nn tXtXtXtXtXtX
1)(
jSj
))(),...,(),((~))(),...,(),(( 2121 nn tXtXtXtXtXtX
Reversibility; stationarity• Lemma: A reversible process is stationary.
• Theorem: A stationary Markov chain is reversible if and only if there exists a collection of positive numbers π(j), jS, summing to unity that satisfy the detailed balance equations
When there exists such a collection π(j), jS, it is the equilibrium distribution
• Proof
Skjjkqkkjqj ,),,()(),()(
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Lemma 1.9 / Corollary 1.10:
If the transition rates of a reversible Markov process with
state space S and equilibrium distribution are
altered by changing q(j,k) to cq(j,k) for
where c>0 then the resulting Markov process is
reversible in equilibrium and has equilibrium distribution
where B is the normalizing constant.
If c=0 then the reversible Markov process
is truncated to A and the resulting Markov
process is reversible with equilibrium distribution
Truncation of reversible processes
Sjj ),(
10
ASkAj \,
ASjjBc
AjjB
\)(
)(
Ajk
j
Ak
)(
)(
A
S\A
Time reversed processX(t) reversible Markov process X(-t) also, butLemma 1.11: tijdshomogeneity not inherited for
non-stationary process
Theorem 1.12 : If X(t) is a stationary Markov process with transition rates q(j,k), and equilibrium distribution π(j), jS, then the reversed processX(-t) is a stationary Markov process with transition rates
and the same equilibrium distribution
Theorem 1.13: Kelly’s lemmaLet X(t) be a stationary Markov processwith transition rates q(j,k). If we can find a collection of numbers q’(j,k) such that q’(j)=q(j), jS, and a collection of positive numbers (j), jS, summing to unity, such that
then q’(j,k) are the transition rates of the time-reversed process, and (j), jS, is the equilibrium distribution of both processes.
)(
),()(),('
j
jkqkkjq
Skj ,
),(')(),()( jkqkkjqj Skj ,
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Kolmogorov’s criteria• Theorem 1.8:
A stationary Markov chain is reversible iff
for each finite sequence of states
Notice that
),(),()...,(),(
),(),()...,(),(
122311
113221
jjqjjqjjqjjq
jjqjjqjjqjjq
nnn
nnn
)0,(),(),()...,(),(
),(),()...,(),(),0()0()(
112231
132211
jqjjqjjqjjqjjq
jjqjjqjjqjjqjqj
nnn
nnn
Sjjj n ,...,, 21
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
equilibrium• Reversibility, stationarity• Truncation• Kolmogorov’s criteria• Summary / Next• Exercises
Summary / next:
• Birth-death process• Simple queue• Reversibility, stationarity• Truncation• Kolmogorov’s criteria
• Nextinput / output simple queuePoisson procesPASTAOutput simple queueTandem netwerk
Exercises[R+SN] 1.3.2, 1.3.3, 1.3.5, 1.5.1, 1.5.2, 1.5.5,
1.6.2, 1.6.3, 1.6.4