Download - ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: [email protected] Exercises: 6
![Page 2: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/2.jpg)
Question 1Consider the MC with transition probability matrix
Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and the mean time that the process spends in state 2 prior to absorption. Verify that the sum of these is the mean time to the absorption.
![Page 3: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/3.jpg)
Question 1Denote by wi1 the mean time the process spends in
state 1 starting in state i prior to the absorption. We have,
w01 = 0w11 = 1 + 0.1w01 + 0.2w11 + 0.5w21 + 0.2w31
w21 = 0 + 0.1w01 + 0.2w11 + 0.6w21 + 0.1w31
w31 = 0
The solution isw01 = 0, w11 = 1.8182, w21 = 0.9091, w31 = 0.
![Page 4: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/4.jpg)
Question 1Similarly, denote by wi2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have,
w02 = 0w12 = 0 + 0.1w02 + 0.2w12 + 0.5w22 + 0.2w32
w22 = 1 + 0.1w02 + 0.2w12 + 0.6w22 + 0.1w32
w32 = 0
The solution isw02 = 0, w12 = 2.2727, w22 = 3.6364, w32 = 0.
![Page 5: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/5.jpg)
Question 1Denote by vi the mean time to the absorption
starting in state i prior to the absorption. We have,
v0 = 0v1 = 1 + 0.1v0 + 0.2v1 + 0.5v2 + 0.2v3
v2 = 1 + 0.1v0 + 0.2v1 + 0.6v2 + 0.1v3
v3 = 0
The solution is v0 = 0, v1 = 4.0909, v2 = 4.5455, v3 = 0.We have verified that,
v1 = w11 + w12
![Page 6: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/6.jpg)
Question 2Consider the MC with transition probability matrix
Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and the mean time that the process spends in state 2 prior to absorption. Verify that the sum of these is the mean time to the absorption.
![Page 7: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/7.jpg)
Question 2Denote by wi1 the mean time the process spends in state 1 starting in state i prior to the absorption. We have,
w01 = 0w11 = 1 + 0.5w01 + 0.2w11 + 0.1w21 + 0.2w31
w21 = 0 + 0.2w01 + 0.1w11 + 0.6w21 + 0.1w31
w31 = 0
The solution is,w01 = 0, w11 = 1.290, w21 = 0.3225, w31 = 0.
![Page 8: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/8.jpg)
Question 2Similarly, denote by wi2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have,
w02 = 0w12 = 0 + 0.5w02 + 0.2w12 + 0.1w22 + 0.2w32
w22 = 1 + 0.2w02 + 0.1w12 + 0.6w22 + 0.1w32
w32 = 0
The solution isw02 = 0, w12 = 0.3230, w22 = 2.5808, w32 = 0.
![Page 9: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/9.jpg)
Question 2Denote by vi the mean time to the absorption starting in state i prior to the absorption. We have,
v0 = 0v1 = 1 + 0.5v0 + 0.2v1 + 0.1v2 + 0.2v3
v2 = 1 + 0.2v0 + 0.1v1 + 0.6v2 + 0.1v3
v3 = 0
The solution is v0 = 0, v1 = 1.613, v2 = 2.9033, v3 = 0.We have verified that,
v1 = w11 + w12
![Page 10: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/10.jpg)
Question 3
Consider the MC in question 1. Starting in state 1, determine the probability that the process is absorbed into state 0. Compare this with the (1,0)th entry in the matrix powers P2, P4, P8 and P16.
![Page 11: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/11.jpg)
Question 3
Denote by ui the probability that the MC is absorbed by 0 starting in state i. We have,
u0 = 1u1 = 0.1u0 + 0.2u1 + 0.5u2 + 0.2u3
u2 = 0.1u0 + 0.2u1 + 0.6u2 + 0.1u3
u3 = 0
The solution is,u0 = 1, u1 = 0.4091, u2 = 0.4545, u3 = 0.
![Page 12: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/12.jpg)
Compare:u1 = 0.4091
![Page 13: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/13.jpg)
Consider a (4 x 4) transition probability matrix,
By definition,
1
01 10t
t XXPu
Question 3
10
...
...
...
...
....
....
....
;
...
...
...
023013201210111000210
30
20
10
00
131211102
30
20
13121110
00
XXPppppppppp
p
p
p
p
pppp
p
p
pppp
p
P
P
![Page 14: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/14.jpg)
But for our case, p00 = 1, p03 = 0. Question 3
2
10
2012101110
3013201210111000210
10t
t XXP
ppppp
ppppppppp
1 21 p11 p22
p10
p20
30 1
p21
p12
p13
p23
![Page 15: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/15.jpg)
Let:
1.F(t) be the set of t-step first passage paths from state 1 to state 0
2.G(n-t) be the set of (n-t)-step paths from state 0 to state 0
3.H(t) be the set of paths that is formed jointly by F(t) followed by G(n-t). Note: paths are n-step.
Question 3-Optional
![Page 16: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/16.jpg)
Question 3-Optional
01 0G(n-t)F(t)
H(t)
![Page 17: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/17.jpg)
Let L(n) be the set of n-step paths from state 1
to state 0. s.t.
Question 3-Optional
n
ttn
n
t
n
t
n
t
n
t
n
t
n
t
n
t
XXPtf
tnGPtFPtnGPtFP
tnGtFPtHP
tHPtHPnLP
tHnL
10,1
11
11
11
1
00
,)(
)(
npnLP 10
![Page 18: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/18.jpg)
f1,0(t) is the t-step first passage probability from
state 1 to state 0.
If state 0 is an absorbing state,
Also, trivially,
Question 3-Optional
100 tn XXP
10 00,1 XXPtf t
n
tt
n
ttn
n
XXP
XXPtf
pnLP
10
10,1
10
10
00
![Page 19: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/19.jpg)
Question 3-Optional
10
101 10
p
XXPut
t
![Page 20: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/20.jpg)
Question 4
Which of the following MC is regular:
a)
b)
![Page 21: ST3236: Stochastic Process Tutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6](https://reader036.vdocuments.site/reader036/viewer/2022062305/5697c0041a28abf838cc429c/html5/thumbnails/21.jpg)
Question 4
a) YES, because
(all entries are greater than 0)
b) NO, because it has absorbing states