intro to markocchain chutes-and-ladders
TRANSCRIPT
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
1/38
Mehran University of Engineering and Technology, Jamshoro
-This class will meet @: 8:30 a.m -10:30 a.m (Mondays)
10:30 a.m - 12:30 p.m (Thursdays)10:30 a.m - 12:30 p.m (Wednesdays)10:00 a.m -11:30 a.m (Fridays)
Today's Lecture: Lecture # 35 Lec # 37
15-03-2012
Introduction to Markov ChainsPS: Some slides are taken from Dr. Deckelman site
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
2/38
Mr. Markov Plays Chutes and
Ladders
Introduction
The Concept of a Markov Chain
The Chutes and Ladders Transition Matrix
Simulation Techniques
Repeated Play
Conclusion
4/24/2012 Principles of Teletraffic Engineering 2
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
3/38
4/24/2012 Principles of Teletraffic Engineering 3
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
4/38
Introduction
4/24/2012 Principles of Teletraffic Engineering 4
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
5/38
How the Game is Played [1]Chutes and Ladders is a board game where players
spin a pointer to determine how they will advance
The board consists of 100 numbered squarese o ec ve s o o sq re 100
The spin of the pointer determines how manysquares the player will advance at his turn with
equal probability of advancing from 1 to 6 squaresHowever, the board is filled with chutes, which
move a player backward if landed on
4/24/2012 Principles of Teletraffic Engineering 5
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
6/38
How the Game is Played [2]There are also ladders, which advance a player
Chutes have pictures of bad behavior which leadsto disasters
Ladders have pictures of ood behavior leadin
4/24/2012 Principles of Teletraffic Engineering 6
to rewards
Most of the chutes and ladders produce relativelysmall changes in position, but several produce
large gains or losses
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
7/38
The Concept of aov
4/24/2012 Principles of Teletraffic Engineering 7
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
8/38
Topics CoveredTransition matrix
Probability vectors
Absorbing vs. non-absorbing Markov chainsSteady-state matrices
4/24/2012 Principles of Teletraffic Engineering 8
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
9/38
Markov ChainsA Markov Chain is a weighted digraph
representing a discrete-time system that can be
in any number of discrete statesThe transition matrix for a Markov chain is the
transpose of a matrix of probabilities of movingfrom one state to another
Pij = probability of moving from state i to j
4/24/2012 Principles of Teletraffic Engineering 9
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
10/38
Transition Matrix
Below, is the layout of the transition matrix for
Chutes and Ladders (101x101)
p0,0 p0,1 p0,100
p1,0 p1,1 p1,100
. . .
. . .
p100,0 p100, 1 . p100,100
4/24/2012 Principles of Teletraffic Engineering 10
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
11/38
Three properties which identify a state
model as being a Markov model:
1) The probability of moving from state i to j isindependent of what happened before moving to
state j and how one got to state i (Markovss o
2) Sum of probabilities for each state must be one
3) X(t) = probability distribution vector of the
probability of the system being in each of thestates at time n
4/24/2012 Principles of Teletraffic Engineering 11
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
12/38
Probability VectorThe probability vector is a column vector in which the
entries are nonnegative and add up to one
The entries can represent the probabilities of finding a
4/24/2012 Principles of Teletraffic Engineering 12
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
13/38
Two types of Markov Chains1) Absorbing Markov Chains
Can not get out of certain states
Once a system enters an absorbing state, the systeme s s e o e o
2) Non-absorbing Markov Chains
Can always get out of every state
4/24/2012 Principles of Teletraffic Engineering 13
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
14/38
Absorbing Markov Chains
(Chutes and Ladders)
Two conditions must be met for a MarkovChain to be absorbing:
Must have at least one state which cannot be leftonce as een en ere
It must be possible, through a series of one or moremoves, to reach at least one absorbing state fromevery non-absorbing state (given enough time, everysubject will eventually be trapped in an absorbingstate)
4/24/2012 Principles of Teletraffic Engineering 14
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
15/38
Common QuestionA common question arising in Markov-chain models
is, what is the long-term probability that the system
will be in each state?-
called the steady-state vector of the Markov chain
4/24/2012 Principles of Teletraffic Engineering 15
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
16/38
Steady-state matrixThe steady-state probabilities are average probabilities
that the system will be in a certain state after a large
number of transition periods-
independent of the initial distribution
Long-term probabilities of being on certain squares
mPn
n=
lim
4/24/2012 Principles of Teletraffic Engineering 16
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
17/38
The Chutes and
Matrix
4/24/2012 Principles of Teletraffic Engineering 17
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
18/38
Analytical ComputationProgramming Language: Java
Objectives:
Compute the transition matrixCompute t e pro a i ity vectors
4/24/2012 Principles of Teletraffic Engineering 18
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
19/38
Techniques of Transition MatrixHow we did the analytic computations2D array of integers of size 101 by 101
Each array entry represents a move from one square too e
i.e. Pij represents the players move from position i to position j
A square with a ladder beginning in that square is considered apseudo-state and has a 0 probability of landing on it
A square with a chute beginning in that square is considered apseudo-state and also has a 0 probability of landing on it
4/24/2012 Principles of Teletraffic Engineering 19
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
20/38
Results of Transition MatrixThe matrix is way too large to show on this slide
Some example probabilities computed:
If a player is on square 97, the probability of staying ona square s 3 an e pro a y o mov ng o
square 78 is 1/6
4/24/2012 Principles of Teletraffic Engineering 20
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
21/38
Techniques of Probability VectorsHow we did the analytic computations
1D array of integers of size 101
Probability vectors Vn represent the probability of being
Vo = {1,0,0} and means that the probability of being onsquare 0 is 1 or 100%
Vo * P = V1;
V1 * P = V2, etc
4/24/2012 Principles of Teletraffic Engineering 21
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
22/38
Results of Transition MatrixAfter 1000 moves, the probability vector reached a
limit of {0,0,1}
This means that after 1000 moves, the game is
4/24/2012 Principles of Teletraffic Engineering 22
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
23/38
Simulation
Techniques
4/24/2012 Principles of Teletraffic Engineering 23
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
24/38
SimulationProgramming Language: C++
Objectives:
Find frequencies for being at each positionFind mean num er o moves to win
Find standard deviation
Simulate a large number of games
4/24/2012 Principles of Teletraffic Engineering 24
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
25/38
Technique OneHow we simulated the gameArray of 101 integers representing the board
Each array entry represented a state Psuedo-states held the value at the end of the ladder or the
chutei.e. Index 1 represented square 1 and held 38. If index became1 it would move to square 38.
Normal states held the index of that state. Index would be
compared to value and be the same so index would stay at itsvalue.
4/24/2012 Principles of Teletraffic Engineering 25
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
26/38
Technique OneWhen index became 100 game would be over.
If index was greater than 100 it would reset to itsprevious. This repeats until index 100 is hit.
Mean and standard deviation were calculatedthroughout the game.
Printed most moves to win and least moves.
Printed number of times each square was landed on for
250,000 games and frequency of each.
4/24/2012 Principles of Teletraffic Engineering 26
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
27/38
Technique One ResultsWe ran 250,000 games
Mean is approximately 39.65 moves to reach square
100.e stan ar ev at on s approx mate y 24.00
4/24/2012 Principles of Teletraffic Engineering 27
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
28/38
Technique TwoRun the game as a non-absorbing.
i.e. If the index is 100 or above subtract 100 to run the
game as a non-ending board..
This was done to compare to the non-absorbinganalytical model.
4/24/2012 Principles of Teletraffic Engineering 28
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
29/38
Repeated Play
4/24/2012 Principles of Teletraffic Engineering 29
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
30/38
Analytic ComputationProgramming Language: Java
Objectives:
Compute the non-absorbing matrixCompute t e corresponding pro a i ity vectors
4/24/2012 Principles of Teletraffic Engineering 30
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
31/38
Techniques of Repeated PlayHow we did the analytic computations:
Similar to transition matrix, except square 100 cannot be
landed on,
board (Similar to Monopoly)
The game cannot be won using this method
Comparison will be done to simulation results
4/24/2012 Principles of Teletraffic Engineering 31
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
32/38
Theoretical Experimental
=.0 =.0
%581.05 =P%552.05 =P
%89.226 =P%94.226 =P
%09.242 =P %13.242 =P
%573.065 =P%599.065 =P
%452.099 =P %441.099 =P
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
33/38
Conclusion
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
34/38
The Theoretical Result and TheExperimental Result matched
The Markov Chain works in Chutes andLadders
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
35/38
Chutes and Ladders is a game for 3-6 years.
We can make it more interesting by
changing some rules
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
36/38
Special Thanks to Dr. Deckelman
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
37/38
Sources Mooney and Swift. A Course In Mathematical
Modeling. MAA Publications, 1999.
4/24/2012 Principles of Teletraffic Engineering 37
-
8/2/2019 Intro to MarkocChain Chutes-And-Ladders
38/38
End
4/24/2012 Principles of Teletraffic Engineering 38