markov models markov chains · 26/02/2018 markov models 85 markov models •advantage: it provides...
TRANSCRIPT
MARKOV MODELSMARKOV CHAINS
Lecture 4.
26/02/2018 Markov Models 82
Markov Models
• Markov Model: is a stochastic model to describe a randomly changing system,
• by Andrej Andrejevics Markov – russan mathematician, 1856 – 1922
26/02/2018 Markov Models 83
Markov Models
• Example: Lupus
26/02/2018 Markov Models 84
Markov Models
• Markov Model: is a stochastic model to describe a randomly changing system,
• properties:
• it is a stochastic process without after-effects = the future states depend only on the current
state,
• the behaviour of the system depends directly on the preceding event, and is independent
from the past,
• the system states and the transition between the states are to be considered as a random event.
26/02/2018 Markov Models 85
Markov Models
• advantage: it provides a more powerful way of modelling systems, that are repairable,allowing variables such as the time taken to repair a system to be incorporated,
• described by state diagrams (chaines), eg: the reliablity graph of a two-state system:
• where:
• λ = failure rate [1/hour] – transition rate,
• µ = repair rate [1/hour] – transition rate, MTTR= 1/µ,
• 0: represents the working conditions,
• 1: represents the not working conditions.
26/02/2018 Markov Models 86
Markov Models
• Markov – chain is a stochastic model, in which the probability of an event depends
directly on the preceding event.
• independent of any previous event,
• the model detrmines the future behaviour of the given system,
• state of the system:
• several possible states (e.g. working/not working for just one element),
• state transitions:
• random events/stochastic event.
26/02/2018 Markov Models 87
Markov Models
• state transition described by:• state transition frequencies, rates [1/h] (λ, µ)
• constatnt rates: homogenous Markov-chain - described by two values:
• the probability of the initial state, Pi(0),
• the probability of the state transition Pji(∆t), where i menas the initial state and j means thefollowing state,
• example:
• if j=i+1, Pji(∆t)= λji∆t – fauliure,
• if j=i-1, Pji(∆t)= µji∆t – repairation,• time-dependent rates: semi Markov-chain,
• representation:• state graph,
• mathematical model:• differential equation system.
26/02/2018 Markov Models 88
Markov Models
• application in the reliability theory:
• modelling systems with multiple failure states:
• eg. redundant systems,
• modelling systems with multidirectional state transitions:
• eg. repairable systems,
• application in the modelling:
• to determine the status of the sytem,
• knowing the architecture and the properties of a system real engineering task,
• determination of the transition rates:
• based on the architecture and the properties of the system/components.
26/02/2018 Markov Models 89
Markov Models
• application in the modelling:
• types of the graphs of a non-repairable system:
• transient (operable) states,
• absorbent (inoperable) states,
• a systems with multiple failure states may have several absorbent states (eg. passive and dangerous).
0
1
2
λ10
λ21
0
1 2
λ10 λ20
26/02/2018 Markov Models 90
Markov Models
• application in the modelling:
• types of the graphs of a repairable system:
• the states of a repairable system are achievable
periodically, that’s why those forme a closed conditions
set
0
1
2
λ10
λ21
μ01
μ12
26/02/2018 Markov Models 91
Mathematical Model
• the state model can be described mathematically by state equations
• steps to determine the state equations:
1. determination of the difference equations,
2. conversion to differential equation,
3. conversion to matrix form.
26/02/2018 Markov Models 92
Mathematical Model
0
1
2
λ10
λ21
ttPtPttP
ttPttPtPttP
ttPtPttP
ttptptPtPttP
21122
21110011
10000
101010000
tPtP
tPtPtP
tPtP
tPdt
tdP
tPt
tPttP
t
121
'
2
121010
'
1
010
'
0
'
00
0
01000
lim
tP
tP
tP
tP
tP
tP
2
1
0
21
2110
10
'
2
'
1
'
0
00
0
00
tPAtP '
example: a 3 state system –
passive redundancy with n=2
26/02/2018 Markov Models 93
Mathematical Model
• remarks:
• the transition matrix is deduceable directly from the state graph,
• in the transition matrix, the sum of column is zero,
• in the main diagonal, the negative values in a given column are equals with the sum of the other
values.
0
1
2
λ10
λ21
tP
tP
tP
tP
tP
tP
2
1
0
21
2110
10
'
2
'
1
'
0
00
0
00
tPAtP '
26/02/2018 Markov Models 94
Examples
0
1 2
λ10 λ20
0
1
2
λ10
λ21
μ01
μ12
tP
tP
tP
tP
tP
tP
2
1
0
20
10
2010
'
2
'
1
'
0
00
00
00
tP
tP
tP
tP
tP
tP
2
1
0
1221
12012110
0110
'
2
'
1
'
0
0
0
• models of previously learned systems:
• single system/element/component without repair:
26/02/2018 Markov Models 95
Markov Models
0 1
λ
tetP
11
tetP
0
1T
0
0
connection?
26/02/2018 Markov Models 96
Markov Models
• models of previously learned systems:
• active redundancy (parallel), 1 out of n system (here n=2):
0 1 2
2λ λ
tteetP
2
1 2
tetP
2
0
2
3T tPtPtR 10
00
02
002
connection?
𝑃2 𝑡 = 1 − 2𝑒−λ𝑡 − 𝑒−2λ𝑡 = 𝑄(𝑡)
26/02/2018 Markov Models 97
Markov Models
• models of previously learned systems:
• passive redundancy:
0 1 2
λ λ
tettR
1
2T
00
0
00
connection?
26/02/2018 Markov Models 98
Markov Models?
• Solution of the differential equation system?
• with the aim of Laplace and Inverse Laplace Transformation
26/02/2018 Markov Models 99
Markov Models
• more information:
• https://www.youtube.com/watch?v=EqUfuT3CC8s
• https://www.youtube.com/watch?v=Ws63I3F7Moc
• https://www.youtube.com/watch?v=uvYTGEZQTEs
• https://www.youtube.com/watch?v=afIhgiHVnj0
End of Lecture 4.
Thank you for your attention!