markov models markov 26/02/2018 markov models 85 markov models •advantage: it provides a more...

Download MARKOV MODELS MARKOV 26/02/2018 Markov Models 85 Markov Models •advantage: it provides a more powerful

Post on 10-Feb-2020

2 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • MARKOV MODELS MARKOV CHAINS

    Lecture 4.

  • 26/02/2018 Markov Models 82

    Markov Models

    • Markov Model: is a stochastic model to describe a randomly changing system,

    • by Andrej Andrejevics Markov – russan mathematician, 1856 – 1922

  • 26/02/2018 Markov Models 83

    Markov Models

    • Example: Lupus

  • 26/02/2018 Markov Models 84

    Markov Models

    • Markov Model: is a stochastic model to describe a randomly changing system,

    • properties:

    • it is a stochastic process without after-effects = the future states depend only on the current

    state,

    • the behaviour of the system depends directly on the preceding event, and is independent

    from the past,

    • the system states and the transition between the states are to be considered as a random event.

  • 26/02/2018 Markov Models 85

    Markov Models

    • advantage: it provides a more powerful way of modelling systems, that are repairable, allowing variables such as the time taken to repair a system to be incorporated,

    • described by state diagrams (chaines), eg: the reliablity graph of a two-state system:

    • where:

    • λ = failure rate [1/hour] – transition rate,

    • µ = repair rate [1/hour] – transition rate, MTTR= 1/µ,

    • 0: represents the working conditions,

    • 1: represents the not working conditions.

  • 26/02/2018 Markov Models 86

    Markov Models

    • Markov – chain is a stochastic model, in which the probability of an event depends

    directly on the preceding event.

    • independent of any previous event,

    • the model detrmines the future behaviour of the given system,

    • state of the system:

    • several possible states (e.g. working/not working for just one element),

    • state transitions:

    • random events/stochastic event.

  • 26/02/2018 Markov Models 87

    Markov Models

    • state transition described by: • state transition frequencies, rates [1/h] (λ, µ)

    • constatnt rates: homogenous Markov-chain - described by two values:

    • the probability of the initial state, Pi(0),

    • the probability of the state transition Pji(∆t), where i menas the initial state and j means the following state,

    • example:

    • if j=i+1, Pji(∆t)= λji∆t – fauliure,

    • if j=i-1, Pji(∆t)= µji∆t – repairation, • time-dependent rates: semi Markov-chain,

    • representation: • state graph,

    • mathematical model: • differential equation system.

  • 26/02/2018 Markov Models 88

    Markov Models

    • application in the reliability theory:

    • modelling systems with multiple failure states:

    • eg. redundant systems,

    • modelling systems with multidirectional state transitions:

    • eg. repairable systems,

    • application in the modelling:

    • to determine the status of the sytem,

    • knowing the architecture and the properties of a system real engineering task,

    • determination of the transition rates:

    • based on the architecture and the properties of the system/components.

  • 26/02/2018 Markov Models 89

    Markov Models

    • application in the modelling:

    • types of the graphs of a non-repairable system:

    • transient (operable) states,

    • absorbent (inoperable) states,

    • a systems with multiple failure states may have several absorbent states (eg. passive and dangerous).

    0

    1

    2

    λ10

    λ21

    0

    1 2

    λ10 λ20

  • 26/02/2018 Markov Models 90

    Markov Models

    • application in the modelling:

    • types of the graphs of a repairable system:

    • the states of a repairable system are achievable

    periodically, that’s why those forme a closed conditions

    set

    0

    1

    2

    λ10

    λ21

    μ01

    μ12

  • 26/02/2018 Markov Models 91

    Mathematical Model

    • the state model can be described mathematically by state equations

    • steps to determine the state equations:

    1. determination of the difference equations,

    2. conversion to differential equation,

    3. conversion to matrix form.

  • 26/02/2018 Markov Models 92

    Mathematical Model

    0

    1

    2

    λ10

    λ21

             

         

           

          ttPtPttP

    ttPttPtPttP

    ttPtPttP

    ttptptPtPttP

    

    

    

    

    21122

    21110011

    10000

    101010000

    

         

     

       

       

         

       tPtP

    tPtPtP

    tPtP

    tP dt

    tdP

    tP t

    tPttP

    t

    121

    '

    2

    121010

    '

    1

    010

    '

    0

    '

    0 0

    0

    010 00

    lim

    

    

    

    

    

     

    

    

     

     

     

     

     

       

      

      

      

    

      

      

    tP

    tP

    tP

    tP

    tP

    tP

    2

    1

    0

    21

    2110

    10

    '

    2

    '

    1

    '

    0

    00

    0

    00

    

       tPAtP '

    example: a 3 state system –

    passive redundancy with n=2

  • 26/02/2018 Markov Models 93

    Mathematical Model

    • remarks:

    • the transition matrix is deduceable directly from the state graph,

    • in the transition matrix, the sum of column is zero,

    • in the main diagonal, the negative values in a given column are equals with the sum of the other

    values.

    0

    1

    2

    λ10

    λ21

     

     

     

     

     

       

      

      

      

    

      

      

    tP

    tP

    tP

    tP

    tP

    tP

    2

    1

    0

    21

    2110

    10

    '

    2

    '

    1

    '

    0

    00

    0

    00

    

       tPAtP '

  • 26/02/2018 Markov Models 94

    Examples

    0

    1 2

    λ10 λ20

    0

    1

    2

    λ10

    λ21

    μ01

    μ12

     

     

     

     

     

       

      

      

      

    

      

      

    tP

    tP

    tP

    tP

    tP

    tP

    2

    1

    0

    20

    10

    2010

    '

    2

    '

    1

    '

    0

    00

    00

    00

    

     

     

     

     

     

       

      

      

      

    

    

    

      

      

    tP

    tP

    tP

    tP

    tP

    tP

    2

    1

    0

    1221

    12012110

    0110

    '

    2

    '

    1

    '

    0

    0

    0

    

    

    

  • • models of previously learned systems:

    • single system/element/component without repair:

    26/02/2018 Markov Models 95

    Markov Models

    0 1

    λ

      tetP 11

      tetP 0 

    1 T

     

      

    0

    0

    connection?

  • 26/02/2018 Markov Models 96

    Markov Models

    • models of previously learned systems:

    • active redundancy (parallel), 1 out of n system (here n=2):

    0 1 2

    2λ λ

       tt eetP  21 2  

Recommended

View more >