jiann-ming wu, ya-ting zhou, chun-chang wu national dong hwa university department of applied...
TRANSCRIPT
Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu
National Dong Hwa University
Department of Applied Mathematics
Hualien, Taiwan
Learning Markov-chain embedded recurrence relations for chaotic time series analysis
Outline
Introduction High-order Markov processes for stochastic
modeling Nonlinear recurrence relations for deterministic
modelingRecurrence relation approximation by
supervised learning of radial or projective basis functions
Markov-chain embedded recurrence relationsNumerical Simulations Conclusions
High-order Markov assumptionLet Z[t] denote time series, where t is positive
integersHigh-order Markov assumption-
Given chaotic time series are oriented from a generative source well characterized by a high-order Markov process.
An order- Markov process obeys memory-less property
Current event only depends on instances of most recently events instead of all historic events
])[Z],...,1[Z|][ZPr(])1[Z],...,[Z],...,1[Z|][ZPr( tttttt
Recurrence relationConditional expectation of an upcoming
event to most recently events is expressed by a recurrence relation
eventcurrent denotes
][][
events previous denotes
])[],...,1[(][
]),[(G][
])[],...,1[(G
] -z[t,1],-z[t|Z[t]][
tzty
tztzt
where
tty
lyequivalent
tztz
ty
T
x
x
1000,,
]),[]2[]1[sin(
)][],...,1[G(][
5
521
t
tzatzatza
tztztz
Ttztztzt ][,],2[],1[][ x
])[sin(
])[(G][T t
tty
xa
x
][][ tytz
T21 ),,,( aaa a
Recurrence relation for time series modeling
predictor
target
0 500 1000 15000.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Mackey-Glass 30 chaotic time series data
Chaotic time series
Laser data 10000 from the SFI competition
RECURRENCE RELATION APPROXIMATION
• Learning neural networks for approximating underlying recurrence relation
• F denotes a mapping realized by radial or projective basis functions
• denotes adaptive network parameters
)][(])[(G ][ tFtty xx
Recurrence relation approximation
Form paired predictor and target by assigning
Define the mean square error of approximating
Apply Levenberg-Marquardt learning to resolve unconstrained optimization
Apply the proposed pair-data generative model to formulate F
)(min Eopt
])[],[( tytx
ttztytztzt T allfor ][][ and ])[],...,1[(][ x
)][(by ][ tFty x
2)|][(][(1
)( tFtyN
Et x
Pair-data generative model (PGM)
K sub-models
Mixtures of paired Gaussians
A stochastic model for formation emulation of given paired data
Each time one of joined pairs is selected according to a set of prior probabilities
Apply the selected paired Gaussians to generate paired data
])[],[( tytx
Each pair is exactly generated by a sub-modelLet denote the exclusive
membership of where denotes a unitary vector with the ith bit
active
By exclusive membership
The conditional expectation of y to given x is defined by
r denotes local means of the target variable
Exclusive Memberships
,,,][ M21 eeeδ t
model-subkth by the generated is ][],[ if ][ tytt k xeδ
0,,1,,0 i e
][],[ tytx
ie
rδx Ty
Overlapping membershipsA Potts random variable is applied to
encode overlapping membershipThe probability of being the kth state is set
to
where modulates the overlapping degree and
denotes local mean of the predictor
δ
)][exp(Pr2
kk t μxeδ
kμ
Normalized radial basis functions ( NRBF )
The conditional expectation exactly sketches a mapping realized by normalized radial basis functions
][exp
][exp
,,Let
][exp
][expPr
followsit
1Pr and )][exp(Pr Since
2
2
1
1
2
2
1
2
k
hh
kkT
TM
M
hh
kkk
M
kkkk
t
try
vv
t
tv
t
μx
μxrvx
v
μx
μxeδ
eδμxeδ
Figure 4
0 500 1000 15000
0.5
1
1.5source
0 50 100 150 200 250 300 350 400 450 5000
0.5
1
1.5approximation
0 50 100 150 200 250 300 350 400 450 500-4
-2
0
2
4x 10
-3 approximating error
Figure 9
Mackey-Glass 17 chaotic time series data
Multiple recurrence relationsMultiple recurrence relations for modeling
more complex chaotic time series
Chaotic time series
Laser data 10000 from the SFI competition
Markov-chain embedded recurrence relations
A Markov chain of PGMs (pair-data generative models)
Transition matrix
denotes the probability of transition from model i to model j
, jiT
jiT ,
Data generation
Emulate data generation by a stochastic Markov chain of PGMs
Inverse problem of Markov chain embedded PGMs
ies.probabilitn transitioEstimate 2.
on.segmentatifor points switching Find 1.
as stated becan tion reconstruc modelfor problem inverse The
PGMs embeddedchain -Markovby generated sequence orderedan Given
Segmentation for phase changeA time tag is regarded as a switching point
if its moving average error greater than a threshold value
2))];[(][(][ θxt
ti
tFtyterror
A simple rule for merging two PGMsThe goodness of fitting the ith PGM to
paired data in Sj is defined by
Two PGMs are merged. Si and Sj are regarded from the same PGM if (Ei,j+Ej,i)/2 is less than a threshold value
2
][, ))];[(][(
1E j
Stiji
j
tFtyS
θxx
state.hidden same theform oriented regarded are and
2
1 If ,,
ji
ijji
SS
EE
NUMERICAL SIMULATIONS – Synthetic data
on.segmentatilength -fixedfor sampling regressive-auto of size window the:
thresholdpositive determined-pre a :
scale. short time :
reduction after stateshidden required ofnumber the:
0N
K
] 200 0.1, 5, 3, [),,,(
exp,cos,sin : functionsnonlinear -post
0
321
NK
xxx TTT
aaa
matrix nTranslatio
Temporal sequence generated by MC-embedded PGMs
Numerical results – original and reconstructed MC-embedded PGMs
matrix nTranslatio
Chaotic time series
Markov chain embedded recurrence relations
Generated chaotic time series
Laser data 10000 from the SFI competition
M=60,[K, , , N0 ] = [ 10, 10, 0.001, 500 ]
Learning
Conclusions This work has presented learning Markov-chain
embedded recurrence relations for complex time series analysis.
Levenberg-Marquardt supervised learning of neural networks has been shown potential for extracting essential recurrence relation underlying given time series
Markov-chain embedded recurrence relations are shown applicable for characterizing complex chaotic time series
The proposed systematic approach integrates pattern segmentation, hidden state absorption and transition probability estimation based on supervised learning of neural networks