a novel structural ar modeling approach for a continuous time linear markov system
DESCRIPTION
A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System. Demeshko M., Washio T. and Kawahara Y. Institute of Scientific and Industrial Research, Osaka University. Introduction. Continuous time, multivariate, stationary linear Markov system. ≈. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/1.jpg)
A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System
Demeshko M., Washio T. and Kawahara Y.
Institute of Scientific and Industrial Research, Osaka University
![Page 2: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/2.jpg)
▫ Continuous time, multivariate, stationary linear Markov system
2
Introduction
Discrete Vector Autoregressive (DVAR)
model
Discrete Autoregressive Moving Average (DARMA)
model
Exactly represented
≈
Approximated
≈
We focus on objective system
exactly described by AR
processes.
![Page 3: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/3.jpg)
3
Introduction
Not structural
Each mathematical relation in DVAR
model doesn’t have a bijective
correspondence to an individual process in the objective system.
LimitationsDVAR model
Eq. 1
Eq. N
Process 1Eq.
2… Process 2
Process N
…
We aim to analyse the mechanism of the
objective system and DVAR model is not applicable for such
analysis.
DVAR model System
![Page 4: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/4.jpg)
4
DVAR model
They are equivalent for any regular Q,where W(t)=QU(t).
U(t)+t)jY(t=Y(t)p
1j
j
where Φj are d×d coefficient matrices. U(t) is a d-dimensional unobserved noise vector.
The lack of structurality of DVAR model
The DVAR model doesn’t represent the system uniquely, since their correspondence depends on
the choice of U(t).
W(t)+t)jY(tQ=Y(t)Q 1-p
1j
1-
j
Process 1
…
Process 2
Process N
…
System
…
Equations
Equations
u1(t)+u2(t)+
uN(t)+
w1(t)+w2(t)+
wN(t)+…
…
![Page 5: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/5.jpg)
5
Structural VAR model (unique P=Q)
W(t)+t)jY(t=Y(t)p
0j
j
where Ψ0=I–P-1, Ψj=P-1Φj, W(t)=PU(t).
DVAR model
U(t)+t)jY(t=Y(t)p
1j
j
≈
W(t),+(t)Y=(t)Y (m)1-p
0m
(p)
mS
Continuous time VAR (CTVAR) model
Stru
ctur
al
VAR models structurality
Process 1
…Process
2
Process N
…
System
…
… Each equation of the SVAR and CTVAR models have one-to-one correspondence
to the system’s processes.
(t)w+(t)=(t) 1(m)1
1-p
0m
1(p)1 ysy m
(t)w+(t)=(t) N(m)1
1-p
0m
(p)N ysy N
m
…
Equationsu1(t)+u2(t)+
uN(t)+
…
Equationsw1(t)+w2(t)+
wN(t)+
…Equations
w1(t)+w2(t)+
wN(t)+
…
![Page 6: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/6.jpg)
6
We propose a new approach of SVAR and CTVAR modeling without any strong assumptions or domain knowledge on the objective system.
Related WorkPast SVAR modeling approaches
required assumptions and domain knowledge .
• Acyclic dependency among variables in Y(t);• Non-Gaussianity of noises;• All noises are mutually independent.
Hyvärinen et al. 2008
Significantly limit the applicability of the approaches.
![Page 7: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/7.jpg)
7
Research objectives• To show that a DVAR model uniquely represents the
objective system, if the system is continuous time, multivariate, linear Markov system.
• To clarify mathematical relations among a Continuous time VAR (CTVAR) model, a Structural VAR (SVAR) model and a DVAR model of the system.
• To propose a new approach, CSVAR modeling to derive the CTVAR and the SVAR models by using the DVAR model obtained from observed time series data.
• To demonstrate the accuracy and the applicability of CSVAR modeling using artificial and real world time series.
![Page 8: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/8.jpg)
8
Proposed principle and algorithm
![Page 9: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/9.jpg)
9
CTVAR model
SVAR model
This approximation is consistent.
Approximation error when
CTVAR model discrete approximationProcess
1Process 2
Process N
…
System
W(t)+(t)Y=(t)Y (m)1-p
0m
(p)
mS
CTVAR model
n
k
knn tktY
kknn
ttY
0
)(!)!(
!)1(1)(
High order finite difference approximation
Bijectiverelation
Continuous time domain
Discrete time domain
![Page 10: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/10.jpg)
Relation of CTVAR and SVAR models
10
p
0m0
mm tSI
mm
p
jm
jj tS
j!j)!-(mm!)1(
where Sp= –I.
)Ψ(IΨ t)1(I m!m)!-(p
p!t)1(m!m)!-(j
j!1 01-p
p-1
1
p-11
0
pp
m
mpj
p
mj
mpItS
I m!m)!-(p
p!t)1(m!m)!-(j
j!t)1( p-m11
m
pm
j
p
mj
mmS
where 1≤ m≤ p–1 and Sp= –I.
Theorem 1
CTVAR and SVAR models have bijective
correspondence.
Theorem 2
Under the assumption that objective system is stable and controllable
![Page 11: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/11.jpg)
11
Assuming that an objective system is continuous time, multivariate, linear Markov system represented by AR
processes.
Relation of SVAR and DVAR models
CTVAR model is given
SVAR model
If a unique P, i.e., Ψ0 are
given.
Bijectiverelation
DVAR model
W(t)+t)jY(t=Y(t)p
0j
j
U(t)+t)jY(t=Y(t)p
1j
j
Ψj=(I–Ψ0)Φj
W(t) =(I–Ψ0)U(t)
Φj=(I–Ψ0)-1Ψj
U(t) =(I–Ψ0)-1W(t)
Lemma 1
Lemma 2
Relation of SVAR and DVAR models
![Page 12: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/12.jpg)
12
Relation of SVAR and DVAR models
P=I–Ψ0=(–1)p+1Δt-pΦp-1
Theorem 3
GivesSVAR model DVAR model
SVAR and DVAR models have bijective correspondence, because a unique Ψ0, i.e., P, is
given as follows.Lemma 1Ψj=(I–Ψ0)Φj )Ψ(IΨ t)1(I
m!m)!-(pp!t)1(
m!m)!-(jj!1 0
1-p
p-1
1
p-11
0
pp
m
mpj
p
mj
mpItS
I m!m)!-(p
p!t)1(m!m)!-(j
j!t)1( p-m11
m
pm
j
p
mj
mmS
Theorem 2
p
0m0
mm tSI
mm
p
jm
jj tS
j!j)!-(mm!)1(
Theorem 1
By the constraints between CTVAR and SVAR, we obtain a relation of SVAR and DVAR.
![Page 13: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/13.jpg)
13
CSVAR modeling algorithm
DVAR model
SVAR model
CTVAR model
Y(t) data set
Maximum-Likelihood method
Lemma 1 and Theorem 3
Theorem 2
Estimate
Derive
Derive
![Page 14: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/14.jpg)
14
CTVARmodel
GenerateS
SVARmodel
DVARmodel
S*
Ψ
Ψ*
Φ
Φ*
ArtificialDVAR data
setMATLABestimation
Parameters generation using provided relations
Com
pare
Numerical Performance Evaluation Using Artificial simulation data
![Page 15: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/15.jpg)
15
Artificial data generationCTVAR modelSm (m=0,…,p–
1)
Each element of Sm is a uniformly distributed random value in the interval (–1.5, 1.5) , for given AR order p, dimension d, number of data points N .SVAR model
Ψj (j=0,…,p) Estimated from Sm by Theorem 1.
DVAR model
Φj (j=1,…,p)
Estimated from Ψj by Lemma 2.
Check the stability and the controllability Generate DVAR data set
U(t)+t)jY(t=Y(t)p
1j
j
Where U(t) is independently distributed Gaussian time series with zero mean value, standard deviation from [0.3, 0.7].
![Page 16: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/16.jpg)
16
Accuracies evaluation
Over different dimensions d, when N=1000,
and р=2.
p
1k2*,
2,
ij ,*
,1Aij ijkij ijk
ijkijkX
xx
xx
p
Accu
racy
Accu
racy
Accu
racy Over different
AR orders p, when d=5 and
N=1000.
Over different numbers of data points N, when d=5 and р=2.
d p
N
S
Ф
AAA
S
Ф
AAA
S
Ф
AAA
![Page 17: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/17.jpg)
17
Comparison with AR-LiNGAM method
(a) non-Gaussian and acyclic
(b) non-Gaussian and
cyclic
(c) Gaussian and acyclic(d) Gaussian and cyclic
d=5, N=1000, p=2 and q=2.
AR-LiNGAM is a past
representative SVAR modeling
method.
![Page 18: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/18.jpg)
Performance demonstration in practical application
О сн ов н о й п од в и ж н ы й от р а ж ат е л ь
С т а ц и о н а р н ы й от р а ж ат ел ь
А к ти в н а я зо н а
Д о п ол н и т ел ь н ы й п о дв и ж н ы й от р а ж ат ел ь
Ко р п ус р е ак то р а
З а м едл и т ел ь
Reactor core
Moderator
StationaryreflectorMain
neutronreflectorAdditional neutronreflector
Reactor body
18Peak pulse power 1500 МWPulse Frequency 5 Hz
1.2 m
4.5 m
18Nuclear Reactor IBR-2
situated in JINR, Dubna.
![Page 19: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/19.jpg)
19
Axial deviations of the additional reflector (XA)
Performance demonstration in practical application
Energy of power pulses Q
Axial deviations of the main reflector (XQ)Neutrons
Reactor core
![Page 20: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/20.jpg)
20
SVAR
A
Q
XXQ
-1.110.530.53-0.10-0.860.602.7211.791.09
0
A
Q
XXQ
0.78-0.490.17-0.610.580.14-2.14-4.220.07
1
A
Q
XXQ
-1.000.000.000.00-1.000.000.000.001.00-
2
Q XQ XA
Q XQ XA
Q XQ XA
CTVAR
A
Q
XXQ
S
-1.151.160.24-0.010.150.125.7821.291.09-
0
A
Q
XXQ
S
0.46-0.150.060.200.390.02--0.170.050.05
1
Q XQ XA
Performance demonstration in practical application
Q XQ XA
![Page 21: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/21.jpg)
21
DVAR
A
Q
XXQ
0.46-0.150.060.200.390.02--0.170.050.05
1
Q XQ XA
Performance demonstration in practical application
A
Q
XXQ
0.210.050.08-0.120.190.010.030.150.03
2
Q XQ XA
?
![Page 22: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/22.jpg)
22
• We showed that the DVAR model uniquely represents the continuous time, multivariate, linear Markov system.
• We clarified mathematical relations between the CTVAR, the SVAR and the DVAR models.
• Proposed modeling approach accurately derives the CTVAR and the SVAR models from the DVAR model under a generic assumption.
• We demonstrated the practical performance of our proposed approach through some numerical experiments using both artificial and real world data.
Conclusion
![Page 23: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/23.jpg)
23
References• [1] P.J. Brockwell and R.A. Davis, “Time Series: Theory and Methods”, Springer, 2nd ed.,
1991.• [2] O. Stamer, R.L. Tweedie and P.J. Brockwell, “Existence and Stability of Continuous Time
Threshold ARMA Process”, Statica Sinica, Vol.6, 1996, 715-732.• [3] J. Gottschalk, “An Introduction into the SVAR Methodology: Identification, Interpretation
and Limitations of SVAR models”, Kiel Working Paper, No.1072. Institute of World Economics, Kiel, 2001.
• [4] A. Moneta, D. Entner, P. O. Hoyer and A. Coad. Causal inference by independent component analysis: Theory and applications. Oxford Bulletin of Economics and Statistics, 75(5): 705-730, 2013..
• [5] A. Hyvärinen, S. Shimizu and P. Hoyer, “Casual Modeling Combining Instantaneous and Lagged Effects: an Identifiable Model Based on Non-Gaussianity”, Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 2008, 424-431.
• [6] B. Pfaff and T. Kronberg, “VAR, SVAR and SVEC Models: Implementation within R Package vars”, Journal of statistical software, Vol.27, No.4, 2008, 1-32.
• [7] L. Kilian, “Structural Vector Autoregressions, Handbook of Research Methods and Applications on Empirical Macroeconomics”, Edward Elger, 2011.
• [8] J. Pearl, "Causality: Models, Reasoning, and Inference", Chap.1 and Chap.5, Cambridge University Press, 2000.
• [9] F. Fisher, “A Correspondence Principle for Simultaneous Equation Models”, Econometrica, Vol. 38, № 1, 1970, 73-92.
• [10] Y. Iwasaki and H.A. Simon, “Causality and Model Abstraction”, Articial Intelligence, Vol.67, 1994, 143-194.
![Page 24: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/24.jpg)
24
• [11] G. Lacerda, P. Spirtes, J. Ramsey, and P. O. Hoyer, “Discovering cyclic causal models by independent components analysis, Proceedings of the 24th conference on Uncertainty in Artificial Intelligence, 2008.
• [12] J.M. Mooij, D. Janzing and B. Scholkopf, “From Ordinary Differential Equations to Structural Causal Models: the deterministic case”, Proceedings of the 29th conference on Uncertainty in Artificial Intelligence, 2013.
• [13] Y. Kawahara, S. Shimizu and T. Washio, “Analyzing Relationships between ARMA Processes Based on Non-Gaussianity of External Influences”, Neurocomputing, Vol. 74, № 12-13, 2011, 2212-2221.
• [14] R.J. LeVeque, “Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-state and Time-dependent Problems”, SIAM e-books, 2007.
• [15] P. Olver, “Applications of Lie groups to differential equations”, Springer, New-York, 1993, 318.
• [16] B. L. Shea, “Estimation of multivariate time series”, Journal of Time Series Analysis, Vol. 8, 1987, 95–110.
• [17] S. Shimizu, T. Inazumi, Y. Sogawa, A. Hyvärinen, Y. Kawahara, T. Washio, P. O. Hoyer and K. Bollen, “DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model”, J. of Machine Learning Research, Vol. 12, 2011, 1225-1248.
• [18] Yu. N. Pepyolyshev, “Spectral characteristics of power noise parameters and fluctuations of neutron reflectors of nuclear reactor IBR-2”, Preprint, JINR, Dubna (in Russian), 1988.
• [19] M. Voortman, D. Dash and M.J. Druzdzel, "Learning Why Things Change: The Difference-Based Causality Learner", Proceedings of the 26th conference on Uncertainty in Artificial Intelligence, 2010
References
![Page 25: A Novel Structural AR Modeling Approach for a Continuous Time Linear Markov System](https://reader036.vdocuments.site/reader036/viewer/2022070421/568161c9550346895dd1ad0c/html5/thumbnails/25.jpg)
25
Thank you for your attention