![Page 1: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/1.jpg)
Learning Granger causality for multivariate time series using state-space models
Anawat Nartkulpat ID 5930562521
Advisor: Assist. Prof. Jitkomut Songsiri
Department of Electrical Engineering, Faculty of Engineering Chulalongkorn University
Senior Project Proposal 2102490 Year 2019
1
![Page 2: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/2.jpg)
Introduction
๐ฆ1
๐ฆ2
๐ฆ3
๐ฆ4 ๐ฆ5
Time series data
Causality pattern
2
![Page 3: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/3.jpg)
Model estimation
Computing Granger causality
Significance test/classification of zero causality
- Non-Iinear models - Linear models
- VAR - VARMA - State-space
๐ฆ1
๐ฆ2
๐ฆ3
๐ฆ4 ๐ฆ5
- Clustering methods - GMM [1]
- Statistical methods - Parametric - Non-parametric
- Permutation test - Bootstraps
Introduction
3
![Page 4: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/4.jpg)
Objectives
โข To develop a scheme for classifying the zero patterns of the Granger causality of state-space models using the permutation test.
โข To compare the performance of the permutation test with the Gaussian mixture models method in classification of zero and non-zero entries of Granger causality matrix obtained from state-space model
4
![Page 5: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/5.jpg)
Methodology
Figure 1. The scheme for learning GC pattern using state-space models.
Subspace identification
GC estimation
Estimated state-space matrices
๐ด , ๐ถ , ๐ , ๐ , ๐
Computing permutation distributions
under ๐ป0: ๐น๐๐ = 0
Cumulative permutation distribution ฮฆ๐๐(๐ฅ)
๐น
Significance test and
thresholding
GC zero pattern
Time series data
5
![Page 6: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/6.jpg)
Ground truth model
Generate a VAR model with sparse
GC matrix
Generate a diagonal filter
A State-space model
๐ฅ ๐ก + 1 = ๐ด๐ฅ ๐ก + ๐ต๐ค ๐ก ๐ฆ ๐ก = ๐ถ๐ฅ ๐ก
equivalent to ๐บ ๐ง ๐ดโ1(๐ง)
ฮฃ๐ค
๐ฆ ๐ก ๐ก=1๐ ๐บ ๐ง
๐ดโ1 ๐ง
Figure 2. The scheme for generating ground truth model and time series data.
6
Time series data
![Page 7: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/7.jpg)
Subspace identification
We consider estimating parameters ๐ด, ๐ถ, ๐, ๐, ๐ of a stochastic state-space model ๐ฅ ๐ก + 1 = ๐ด๐ฅ ๐ก + ๐ค ๐ก ๐ฆ ๐ก = ๐ถ๐ฅ ๐ก + ๐ฃ(๐ก)
This method is based on orthogonal projection. Suppose that the outputs ๐ is known, it was shown in [2] that
๐ช๐ โก ๐๐|2๐โ1 /๐0|๐โ1 = ๐๐/๐๐ (Projecting the future outputs onto the past output space)
๐ช๐ = ฮ๐๐ ๐ โน ๐ ๐ = ฮ๐โ ๐ช๐ and ๐ ๐+1 = ฮ๐โ1
โ ๐ช๐โ1
๐ ๐+1
๐๐|๐=
๐ด๐ถ
๐ ๐ +๐๐ค
๐๐ฃ โน
๐ด
๐ถ =
๐ ๐+1
๐๐|๐๐ ๐
โ
๐ ๐
๐ ๐ ๐ =
1
๐
๐๐ค
๐๐ฃ
๐๐ค
๐๐ฃ
๐
7
![Page 8: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/8.jpg)
Granger causality of state-space model
The measure of the Granger causality from ๐ฆ๐ to ๐ฆ๐ is defined by
๐น๐๐ = logฮฃ๐๐
๐
ฮฃ๐๐
where ฮฃ๐๐ is the covariance of the prediction error given all other ๐ฆ๐ and ฮฃ๐๐๐ is the
covariance of the prediction error given all other ๐ฆ๐ except ๐ฆ๐ [3]. Together with this definition, we can define a Granger causality matrix
๐น =
๐น11 ๐น12 โฏ ๐น1๐
๐น21 ๐น22 โฏ ๐น๐2
โฎ โฎ โฑ โฎ๐น๐1 ๐น๐2 โฏ ๐น๐๐
8
![Page 9: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/9.jpg)
For a state-space model ๐ฅ ๐ก + 1 = ๐ด๐ฅ ๐ก + ๐ค ๐ก ๐ฆ ๐ก = ๐ถ๐ฅ ๐ก + ๐ฃ(๐ก)
where ๐ฌ๐ค๐ฃ
๐ค๐ฃ
๐ป=
๐ ๐๐๐ ๐
, we consider the reduced model with
๐ฅ ๐ก + 1 = ๐ด๐ฅ ๐ก + ๐ค ๐ก ๐ฆ๐ ๐ก = ๐ถ๐ ๐ฅ ๐ก + ๐ฃ๐ (๐ก)
where ๐ฆ๐ is ๐ฆ without ๐ฆ๐ and ๐ถ๐ is ๐ถ without ๐๐กโ row.
We have ฮฃ = CPCT + V and ฮฃ๐ = ๐ถ๐ ๐๐ ๐ถ๐ ๐ + ๐๐ where ๐๐ is V without ๐๐กโ row and column and P is solved from DARE [1]
๐ = ๐ด๐๐ด๐ โ ๐ด๐๐ถ๐ + ๐ ๐ถ๐๐ถ๐ + ๐ โ1 ๐ถ๐๐ด๐ + ๐๐ + ๐
9
Granger causality of state-space model
![Page 10: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/10.jpg)
Permutation test
- The distribution of ๐น๐๐ is unknown
- The null hypothesis ๐ป0: ๐น๐๐ = 0 is to be tested
Justification: Under the true null hypothesis, ๐ฆ๐
is not Granger-caused by ๐ฆ๐, so rearranging data
in channel ๐ฆ๐ does not change the outcome.
So, we may form a distribution of ๐น๐๐ under ๐ป0 empirically from the permutations of data [4].
Figure 3. One of the possible permutations.
Window size ๐
10
![Page 11: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/11.jpg)
11
Permutation test
Figure 4. The scheme for calculating ๐-values.
โฎ
Permutation 1
Original ๐ฆ๐
๐น ๐๐(1)
๐น ๐๐(2)
๐น ๐๐(๐)
Cumulative permutation distribution
ฮฆ๐๐
๐๐๐ = 1 โ ฮฆ๐๐ ๐น ๐๐ for ๐ = 1, โฆ , ๐
Permutation 2
Permutation ๐
๐น ๐๐
![Page 12: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/12.jpg)
12
The number of permutations (๐) is the factorial of the number of windows. Since this number can be very large, we performed a Monte-Carlo permutation test in which the number of permutations used is smaller than the number of all possible permutations and the samples of rearrangement are drawn randomly [5]. Repeat for all ๐ and the ๐-value matrix is then obtained
๐11 ๐12 โฏ ๐1๐
๐21 ๐22 โฏ ๐2๐
โฎ โฎ โฑ โฎ๐๐1 ๐๐2 โฏ ๐๐๐
For a given significance level ๐ผ, ๐น๐๐ can be tested to decide that ๐น๐๐ = 0 or not by thresholding the ๐-values.
Permutation test
![Page 13: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/13.jpg)
Preliminary results
An experiment was performed to study how the number of permutation (๐) affects the performance of the permutation test
Hypothesis: The performance of the permutation test increase with the number of permutation
Control variable: We generated 15 ground truth models with the following specification - 5 channels - 20 states (generate a VAR models of order 2 and a filter with 2 poles) - 1000 time points for time series data The estimation of state-space models were done with exactly 20 states. Data were partitioned into 10 windows in permutation tests.
13
![Page 14: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/14.jpg)
14
Preliminary results
Figure 5. The performance of the permutation test.
![Page 15: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/15.jpg)
Figure 6. An example of the results after thresholding.
15
Preliminary results
![Page 16: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/16.jpg)
Project Planning
16
Figure 7. The Gantt chart of this project.
![Page 17: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/17.jpg)
17
Project Planning
We plan to do the following experiments in the next semester.
โข Experiment 2: Compare the performance of the permutation test with every permutations with the Monte-Carlo permutation. Experiment settings: With the same data, we use large window size (๐ = 5) for the normal permutation test so that the number of permutation is 5! = 120 which is feasible. Then we compare the result with the Monte-Carlo permutation test. โข Experiment 3: Study the effect of the number of states used in subspace identification on the performance of the permutation test. Experiment settings: With the same data, we estimate state-space models with different assumption on the number of states and compare them.
![Page 18: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/18.jpg)
18
Project Planning
โข Experiment 4: Compare the performance and the computational cost of the permutation test with the GMM method Experiment settings: With the same data, we compare the performance measures of the permutation test with the GMM method proposed in [1]. The parameters of the permutation test are chosen based on the previous result. The computation time is also compared.
![Page 19: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/19.jpg)
Reference [1] Songsiri, J., Learning brain connectivity from EEG time series, 2019
[2] Van Overschee P, De Moor BL. Subspace identification for linear systems: Theoryโ ImplementationโApplications. Springer Science & Business Media; 2012. [3] L. Barnett and A. K. Seth, โGranger causality for state space models,โ Physical Review E, vol. 91, no. 4, 2015 [4] Thomas E Nichols and Andrew P Holmes. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Human brain mapping, 15(1):1-25, 2002. [5] Good, P., Permutation, parametric, and bootstrap tests of hypotheses. 2005.
19
![Page 20: Learning Granger causality for multivariate time series](https://reader033.vdocuments.site/reader033/viewer/2022061102/629cb9f22a219232150228ca/html5/thumbnails/20.jpg)
Q&A
20