8. Multivariate Linear Time Series.

MA6622, Ernesto Mordecki, CityU, HK, 2010

References for this Lecture:

Introduction to Time Series and Forecasting. P.J. Brockwelland R. A. Davis, Springer Texts in Statistics (2002)

Analysis of Financial Time Series (Chapter 8). Ruey S. Tsay.Wiley (2002) [Available Online]

1

Main Purpose of Lectures 8 and 9:

Model the time evolution of a portfolio contanining d assets,with returns

X(0),X(1), . . . ,X(n)

where

X(t) =

X1(t)...

Xd(t)

(t = 0, . . . , n)

through a multivariate linear time series model.

2

Plan of Lecture 8

• Report stylized facts in multivariate financial time series inother words, how these series look like, from a statisticalpoint of view.

• Introduce some necessary facts from matrices and multivari-ate statistics.

• Testing normality (key issue in finance)

• Testing multivariate normality, i.e. whether we can assumethat a sample of multivariate data (vectorial data) can beassumed to be normaly distributed.

• Introduce the concepts of Multivariate Time Series

3

8a. Stylized facts in multivariate financial time series

Empirical observations on daily returns of financial time seriesled to the following 4 stylized facts, widely understood to beempirical truths, to which theories must fit.

(M1) Multivariate Return series show little evidence of cross-correlation,except for concurrent (i.e. contemporaneous) returns.

The cross-covariance, or covariance cov(X(s),X(t)′)

for s 6=t, is generally negligible, as in the one dimensional case.When t = s and i 6= j the (concurrent) covariances cov

(

Xi(t), Xj(t))

can be non negligible due to factors affecting the whole mar-ket.

4

(M2) Multivariate series of absolute returns show profound serialcorrelations for different times (cross-correlation).

As in the one dimensional case, large movements in one stocktend to be followed by large movements in this stocks, andalso in other stocks of the same market. As previously, finan-cial time series are uncorrelated but not independent.

(M3) The covariance structure of concurrent returns vary over time.

Consistenly with the same phenomena of volatility time varia-tion in the univariate case, and with the previous phenomenaof clustering of large returns, it seems that the covariance ofX(t) vary with t. (This raises the question of modelling thisphenomena, for instance with mutivariate GARCH processes).

5

(M4) Extreme returns in one asset often conicide with extreme re-turns in several other assets.

This fact asserts that in high volatility periods of the market,assets seem to be more correlated, and has as limit statementthat “correlations go to one in times of market stress”.

6

8b. Elements of multivariate statistics

Given a random vector

X =

X1...

Xd

we denote its transpose by

X′ = (X1, . . . , Xd)

Given two vectors X and Y

• the product XY′ is a matrix:

XY′ =

X1...

Xd

(Y1, . . . , Yd) =

X1Y1 . . . X1Yd... ... ...

XdY1 . . . XdYd

7

• while X′Y is a number:

(X1, . . . , Xd)

X1...

Xd

= X1Y1 + · · · + XnYn.

The expectation of the random vector X is

EX = (EX1, . . . ,EXd)′,

the variance-covariance matrix of X is

Σ = cov(X) = [cov(Xi, Xj)]i,j=1,...,d.

The correlation matrix is

ρ(X) =[ cov(Xi, Xj)√

var(Xi)var(Xj)

]

i,j=1,...,d

8

Given a matrix A and a vector b we have:

E(AX+b) = AEX+b, cov(AX+b) = Acov(X)A′.

Definition The vector Z = (Z1, . . . , Zd)′ is a gaussian or

normal standard vector when Z1, . . . , Zd are independent stan-dard normal random variables.

For a standard normal vector

•EZ = (0, . . . , 0)′,• cov(Z) = Id, the d × d identity matrix.

A gaussian or normal vector Z with mean µ and covariance Σis obtained as

X = µ + AZ,

where the matrix A satisfies AA′ = Σ

9

•A is called a square root of Σ,

• Given a covariance matrix Σ, its squared root A always exist(linear algebra).

We denoteX ∼ Nd(µ, Σ).

Given X ∼ Nd(µ, Σ)

•EX = µ + AEZ = µ.

• For the variance-covariances matrix:

cov(X) = cov(AZ) = Acov(Z)A′ = AIdA′ = AA′ = Σ.

10

8c. Testing Univariate Normality.

Many results in statistics of time series are built on the hyphote-sis of gaussian returns (for instance, the Black-Scholes model).

It is then important to determine whether a sample of univariatereturns are gaussian

Quantile-Quantile Plot (QQ - Plot)

Is a visual test for univariate gaussianity.

Suppouse you want to know if the following sample of 9 values

0.22, 2.29, 2.06, 7.32, 7.05, 0.14, 7.51, 9.15, 4.21

can be considered sampled from a normal random variable.

11

In order to perform the QQ-Plot test

STEP 1. Order the sample in increasing order:

0.14, 0.22, 2.06, 2.29, 4.21, 7.05, 7.32, 7.51, 9.15

STEP 2. Compute x(i) = Φ(

(i − 1/2)/9)−1

(i = 1, . . . , 9)from the normal standard table. (These are representativepoints of 9 equiprobable intervals.) Prepare the table:

x(i) -1.593 -0.967 -0.589 -0.282 0 0.282 0.589 0.967 1.593y(i) 0.14 0.22 2.06 2.29 4.21 7.05 7.32 7.51 9.15

12

STEP 3. Plot the points (x(i), y(i)) for i = 1, . . . , 9:

-1.5 -1 -0.5 0 0.5 1 1.50

2

4

6

8

QQ Plot

If the ploted points fit approximately a straight line, the gaussianhypothesis is not rejected.

13

Skewness-Kurtosis Jarque-Bera test.

A second practical procedure is to simultaneously test whetherthe third and fourth centered moments corresponds to that ofa normal random variable. Given a sample X(1), . . . , X(n)

STEP 1. Estimate the mean and the variance by

X̄ =1

n

n∑

k=1

X(k), σ̄2 =1

n

n∑

k=1

(

X(k) − X̄)2

STEP 2. Compute the empirical skewness and kurtosis, by

γ̄ =1n

∑nk=1

(

X(k) − X̄)3

σ̄3, κ̄ =

1n

∑nk=1

(

X(k) − X̄)4

σ̄4−3

14

STEP 3. Compute the Jarque-Bera statistic as

QJB = n(1

6γ̄2 +

1

24κ̄2

)

∼ χ22,

that has, for big values of n, a Chi-square distribution withtwo degrees of freedom

STEP 4. Big values of QJB indicate that the skewness and/orthe kurtosis do not vanish (as should happen under normal-ity).

Then (with a 95% confidence) if

QJB > 5.99 = t2,0.95,

reject the hyphotesis of normality

Remark This test is valid for big values of n. For smallvalues we prefer the QQ-plot.

15

8d. Testing Multivariate Normality.

First remark: It is not enough to have normality for the co-ordinates of a vector (univariate marginals) in order to have anormal vector.

Suppose we want to test whether a given a vectorial sampleX(1), . . . ,X(n) is normal.

QQ Chi Square Plot

Based on the fact that, given a vector X ∼ Nd(µ, Σ), therandom variable

(X − µ)′Σ−1(X − µ) ∼ χ2d,

we construct a new sample. Its sample mean is

X̄ =1

n

n∑

k=1

X(k),

16

and the sample covariance matrix

Σ̄ =1

n

n∑

k=1

(

X(k) − X̄)(

X(k) − X̄)′

.

Invert the matrix Σ̄, and construct a new univariate sample ofthe form

D2k =

(

X(k) − X̄)′

Σ̄−1(

X(k) − X̄)

, k = 1, . . . , n.

For big values of n behaves rhoghly like a sample of χ2d inde-

pendent random variables.

Then, test this hypothesis with a QQ-plot.

The procedure is the same as in the univariate gaussian case,with the diffence that

x(i) = Fχ22

(

(i − 1/2)/n)

17

where Fχ22

is the χ22 distribution (i.e. one should use the Chi

square distribution with 2 degrees of freedom instead of thenormal table).

18

Skewness-Kurtosis Multivariate Test

We can also test for multivariate skewness and kurtosis.

Compute

Djk =(

X(j) − X̄)′

Σ̄−1(

X(k) − X̄)

, j, k = 1, . . . , n,

The statistics

γd =1

n2

n∑

i,j=1

D3ij, κd =

1

n

n∑

i=1

D2ii − d(d + 2)

have, when the random vector sample is normal, asymptoticsdistributions

1

6n γd ∼ χ2

d(d+1)(d+2)/6,κd

√

8d(d + 2)/n∼ N (0, 1).

We construct then two tests based on these facts.

19

8e. Multivariate time series

Let X(0), . . . ,X(n) be the stochastic returns of a portfolio with

d assets, where X(t) =(

X1(t), . . . , Xd(t))′

,

For simplicity of exposition we assume that we have two assetsA and B, and our returns are

X(t) =

[

XA(t)XB(t)

]

.

The vector of expectations is

µ(t) =

[

µA(t)µB(t)

]

=

[

EXA(t)EXB(t)

]

,

and the covariance matrix

Γ(t+h, t) =

[

cov(XA(t + h), XA(t)) cov(XA(t + h), XB(t))cov(XB(t + h), XA(t)) cov(XB(t + h), XB(t))

]

20

Definition We say that the bivariate series is weakly sta-tionary when the expectations µ(t) ≡ µ does not depend on t,and also the the covariance matrix Γ(t + h, t) = Γ(h) does notdepend on t.

We have

Γ(h) =

[

ΓAA(h) ΓAB(h)ΓBA(h) ΓBB(h)

]

=

[

cov(XA(h), XA(0)) cov(XA(h), XB(0))cov(XB(h), XA(0)) cov(XB(h), XB(0))

]

.

Here

• The diagonal terms are the covariances of the univariate series{XA(t)} and {XB(t)}.

21

• New information appears in the off-diagonal elements, thatare the covariance between different assets.

•We call cov(XA(0), XB(0)) a concurrent covariance: samedate for different assets,

•We call cov(XA(0), XB(h)) a cross coviance: different datesand different assets.

Observe that in general:

• cov(XB(h), XA(0)) 6= cov(XA(h), XB(0)), so the matrixΓ(h) is not symmetric.

•We have

cov(XA(h), XB(0)) = cov(XB(−h), XA(0)),

what simplifies the estimation.

22

Defining

ρij(h) =Γij(h)

√

Γii(0)Γjj(0), for i, j = A,B

we construct the correlation matrix as

R(h) =

[

ρAA(h) ρAB(h)ρBA(h) ρBB(h)

]

23

9. Multivariate Linear Time Series (II).

MA6622, Ernesto Mordecki, CityU, HK, 2010.

References for this Lecture:

Introduction to Time Series and Forecasting. P.J. Brockwelland R. A. Davis, Springer Texts in Statistics (2002)

Analysis of Financial Time Series (Chapter 8). Ruey S. Tsay.Wiley (2002) [Available Online]

24

Main Purpose of Lectures 8 and 9:

Model the time evolution of a portfolio contanining d assets,with returns

X(0),X(1), . . . ,X(n)

where

X(t) =

X1(t)...

Xd(t)

(t = 0, . . . , n)

through a multivariate linear time series model.

25

Plan of Lecture 9

(9a) Plotting the Cross Correlogram of a multivariate time series

(9b) Introduce stationary multivariate time series and white noises.

(9c) Vectorial ARMA processes (VARMA), in particular VAR(1).

(9d) Testing multivariate white noise.

(9e) Comments on Co-integration

26

9a. Plotting the Empirical Cross-correlogram

The cross correlogram of our vectorial time series is a 2 × 2matrix of correlograms.

Correlogram of series 1 Cross-correlogram of series 1,2Cross-correlogram of series 2,1 Correlogram of series 2

In order to construct the cross-correlogram, we perform:

STEP 1. We compute the sample mean X̄ = (X̄A, X̄B) ofboth series:

X̄A =1

n

n∑

t=1

XA(t), X̄B =1

n

n∑

t=1

XB(t).

27

STEP 2. We compute the sample covariance matrix, for h =0, . . . , n0, as

Γ̄(h) =1

n

n−h∑

t=1

(X(t + h) − X̄)(X(t) − X̄)′

Here each term is is a 2 × 2 matrix:

(X(t + h) − X̄)(X(t) − X̄)′

=

[

(XA(t + h) − X̄A)(XA(t) − X̄A) (XA(t + h) − X̄A)(XB(t) − X̄B)(XB(t + h) − X̄B)(XA(t) − X̄A) (XB(t + h) − X̄B)(XB(t) − X̄B)

]

In particular for h = 0 the diagonal of the matrix gives

σ̄2A = Γ̄AA(0), σ̄2

B = Γ̄BB(0).

28

STEP 3. We compute the correlation matrix for h = 0, 1, . . . , n0as

R(h) =

Γ̄AA(h)

σ̄2A

Γ̄AB(h)σ̄Aσ̄B

Γ̄BA(h)σ̄Aσ̄B

Γ̄BB(h)

σ̄2B

STEP 4. We plot four graphics (h = 0, 1, . . . , n0):(

h, Γ̄AA(h)) (

h, Γ̄AB(h))

(

h, Γ̄BA(h)) (

h, Γ̄BB(h))

29

In this cross-correlograms:

• The diagonal gives information about the individual behaviourof each asset,

• the upper right correlogram shows the correlation of futurevalues of A against present values of B.

• the lower left correlogram shows the correlation of future val-ues of B against present values of A.

• Past values of A against present values of B are the same asfuture of B against present of A (ΓAB(−h) = ΓBA(h)), itis not necessary to plot them, and

• Past values of B against present values of A are the same asfuture of A against present of A (ΓBA(−h) = ΓAB(h)).

30

9b. Stationary Multivariate time series

Definition A multivariate series {X(t)} is

(a) a weak white noise: weakly stationary, EX(t) = 0 for all tand

Γ(h) =

{

Σε when h = 0

0 when h 6= 0

(b) a strict white noise: i.i.d. random vectors, with EX(t) = 0and covariance matrix Σε.

(c) a gaussian or normal white noise: strict white noise withdistributions N (0, Σε)

Remark In all cases Σε is a covariance matrix. The valuesof {X(t)} can have concurrent correlation (same time), but notcross-correlations (different times)

31

9c. Vectorial ARMA process (VARMA)

Definition {X(t)} is a vectorial ARMA(p,q) process if it iscentered weakly stationary and

X(t) − Φ1X(t − 1) − · · ·ΦpX(t − p)

= ε(t) − Θ1ε(t − 1) − · · · − Θqε(t − q)

where {ε(t)} is a weak white noise with covariance Σε. HereΦi and Θi are d × d matrices.

Example VAR(1) process satisfies

X(t) = ΦX(t − 1) + ε(t),

with {ε(t)} weak white noise. In order to check stationarity, oneshould have (instead of |φ| < 1 for d = 1) that all the eigen-values of the matrix Φ are strictly smaller than one in absolutevalue.

32

In order to estimate the matrix φ we must solve a matrix equa-tion (i.e. d × d linear equations) of the form

Γ̄(1) = Φ1Γ̄(0),

that can be solved computing the inverse of the matrix Γ̄(0),and post-multiplying both sides of the equation by this inversematrix we obtain

Φ̄(1) = Γ̄(1)Γ̄(0)−1.

Let us examine two particular cases: d = 1 and d = 2.

33

Case d = 1

In this case matrices are numbers. As our process is centered

Γ̄(0) = σ̄2X =

1

n

n∑

k=1

X(t)2

Γ̄(1) = cov(1) =1

n

n∑

k=1

X(t)X(t − 1)

giving the estimate

φ̄ = Γ̄(1)Γ̄(0)−1 =

∑nk=1 X(t)X(t − 1)∑n

k=1 X(t)2

from the previous lecture.

34

Case d = 2

Assume then that we have two financial time series A and B.

X(t) =

[

XA(t)XB(t)

]

, Φ =

[

φAA φABφBA φBB

]

, ε(t) =

[

εA(t)εB(t)

]

The model matrix model is:

X(t) = ΦX(t − 1) + ε(t).

In coordinates one has:

XA(t) = φAAXA(t − 1) + φABXB(t − 1) + εA(t)

XB(t) = φBAXA(t − 1) + φBBXB(t − 1) + εB(t)

In order to estimate Φ we perform:

35

STEP 1. Compute the (symmetric) sample (contemporane-ous) variance-covariance matrix

Γ̄(0) =1

n

n∑

t=1

[

XA(t)2 XA(t)XB(t)

XB(t)XA(t) XB(t)2

]

STEP 2. Compute the (non symmetric) sample cross-covariancematrix with lag h = 1

Γ̄(1) =1

n

n∑

t=1

[

XA(t)XA(t − 1) XA(t)XB(t − 1)XB(t)XA(t − 1) XB(t)XB(t − 1)

]

STEP 3. Invert the matrix Γ̄(0) to obtain

φ̄ = Γ̄(1)Γ̄(0)−1

STEP 4. Estimate the variance-covariance Σ̄ε matrix:

Σ̄ε = Γ̄(0) − φ̄Γ̄(1).

36

9d. Testing Multivariate White Noise.

Complementing the visual analysis of the cross-correlogram ofa bivariate time series X(t) = (XA(t), XB(t))′ we have a Mul-tivariate Portmanteau Test, proposed by Hosking (1980) thatextendes the Ljung and Box test of Lecture 6. The statisticaltest is

H0 : Γ(1) = · · · = Γ(n0) = 0, (X is WN)

Ha : Γ(h) 6= 0 for some h = 1, . . . , n0 (X is not WN).

To compute the test statistic, for each lag h = 1, . . . , n0, wecompute

q(h) = tr[Γ̄(h)′Γ̄(0)−1Γ̄(h)Γ̄(0)−1],

(where tr is the trace of the product of four matrices)

37

The statistic is

Q(n0) = n2n0∑

h=1

1

n − hq(h) ∼ χ2

4n0,

If X is WN, Q(n0) is Chi-Squared distribution with 4n0 degreesof freedom.

For n0 = 10, big values of Q(n0) indicate rejection of H0:

If Q(10) > t40,0.95 = 55.7585 reject H0.

Comments

• The 4 = d2. If d = 3 we have 9n0 degrees of freedom

•When d2n0 is large, we can use the normal approximation

Q(n0) ∼ N (n0, 2n0)

that, for confidence 0.95 and h = 40 gives

th,0.95 ∼ h + 1.645√

2h = 40 + 1.645√

80 = 54.7133.

38