Dates for term tests

1. Friday, February 5

2. Friday, March 5

3. Friday, March 26

The Moving Average Time series of order q, MA(q)

where {ut|t T} denote a white noise time series with variance 2.

Let {xt|t T} be defined by the equation.

1 1 2 2 t t t t q t qx u u u u

Then {xt|t T} is called a Moving Average time series of order q. (denoted by MA(q))

qi

qih

hq

ihii

0

if0

2

The autocorrelation function for an MA(q) time series

The autocovariance function for an MA(q) time series

qi

qihh

q

ii

hq

ihii

0

if0 0

2

0

The mean value for an MA(q) time series

tE x

The autocorrelation function for an MA(q) time series

qi

qihh

q

ii

hq

ihii

0

if0 0

2

0

Comment

“cuts off” to zero after lag q.

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

q

The Autoregressive Time series of order p, AR(p)

where {ut|t T} is a white noise time series with variance 2.

Let {xt|t T} be defined by the equation.

2211 tptpttt uxxxx

Then {xt|t T} is called a Autoregressive time series of order p. (denoted by AR(p))

The mean value of a stationary AR(p) series

p

txE

211

The Autocovariance function (h) of a stationary AR(p) series

Satisfies the equations:

21 10 pp

101 1 pp

212 1 pp

and

011 ppp

phhh p 11 for h > p

Yule Walker Equations

2

1

01 1 p p

with

phhh p 11for h > p

111 1 pp

212 1 pp

111 ppp

The Autocorrelation function (h) of a stationary AR(p) series

Satisfies the equations:

and

or:

h

pp

hh

rc

rc

rch

111

22

11

and c1, c2, … , cp are determined by using the starting values of the sequence (h).

pp xxx 11

pr

x

r

x

r

x111

21

where r1, r2, … , rp are the roots of the polynomial

Conditions for stationarity

Autoregressive Time series of order p, AR(p)

The value of xt increases in magnitude and ut eventually becomes negligible.

i.e. 11 ttt uxx

If 1 = 1 and = 0.

The time series {xt|t T} satisfies the equation:

The time series {xt|t T} exhibits deterministic behaviour.

11 tt xx

For a AR(p) time series, consider the polynomial

pp xxx 11

pr

x

r

x

r

x111

21

with roots r1, r2 , … , rp

then {xt|t T} is stationary if |ri| > 1 for all i.

If |ri| < 1 for at least one i then {xt|t T} exhibits deterministic behaviour.

If |ri| ≥ 1 and |ri| = 1 for at least one i then {xt|t T} exhibits non-stationary random behaviour.

since:

h

pp

hh

rc

rc

rch

111

22

11

i.e. the autocorrelation function, (h), of a stationary AR(p) series “tails off” to zero.

lim 0h

h

and |r1 |>1, |r2 |>1, … , | rp | > 1 for a stationary AR(p) series then

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Special Cases: The AR(1) time

Let {xt|t T} be defined by the equation.

11 ttt uxx

Consider the polynomial

xx 11

1

1r

x

with root r1= 1/1

1. {xt|t T} is stationary if |r1| > 1 or |1| < 1 .

2. If |ri| < 1 or |1| > 1 then {xt|t T} exhibits deterministic behaviour.

3. If |ri| = 1 or |1| = 1 then {xt|t T} exhibits non-stationary random behaviour.

Special Cases: The AR(2) time

Let {xt|t T} be defined by the equation.

2211 tttt uxxx

Consider the polynomial

2211 xxx

21

11r

x

r

x

where r1 and r2 are the roots of (x)

1. {xt|t T} is stationary if |r1| > 1 and |r2| > 1 .

2. If |ri| < 1 or |1| > 1 then {xt|t T} exhibits deterministic behaviour.

3. If |ri| ≥ 1 for i = 1,2 and |ri| = 1 for at least on i then {xt|t T} exhibits non-stationary random behaviour.

This is true if 1+2 < 1 , 2 –1 < 1 and 2 > -1.These inequalities define a triangular region for 1 and 2.

Patterns of the ACF and PACF of AR(2) Time SeriesIn the shaded region the roots of the AR operator are complex

h kk

h kk

h kk

h kk

1

21

-1

2-2

III

IIIIV

2

The Mixed Autoregressive Moving Average Time Series of order p,q The ARMA(p,q) series

The Mixed Autoregressive Moving Average Time Series of order p, ARMA(p,q)

Let 1, 2, … p , 1, 2, … p , denote p + q +1 numbers (parameters).

Let {ut|t T} denote a white noise time series with variance 2.

– independent– mean 0, variance 2.

Let {xt|t T} be defined by the equation. 2211 ptpttt xxxx

Then {xt|t T} is called a Mixed Autoregressive- Moving Average time series - ARMA(p,q) series.

2211 qtqttt uuuu

Mean value, variance, autocovariance function,

autocorrelation function of anARMA(p,q) series

Similar to an AR(p) time series, for certain values of the parameters 1, …, p an ARMA(p,q) time series may not be stationary.

An ARMA(p,q) time series is stationary if the roots (r1, r2, … , rp ) of the polynomial

(x) = 1 – 1x – 2x2 - … - p xp

satisfy | ri| > 1 for all i.

Assume that the ARMA(p,q) time series {xt|t T} is stationary:

Let = E(xt). Then

2211 ptpttt xExExExE

21 p

1 21 p

1 2

1tp

E x

2211 qtqttt uEuEuEuE

0000 21 q

or

The Autocovariance function, (h), of a stationary

mixed autoregressive-moving average time series {xt|t T} be determined by the equation:

ptpttt xxxx 2211

Thus

p 211 now

11 ptptt xxx

qtqttt uuuu 2211

qtqttt uuuu 2211

Hence

tht xxEh

phtpht xxE 11

tqhtqhththt xuuuu 2211

tphtptht xxExxE 11

tqhtqthttht xuExuExuE 11

phh p 11

qhhh uxquxux 11

thtux xuEh where

ptptht xxuE 11

qtqttt uuuu 2211

pthtptht xuExuE 11

qthtqthttht uuEuuEuuE 11

phh uxpux 11

qhhh uuquuuu 11

thtux xuEh note

.0 if 0 where hxuEh thtux

.0 if 0

.0 if and

2

h

huuEh thtuu

We need to calculate:

quxuxux ,,1,0

20 ux

hux note phh uxpux 11

qhhh uuquuuu 11

.0 if 0 and hhux

.0 if 0

.0 if 2

h

hhuu

222201 uxux

222 012 uxuxux

22

22

2

222 etc

The autocovariance function (h) satisfies:

phhh p 11

qhhh uxquxux 11

For h = 0, 1. … , q:

pp 10 1 quxquxux 10 1

101 1 pp 101 quxqux

pqqq p 11 0uxq

for h > q:

phhh p 11

We then use the first (p + 1) equations to determine: (0), (1), (2), … , (p)

We use the subsequent equations to determine:(h) for h > p.

Example:The autocovariance function, (h), for an ARMA(1,1) time series:

11 hh 11 hh uxux

For h = 0, 1:

10 1 10 1 uxux

01 1 01 ux

for h > 1: 11 hh

or 10 1 2

1112

01 1 21

Substituting (0) into the second equation we get:

or

21

2111

211 11

22

1

1111

1

11

Substituting (1) into the first equation we get:

2111

222

1

11111 1

10

22

1

1112

12

111111

1

111

22

1

1121

1

21

for h > 1: 11 hh

22

1

111111 1

112

22

1

1111211 1

123

22

1

1111111 1

11

hhh

The Backshift Operator B

Consider the time series {xt : t T} and Let M denote the linear space spanned by the set of random variables {xt : t T}

(i.e. all linear combinations of elements of {xt : t T} and their limits in mean square).

M is a vector space

Let B be an operator on M defined by:

Bxt = xt-1.

B is called the backshift operator.

Note: 1.

2. We can also define the operator Bk withBkxt = B(B(...Bxt)) = xt-k.

3. The polynomial operator p(B) = c0I + c1B + c2B2 + ... + ckBk

can also be defined by the equation.p(B)xt = (c0I + c1B + c2B2 + ... + ckBk)xt . = c0Ixt + c1Bxt + c2B2xt + ... + ckBkxt

= c0xt + c1xt-1 + c2xt-2 + ... + ckxt-k

ktktt xcxcxcB

21 21

ktktt BxcBxcBxc 21 21

11211 21 ktktt xcxcxc

4. The power series operator p(B) = c0I + c1B + c2B2 + ...

can also be defined by the equation.p(B)xt = (c0I + c1B + c2B2 + ... )xt

= c0Ixt + c1Bxt + c2B2xt + ...

= c0xt + c1xt-1 + c2xt-2 + ...

5. If p(B) = c0I + c1B + c2B2 + ... and q(B) = b0I + b1B + b2B2 + ... are such that

p(B)q(B) = I i.e. p(B)q(B)xt = Ixt = xt than q(B) is denoted by [p(B)]-1.

Other operators closely related to B:

1. F = B-1 ,the forward shift operator, defined by Fxt = B-1xt = xt+1 and

2. = I - B ,the first difference operator, defined by xt = (I - B)xt = xt - xt-1 .

The Equation for a MA(q) time series

xt= 0ut + 1ut-1 +2ut-2 +... +qut-q + can be written

xt= (B) ut + where

(B) = 0I + 1B +2B2 +... +qBq

The Equation for a AR(p) time series

xt= 1xt-1 +2xt-2 +... +pxt-p + +ut

can be written

(B) xt= + ut

where

(B) = I - 1B - 2B2 -... - pBp

The Equation for a ARMA(p,q) time series

xt= 1xt-1 +2xt-2 +... +pxt-p + + ut + 1ut-1 +2ut-2 +... +qut-q

can be written

(B) xt= (B) ut + where

(B) = 0I + 1B +2B2 +... +qBq

and

(B) = I - 1B - 2B2 -... - pBp

Some comments about the Backshift operator B

1. It is a useful notational device, allowing us to write the equations for MA(q), AR(p) and ARMA(p, q) in a very compact form;

2. It is also useful for making certain computations related to the time series described above;

The partial autocorrelation function

A useful tool in time series analysis

The partial autocorrelation function

Recall that the autocorrelation function of an AR(p) process satisfies the equation:

x(h) = 1x(h-1) + 2x(h-2) + ... +px(h-p)

For 1 ≤ h ≤ p these equations (Yule-Walker) become:x(1) = 1 + 2x(1) + ... +px(p-1)

x(2) = 1x(1) + 2 + ... +px(p-2)

...

x(p) = 1x(p-1)+ 2x(p-2) + ... +p.

In matrix notation:

pxx

xx

xx

x

x

x

pp

p

p

p

2

1

121

211

111

2

1

These equations can be used to find 1, 2, … , p, if the time series is known to be AR(p) and the autocorrelation x(h)function is known.

In this case p

ppp ,,, 21

If the time series is not autoregressive the equations can still be used to solve for 1, 2, … , p, for any value of p 1.

are the values that minimizes the mean square error:

2

1

)()(...p

ixit

pixt xxEESM

121

211

111

21

211

111

)(

kk

k

k

kkk

xx

xx

xx

xxx

xx

xx

kkkk

Definition: The partial auto correlation function at lag k is defined to be:

Comment:

The partial auto correlation function, kk is determined from the auto correlation function, (h)

Some more comments:

1. The partial autocorrelation function at lag k, kk, can be interpreted as a corrected autocorrelation between xt and xt-k conditioning on the intervening variables xt-1, xt-2, ... ,xt-k+1 .

2. If the time series is an AR(p) time series than

kk = 0 for k > p

3. If the time series is an MA(q) time series than

x(h) = 0 for h > q

A General Recursive Formula for Autoregressive Parameters and the

Partial Autocorrelation function (PACF)

Letkk

kk

kkk ,,,, 321

denote the autoregressive parameters of order k satisfying the Yule Walker equations:

kkk

kkk13221

223121 kkk

kkk

kkk

kk

kk

kk 332211

Then it can be shown that:

k

jj

kj

k

jjk

kjk

kkkk

1

11

1,111

1

and

kjkjkkk

kj

kj ,,2 ,1 11,1

1

Proof:

The Yule Walker equations:

kkk

kkk13221

223121 kkk

kkk

kkk

kk

kk

kk 332211

In matrix form:

kkk

k

k

kk

k

k

22

1

21

2

1

1

1

1

kkk ρβΡ or

k

k

kk

k

k

k

kk

k

k

k

22

1

21

2

1

and ,

1

1

1

ρβΡ

kkk ρΡβ1

The equations for

1

2

11

12

11

1

1

1

1

1

kkk

k

k

kk

k

k

1,111

13

12

11 ,,,,

kkkk

kkk

11,1

11

1or

k

k

kk

k

k

kk

ρβ

Aρ

AρΡ

001

000

100

where

A and

113

12

11

11 ,,,, k

kkkkk β

The matrix A reverses order

kkkk

kk ρAρβΡ

1,11

1

The equations may be written

11,11

1

kkkkk βAρ

Multiplying the first equations by

kkkkkkk

k βρΡAρΡβ

11

1,11

1

1

kΡ

or kkkk

kk AρΡββ1

1,11

1

kkkk

k ρΡAβ1

1,1

k

kkk Aββ 1,1

Substituting this into the second equation

or

11,11,1

kkkk

kkkk AββAρ

kkk

kkkk Aβρβρ

11,1 1

and kk

kkk

kk

ρβ

Aβρ

1 1

1,1

Hence

k

jj

kj

k

jjk

kjk

kkkk

1

11

1,111

1

and

kjkjkkk

kj

kj ,,2 ,1 11,1

1

kkk

kk Aβββ 1,11

or

Some Examples

Example 1: MA(1) time seriesSuppose that {xt|t T} satisfies the following

equation:

xt = 12.0 + ut + 0.5 ut – 1

where {ut|t T} is white noise with = 1.1.Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.

SolutionNow {xt|t T} satisfies the following equation:

xt = 12.0 + ut + 0.5 ut – 1

Thus:

1. The mean of the series,

= 12.0

The autocovariance function for an MA(1) is

222 21

22

1 0.5 1.1 01 0 1.5125 0

1 0.5 1.1 1 0.605 1

0 1 0 1 0 1

hh h

h h h h

h h h

Thus:

2. The variance of the series,

(0) = 1.5125

and

3. The autocorrelation function is:

0.6051.5125

1 0 1 0

1 0.4 10

0 1 0 1

h hh

h h h

h h

( )

1 1 1

1 1 2

1 2

1 1 1

1 1 2

1 2 1

kkk k

k k k

k

k

k k

4. The partial auto correlation function at lag k is defined to be:

Thus (1)11 1

11 0.4

1

2 2(2)

22 2 2 2

1 1

1 2 2 1 0.4 0.16.19048

1 1 1 0.4 0.841 1

1 1

(3)33 3

1 1 1 1 0.4 0.4

1 1 2 0.4 1 0

2 1 3 0 0.4 0 0.0640.0941

1 .4 0 0.681 1 2

.4 1 .41 1 1

0 .4 12 1 1

(4)44 4

1 1 2 1 1 .4 0 .4

1 1 1 2 .4 1 .4 0

2 1 1 3 0 .4 1 0

3 2 1 4 0 0 .4 0 0.02560.0469

1 .4 0 0 0.54561 1 2 3

.4 1 .4 01 1 1 2

0 .4 1 .42 1 1 1

0 0 .4 13 2 1 1

(5)55 5

0.010240.0234

0.4368

66 77 88 990.0117, 0.0059, 0.0029, 0.0015

10,10 11,11 12,120.0007, 0.0004, 0.00029

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11

Graph: Partial Autocorrelation function kk

Exercise: Use the recursive method to calculate kk

111

1 1, 1

1

1

kk

k j k jjk

k k k kkj j

j

and

11, 1 1 1, 2, , k k k

j j k k k j j k

11 1,1 1we start with

Exercise: Use the recursive method to calculate kk

212 2 1 12 2,2 21

1 1

0.4.19048

1 1 0.4

and2 1 1

1 1 2.2 1 1j

1 .19048 0.4

1.19048 0.4 .0.476192

2 23 3 1 2 1 13 3,3 2 2

1 1 2 2

0.0941, etc1

Example 2: AR(2) time series

Suppose that {xt|t T} satisfies the following equation:

xt = 0.4 xt – 1 + 0.1 xt – 2 + 1.2 + ut

where {ut|t T} is white noise with = 2.1.Is the time series stationary?Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.

1. The mean of the series

1 2

1.22.4

1 1 0.4 0.1

3. The autocorrelation function.Satisfies the Yule Walker equations

1 1 2 1 1

2 1 1 2 1

1 0.4 0.1

1 0.4 0.1

1 1 2 1 1then 0.4 0.1

where h h h h h

h h

hence

1

2

0.40.4444

0.90.4

0.4 0.1 2.7780.9

1 1 2 1 1then 0.4 0.1

where h h h h h

h h

h 0 1 2 3 4 5 6

h 1.0000 0.4444 0.2778 0.1556 0.0900 0.0516 0.0296

h 7 8 9 10 11 12 13

h 0.0170 0.0098 0.0056 0.0032 0.0018 0.0011 0.0006

2. the variance of the series

2 2

1 1 2 1

2.10 5.7522

1 1 0.4 0.4444 0.1 0.2778

4. The partial autocorrelation function.

1,1 1 0.4444

1

1 22,2

1

1

1 1 0.4444

0.4444 .27780.1000

1 0.44441

0.4444 11

1 1

1 2

2 1 33,3

1 2

1 1

2 1

1 1 0.4444 0.4444

1 0.4444 1 0.2778

0.2778 0.4444 0.15560

1 1 0.4444 0.2778

1 0.4444 1 0.4444

1 0.2778 0.4444 1

,in fact 0 for 3k k k

The partial autocorrelation function of an AR(p) time series “cuts off” after p.

Example 3: ARMA(1, 2) time series

Suppose that {xt|t T} satisfies the following equation:

xt = 0.4 xt – 1 + 3.2 + ut + 0.3 ut – 1 + 0.2 ut – 1

where {ut|t T} is white noise with = 1.6.Is the time series stationary?Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.