dates for term tests 1.friday, february 07 2.friday, march 07 3.friday, march 28
TRANSCRIPT
Dates for term tests
1. Friday, February 07
2. Friday, March 07
3. Friday, March 28
The Moving Average Time series of order q, MA(q)
where {ut|t T} denote a white noise time series with variance 2.
Let {xt|t T} be defined by the equation.
1 1 2 2 t t t t q t qx u u u u
Then {xt|t T} is called a Moving Average time series of order q. (denoted by MA(q))
qi
qih
hq
ihii
0
if0
2
The autocorrelation function for an MA(q) time series
The autocovariance function for an MA(q) time series
qi
qihh
q
ii
hq
ihii
0
if0 0
2
0
The mean value for an MA(q) time series
tE x
The autocorrelation function for an MA(q) time series
qi
qihh
q
ii
hq
ihii
0
if0 0
2
0
Comment
“cuts off” to zero after lag q.
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
q
The Autoregressive Time series of order p, AR(p)
where {ut|t T} is a white noise time series with variance 2.
Let {xt|t T} be defined by the equation.
2211 tptpttt uxxxx
Then {xt|t T} is called a Autoregressive time series of order p. (denoted by AR(p))
The mean value of a stationary AR(p) series
p
txE
211
The Autocovariance function (h) of a stationary AR(p) series
Satisfies the equations:
21 10 pp
101 1 pp
212 1 pp
and
011 ppp
phhh p 11 for h > p
Yule Walker Equations
2
1
01 1 p p
with
phhh p 11for h > p
111 1 pp
212 1 pp
111 ppp
The Autocorrelation function (h) of a stationary AR(p) series
Satisfies the equations:
and
or:
h
pp
hh
rc
rc
rch
111
22
11
and c1, c2, … , cp are determined by using the starting values of the sequence (h).
pp xxx 11
pr
x
r
x
r
x111
21
where r1, r2, … , rp are the roots of the polynomial
Conditions for stationarity
Autoregressive Time series of order p, AR(p)
For a AR(p) time series, consider the polynomial
pp xxx 11
pr
x
r
x
r
x111
21
with roots r1, r2 , … , rp
then {xt|t T} is stationary if |ri| > 1 for all i.
If |ri| < 1 for at least one i then {xt|t T} exhibits deterministic behaviour.
If |ri| ≥ 1 and |ri| = 1 for at least one i then {xt|t T} exhibits non-stationary random behaviour.
since:
h
pp
hh
rc
rc
rch
111
22
11
i.e. the autocorrelation function, (h), of a stationary AR(p) series “tails off” to zero.
lim 0h
h
and |r1 |>1, |r2 |>1, … , | rp | > 1 for a stationary AR(p) series then
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Special Cases: The AR(1) time
Let {xt|t T} be defined by the equation.
11 ttt uxx
Consider the polynomial
xx 11
1
1r
x
with root r1= 1/1
1. {xt|t T} is stationary if |r1| > 1 or |1| < 1 .
2. If |ri| < 1 or |1| > 1 then {xt|t T} exhibits deterministic behaviour.
3. If |ri| = 1 or |1| = 1 then {xt|t T} exhibits non-stationary random behaviour.
Special Cases: The AR(2) time
Let {xt|t T} be defined by the equation.
2211 tttt uxxx
Consider the polynomial
2211 xxx
21
11r
x
r
x
where r1 and r2 are the roots of (x)
1. {xt|t T} is stationary if |r1| > 1 and |r2| > 1 .
2. If |ri| < 1 or |1| > 1 then {xt|t T} exhibits deterministic behaviour.
3. If |ri| ≤ 1 for i = 1,2 and |ri| = 1 for at least on i then {xt|t T} exhibits non-stationary random behaviour.
This is true if 1+2 < 1 , 2 –1 < 1 and 2 > -1.
These inequalities define a triangular region for 1 and 2.
Patterns of the ACF and PACF of AR(2) Time SeriesIn the shaded region the roots of the AR operator are complex
h kk
h kk
h kk
h kk
1
21
-1
2-2
III
IIIIV
2
The Mixed Autoregressive Moving Average Time Series of order p,q The ARMA(p,q) series
The Mixed Autoregressive Moving Average Time Series of order p, ARMA(p,q)
Let 1, 2, … p , 1, 2, … p , denote p + q +1 numbers (parameters).
Let {ut|t T} denote a white noise time series with variance 2.
– independent– mean 0, variance 2.
Let {xt|t T} be defined by the equation. 2211 ptpttt xxxx
Then {xt|t T} is called a Mixed Autoregressive- Moving Average time series - ARMA(p,q) series.
2211 qtqttt uuuu
Mean value, variance, autocovariance function,
autocorrelation function of anARMA(p,q) series
Similar to an AR(p) time series, for certain values of the parameters 1, …, p an ARMA(p,q) time series may not be stationary.
An ARMA(p,q) time series is stationary if the roots (r1, r2, … , rp ) of the polynomial
(x) = 1 – 1x – 2x2 - … - p xp
satisfy | ri| > 1 for all i.
Assume that the ARMA(p,q) time series {xt|t T} is stationary:
Let = E(xt). Then
2211 ptpttt xExExExE
21 p
1 21 p
1 2
1tp
E x
2211 qtqttt uEuEuEuE
0000 21 q
or
The Autocovariance function, (h), of a stationary mixed autoregressive-moving average time series {xt|t T} be determined by the equation:
ptpttt xxxx 2211
Thus
p 211 now
11 ptptt xxx
qtqttt uuuu 2211
qtqttt uuuu 2211
Hence
tht xxEh
phtpht xxE 11
tqhtqhththt xuuuu 2211
tphtptht xxExxE 11
tqhtqthttht xuExuExuE 11
phh p 11
qhhh uxquxux 11
thtux xuEh where
ptptht xxuE 11
qtqttt uuuu 2211
pthtptht xuExuE 11
qthtqthttht uuEuuEuuE 11
phh uxpux 11
qhhh uuquuuu 11
thtux xuEh note
.0 if 0 where hxuEh thtux
.0 if 0
.0 if and
2
h
huuEh thtuu
We need to calculate:
quxuxux ,,1,0
20 ux
hux note phh uxpux 11
qhhh uuquuuu 11
.0 if 0 and hhux
.0 if 0
.0 if 2
h
hhuu
222201 uxux
222 012 uxuxux
22
22
2
222
21 2 3 33 2 1 0ux ux ux ux
21 2 2
2 22 3 3
2 21 2 2 2 3 3
h ux(h)
0
-1
-2
-3
2
2
22 2
2 21 2 2 2 3 3
The autocovariance function (h) satisfies:
phhh p 11
qhhh uxquxux 11
For h = 0, 1. … , q:
pp 10 1 quxquxux 10 1
101 1 pp 101 quxqux
pqqq p 11 0uxq
for h > q:
phhh p 11
We then use the first (p + 1) equations to determine: (0), (1), (2), … , (p)
We use the subsequent equations to determine:(h) for h > p.
Example:The autocovariance function, (h), for an ARMA(1,1) time series:
11 hh 11 hh uxux
For h = 0, 1:
10 1 10 1 uxux
01 1 01 ux
for h > 1: 11 hh
or 10 1 2
1112
01 1 21
Substituting (0) into the second equation we get:
or
21
2111
211 11
22
1
1111
1
11
Substituting (1) into the first equation we get:
2111
222
1
11111 1
10
22
1
1112
12
111111
1
111
22
1
1121
1
21
for h > 1: 11 hh
22
1
111111 1
112
22
1
1111211 1
123
22
1
1111111 1
11
hhh
The Backshift Operator B
Consider the time series {xt : t T} and Let M denote the linear space spanned by the set of random variables {xt : t T}
(i.e. all linear combinations of elements of {xt : t T} and their limits in mean square).
M is a vector space
Let B be an operator on M defined by:
Bxt = xt-1.
B is called the backshift operator.
Note: 1.
2. We can also define the operator Bk withBkxt = B(B(...Bxt)) = xt-k.
3. The polynomial operator p(B) = c0I + c1B + c2B2 + ... + ckBk
can also be defined by the equation.p(B)xt = (c0I + c1B + c2B2 + ... + ckBk)xt . = c0Ixt + c1Bxt + c2B2xt + ... + ckBkxt
= c0xt + c1xt-1 + c2xt-2 + ... + ckxt-k
ktktt xcxcxcB
21 21
ktktt BxcBxcBxc 21 21
11211 21 ktktt xcxcxc
4. The power series operator p(B) = c0I + c1B + c2B2 + ...
can also be defined by the equation.p(B)xt = (c0I + c1B + c2B2 + ... )xt
= c0Ixt + c1Bxt + c2B2xt + ...
= c0xt + c1xt-1 + c2xt-2 + ...
5. If p(B) = c0I + c1B + c2B2 + ... and q(B) = b0I + b1B + b2B2 + ... are such that
p(B)q(B) = I i.e. p(B)q(B)xt = Ixt = xt than q(B) is denoted by [p(B)]-1.
Other operators closely related to B:
1. F = B-1 ,the forward shift operator, defined by Fxt = B-1xt = xt+1 and
2. = I - B ,the first difference operator, defined by xt = (I - B)xt = xt - xt-1 .
The Equation for a MA(q) time series
xt= 0ut + 1ut-1 +2ut-2 +... +qut-q + can be written
xt= (B) ut + where
(B) = 0I + 1B +2B2 +... +qBq
The Equation for a AR(p) time series
xt= 1xt-1 +2xt-2 +... +pxt-p + +ut
can be written
(B) xt= + ut
where
(B) = I - 1B - 2B2 -... - pBp
The Equation for a ARMA(p,q) time series
xt= 1xt-1 +2xt-2 +... +pxt-p + + ut + 1ut-1 +2ut-2 +... +qut-q
can be written
(B) xt= (B) ut + where
(B) = 0I + 1B +2B2 +... +qBq
and
(B) = I - 1B - 2B2 -... - pBp
Some comments about the Backshift operator B
1. It is a useful notational device, allowing us to write the equations for MA(q), AR(p) and ARMA(p, q) in a very compact form;
2. It is also useful for making certain computations related to the time series described above;
The partial autocorrelation function
A useful tool in time series analysis
The partial autocorrelation function
Recall that the autocorrelation function of an AR(p) process satisfies the equation:
x(h) = 1x(h-1) + 2x(h-2) + ... +px(h-p)
For 1 ≤ h ≤ p these equations (Yule-Walker) become:x(1) = 1 + 2x(1) + ... +px(p-1)
x(2) = 1x(1) + 2 + ... +px(p-2)
...
x(p) = 1x(p-1)+ 2x(p-2) + ... +p.
In matrix notation:
pxx
xx
xx
x
x
x
pp
p
p
p
2
1
121
211
111
2
1
These equations can be used to find 1, 2, … , p, if the time series is known to be AR(p) and the autocorrelation x(h)function is known.
In this case p
ppp ,,, 21
If the time series is not autoregressive the equations can still be used to solve for 1, 2, … , p, for any value of p ≥ 1.
are the values that minimizes the mean square error:
2
1
)()(...p
ixit
pixt xxEESM
121
211
111
21
211
111
)(
kk
k
k
kkk
xx
xx
xx
xxx
xx
xx
kkkk
Definition: The partial auto correlation function at lag k is defined to be:
Using Cramer’s Rule
Comment:
The partial auto correlation function, kk is determined from the auto correlation function, (h)
The partial auto correlation function at lag k, kk is the last auto-regressive parameter, . if the series was assumed to be an AR(k) series.
If the series is an AR(p) series then
An AR(p) series is also an AR(k) series with k > p with the auto regressive parameters zero after p.
kk
, 0 for kk k k k p
Some more comments:
1. The partial autocorrelation function at lag k, kk, can be interpreted as a corrected autocorrelation between xt and xt-k conditioning on the intervening variables xt-1, xt-2, ... ,xt-k+1 .
2. If the time series is an AR(p) time series than
kk = 0 for k > p
3. If the time series is an MA(q) time series than
x(h) = 0 for h > q
A General Recursive Formula for Autoregressive Parameters and the
Partial Autocorrelation function (PACF)
Letkk
kk
kkk ,,,, 321
denote the autoregressive parameters of order k satisfying the Yule Walker equations:
kkk
kkk13221
223121 kkk
kkk
kkk
kk
kk
kk 332211
Then it can be shown that:
k
jj
kj
k
jjk
kjk
kkkk
1
11
1,111
1
and
kjkjkkk
kj
kj ,,2 ,1 11,1
1
Proof:
The Yule Walker equations:
kkk
kkk13221
223121 kkk
kkk
kkk
kk
kk
kk 332211
In matrix form:
kkk
k
k
kk
k
k
22
1
21
2
1
1
1
1
kkk ρβΡ or
k
k
kk
k
k
k
kk
k
k
k
22
1
21
2
1
and ,
1
1
1
ρβΡ
kkk ρΡβ1
The equations for
1
2
11
12
11
1
1
1
1
1
kkk
k
k
kk
k
k
1,111
13
12
11 ,,,,
kkkk
kkk
11,1
11
1or
k
k
kk
k
k
kk
ρβ
Aρ
AρΡ
001
000
100
where
A and
113
12
11
11 ,,,, k
kkkkk β
The matrix A reverses order
kkkk
kk ρAρβΡ
1,11
1
The equations may be written
11,11
1
kkkkk βAρ
Multiplying the first equations by
kkkkkkk
k βρΡAρΡβ
11
1,11
1
1
kΡ
or kkkk
kk AρΡββ1
1,11
1
kkkk
k ρΡAβ1
1,1
k
kkk Aββ 1,1
Substituting this into the second equation
or
11,11,1
kkkk
kkkk AββAρ
kkk
kkkk Aβρβρ
11,1 1
and kk
kkk
kk
ρβ
Aβρ
1 1
1,1
Hence
k
jj
kj
k
jjk
kjk
kkkk
1
11
1,111
1
and
kjkjkkk
kj
kj ,,2 ,1 11,1
1
kkk
kk Aβββ 1,11
or
Some Examples
Example 1: MA(1) time seriesSuppose that {xt|t T} satisfies the following
equation:
xt = 12.0 + ut + 0.5 ut – 1
where {ut|t T} is white noise with = 1.1.Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.
SolutionNow {xt|t T} satisfies the following equation:
xt = 12.0 + ut + 0.5 ut – 1
Thus:
1. The mean of the series,
= 12.0
The autocovariance function for an MA(1) is
222 21
22
1 0.5 1.1 01 0 1.5125 0
1 0.5 1.1 1 0.605 1
0 1 0 1 0 1
hh h
h h h h
h h h
Thus:
2. The variance of the series,
(0) = 1.5125
and
3. The autocorrelation function is:
0.6051.5125
1 0 1 0
1 0.4 10
0 1 0 1
h hh
h h h
h h
( )
1 1 1
1 1 2
1 2
1 1 1
1 1 2
1 2 1
kkk k
k k k
k
k
k k
4. The partial auto correlation function at lag k is defined to be:
Thus (1)11 1
11 0.4
1
2 2(2)
22 2 2 2
1 1
1 2 2 1 0.4 0.16.19048
1 1 1 0.4 0.841 1
1 1
(3)33 3
1 1 1 1 0.4 0.4
1 1 2 0.4 1 0
2 1 3 0 0.4 0 0.0640.0941
1 .4 0 0.681 1 2
.4 1 .41 1 1
0 .4 12 1 1
(4)44 4
1 1 2 1 1 .4 0 .4
1 1 1 2 .4 1 .4 0
2 1 1 3 0 .4 1 0
3 2 1 4 0 0 .4 0 0.02560.0469
1 .4 0 0 0.54561 1 2 3
.4 1 .4 01 1 1 2
0 .4 1 .42 1 1 1
0 0 .4 13 2 1 1
(5)55 5
0.010240.0234
0.4368
66 77 88 990.0117, 0.0059, 0.0029, 0.0015
10,10 11,11 12,120.0007, 0.0004, 0.00029
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11
Graph: Partial Autocorrelation function kk
Exercise: Use the recursive method to calculate kk
111
1 1, 1
1
1
kk
k j k jjk
k k k kkj j
j
and
11, 1 1 1, 2, , k k k
j j k k k j j k
11 1,1 1we start with
Exercise: Use the recursive method to calculate kk
212 2 1 12 2,2 21
1 1
0.4.19048
1 1 0.4
and2 1 1
1 1 2.2 1 1j
1 .19048 0.4
1.19048 0.4 .0.476192
2 23 3 2 2 1 13 3,3 2 2
1 1 2 2
0.0941, etc1
Example 2: AR(2) time series
Suppose that {xt|t T} satisfies the following equation:
xt = 0.4 xt – 1 + 0.1 xt – 2 + 1.2 + ut
where {ut|t T} is white noise with = 2.1.Is the time series stationary?Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.
1. The mean of the series
1 2
1.22.4
1 1 0.4 0.1
3. The autocorrelation function.Satisfies the Yule Walker equations
1 1 2 1 1
2 1 1 2 1
1 0.4 0.1
1 0.4 0.1
1 1 2 1 1then 0.4 0.1
where h h h h h
h h
hence
1
2
0.40.4444
0.90.4
0.4 0.1 0.27780.9
1 1 2 1 1then 0.4 0.1
where h h h h h
h h
h 0 1 2 3 4 5 6
h 1.0000 0.4444 0.2778 0.1556 0.0900 0.0516 0.0296
h 7 8 9 10 11 12 13
h 0.0170 0.0098 0.0056 0.0032 0.0018 0.0011 0.0006
2. the variance of the series
2 2
1 1 2 1
2.10 5.7522
1 1 0.4 0.4444 0.1 0.2778
4. The partial autocorrelation function.
1,1 1 0.4444
1
1 22,2
1
1
1 1 0.4444
0.4444 .27780.1000
1 0.44441
0.4444 11
1 1
1 2
2 1 33,3
1 2
1 1
2 1
1 1 0.4444 0.4444
1 0.4444 1 0.2778
0.2778 0.4444 0.15560
1 1 0.4444 0.2778
1 0.4444 1 0.4444
1 0.2778 0.4444 1
,in fact 0 for 3k k k
The partial autocorrelation function of an AR(p) time series “cuts off” after p.
Example 3: ARMA(1, 2) time series
Suppose that {xt|t T} satisfies the following equation:
xt = 0.4 xt – 1 + 3.2 + ut + 0.3 ut – 1 + 0.2 ut – 2
where {ut|t T} is white noise with = 1.6.Is the time series stationary?Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.
xt = 0.4 xt – 1 + 3.2 + ut + 0.3 ut – 1 + 0.2 ut – 1
white noise std. dev,. = 1.6.
Is the time series stationary?
(x) = 1 – 1x = 1 – 0.4x has root r1 =1/0.4 =2.5
Since |r1| > 1, the time series is stationary
Find:
1. The mean of the series.
1
3.2 3.2 32 165.333
1 1 0.4 .6 6 3
The autocovariance function (h) satisfies:
1 1 21 1 2ux ux uxh h h h h
For h = 0, 1, 2
for h > q: 1 1h h
i.e.
0.4 1 0.3 1 0.2 2ux ux uxh h h h h
For h = 0, 1, 2
for h > q: 0.4 1h h
0 0.4 1 0 0.3 1 0.2 2ux ux ux
etc.where
1 0.4 0 0.3 0 0.2 1ux ux
2 0.4 1 0.2 0ux
3 0.4 2 4 0.4 3 5 0.4 4
2 20 1.6 2.56,ux
22 22ux
20.4 0.7 0.2 1.6 1.2288
2 21 .4 .3 1.6 1.792,ux
0 1 10.4 2.56 0.3 1.792 0.2 1.288 0.4 3.34336
We use the first two equations to find 0 and 1
Then we use the third equation to find 2
1 0 00.4 0.3 2.56 0.2 1.792 0.4 1.1264
2 1 10.4 0.2 2.56 0.4 0.512
1then 0.4 for 3.h h h
0 00.4 0.4 1.1264 3.34336
201 0.4 0.4 1.1264 3.34336
0 2
0.4 1.1264 3.343364.516571
1 0.4
1 00.4 1.1264 0.4 4.516751 1.1264 2.933029
2 10.4 0.512 0.4 2.933029 0.512 1.68521
The autocovariance, autocorrelation functions
h (h ) (h )
0 4.517 1.0001 2.933 0.6492 1.685 0.3733 0.674 0.1494 0.270 0.0605 0.108 0.0246 0.043 0.0107 0.017 0.004
Spectral Theory for a stationary time series