Linear Filters

denote a bivariate time series with zero mean.

Let

Tt

y

x

t

t :

The time series {yt : t T} is said to be constructed from {xt : t T} by means of a Linear Filter.

sstst xay

Suppose that the time series {yt : t T} is constructed as follows:

X t YtOutput Process

Input Process

LinearFilter

Schematic Diagram of a Linear Filter

hyy

s sxxss sshaa

'' '

dfeae xxs

sis

hi

2

dfAe xxhi

2

The autocovariance function of the filtered series

Thus the spectral density of the time series

{yt : t T} is:

xxxx

s

sisyy fAfeaf

22

Comment A:

is called the Transfer function of the linear filter.

is called the Gain of the filter while

is called the Phase Shift of the filter.

s

siseaA

A

Aarg

Also httxy yxEh

sxxs sha

dfAe xxhi

Thus cross spectrum of the bivariate time series

Tt

y

x

t

t :

is:

xx

sxx

sisxy fAfeaf

Definition:

= Squared Coherency function

jjii

ij

ij ff

fK

2

2

12 ijKNote:

Comment B:

= Squared Coherency function.

yyxx

xy

xy ff

fK

2

2

1 2

22

xxxx

xx

fAf

fA

if {yt : t T} is constructed from {xt : t T} by means of a linear filter

Linear Filterswith additive noise at the output

denote a bivariate time series with zero mean.

Let

t = ..., -2, -1, 0, 1, 2, ...

Tt

y

x

t

t :

Suppose that the time series {yt : t T} is constructed as follows:

ts

stst vxay

The noise {vt : t T} is independent of the series {xt : t T} (may be white)

X t YtOutput Process

Input Process

LinearFilter

Schematic Diagram of a Linear Filter with Noise

N t Additive Noise at Output

t

httyy yyEh

hsshaa vvs s

xxss

'' '

dfedfeae vvhi

xxs

sis

hi

2

The autocovariance function of the filtered series with added noise

continuing

hyy

dffAe vvxxhi 2

s

siseaA where

Thus the spectral density of the time series

{yt : t T} is:

vvxxyy ffAf 2

Also httxy yxEh

sxxs sha

dfea xxs

shis

dfAe xxhi

Thus cross spectrum of the bivariate time series

Tt

y

x

t

t :

is:

xx

sxx

sisxy fAfeaf

Thus

= Squared Coherency function.

yyxx

xy

xy ff

fK

2

2

vvxxxx

xx

ffAf

fA

2

22

1

1

1

1

2

xx

vv

fA

f

Noise to Signal Ratio

Box-Jenkins Parametric Modelling of a Linear Filter

Consider the Linear Filter of the time series {Xt : t T}:

ts

stst XBaXaY

0

0s

ss BaBawhere

0s

sis

i eaeaA and

= the Transfer function of the filter.

{at : t T} is called the impulse response function of the filter since if X0 =1and Xt = 0 for t ≠ 0, then :

ts

stst aXaY

0

for t T

Linear

Filter

Xt at

Also Note:

01

01

ssts

sststtt XaXaYYY

0

1

ss stst XXa

0ssts Xa

Hence {Yt} and {Xt} are related by the same Linear Filter.

ts

stst XBaXaY

0

Definition The Linear Filter

is said to be stable if :

0s

ss BaBa

converges for all |B| ≤1.

Discrete Dynamic Models:

Many physical systems whose output is represented by Y(t) are modeled by the following differential equation:

tYdt

d

dt

d

dt

dI

r

r

r

...

2

2

21

tX

dt

d

dt

d

dt

dI

s

s

s...2

2

21

Where X(t) is the forcing function.

If X and Y are measured at discrete times this equation can be replaced by:

tr

r YI ...221

bts

s XI ...221

where = I-B denotes the differencing operator.

This equation can in turn be represented with the operator B.

tr

r YBBBI ...221

bts

s XBBBI ...2210

tbs

s XBBBBI ...2210

or tt XBYB

sbs

bbb BBBBB ...22

110

where

This equation can also be written in the form as a Linear filter as

Stability: It can easily be shown that this filter is stable if the roots of (x) = 0 lie outside the unit circle.

ttt XBaXBBY 1

Determining the Impulse Response function from the Parameters of

the Filter:

Now

BBBa 1

or BBaB

Hence

...... 2210

221 BaBaIaBBBI r

r

sbs

bbb BBBB ...22

110

Equating coefficients results in the following conclusions:

aj = 0 for j < b.

aj - 1aj-1 - 2aj-2-...- r aj-r= j

or aj = 1j-1 + 2aj-2+...+ r aj-r+ j

for b ≤ j ≤ b+s.

and aj - 1aj-1 - 2aj-2-...- r aj-r= 0

or aj = 1aj-1 + 2aj-2+...+ r aj-r for j > b+s.

Thus the coefficients of the transfer function,

a0, a1, a2,... ,

satisfy the following properties

1) b zeroes a0, a1, a2,..., ab-1

2) No pattern for the next s-r+1 values

ab, ab+1, ab+2,..., ab+s-r

3) The remaining values

ab+s-r+1, ab+s-r+2, ab+s-r+3,...

follow the pattern of an rth order difference equation

aj = 1aj-1 + 2aj-2+...+ r aj-r

Example r =1, s=2, b=3, 1 = a0 = a1 = a2 = 0

a3 = a2 + 0 = 0

a4 = a3 + 1 = 0 + 1

a5 = a4 + 2

= [0 + 1] + 2

= 2w0 + 1 + 2

aj = aj-1 for j ≥ 6.

Transfer function {at}

0 1 2 3 5 6 7 8

Exponentially decaying Transfer Function

s-r+1b zeroes

as

Identification of the Box-Jenkins Transfer

Model with r=2

Recall the solution to the second order difference equation

aj = 1aj-1 + 2aj-2

follows the following patterns:

1) Mixture of exponentials if the roots of

1 - 1x - 2x2 = 0 are real.2) Damped Cosine wave if the roots to

1 - 1x - 2x2 = 0 are complex.

These are the patterns of the Impulse Response function one looks for when identifying b,r and s.

Estimation of the Impulse Response function, aj

(without pre-whitening).

Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation:

ts

stst NXaY

0

Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with

{Xt : t T}. Then

t

sshtsthttXY NXaXEYXEh

0

Suppose that for s > M, as = 0. Then a0, a1, ... ,aM can be found solving the following equations:

tts

shtts NXEXXEa

0

0s

xxs sha

If the Cross autocovariance function, XY(h), and the Autocovariance function, XX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function

01

1011

100

10

10

10

xxMxxxxxy

xxMxxxxxy

xxMxxxxxy

aMaMaM

Maaa

Maaa

Maaa ˆ,ˆˆ ,1,0

In matrix notation this set of linear equations can be written:

Mxxxxxx

xxxxxx

xxxxxx

xy

xy

xy

a

a

a

MM

M

M

M

1

0

01

101

10

1

0

If the Cross autocovariance function, XY(h), and the Autocovariance function, XX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function

0ˆ1ˆˆ

1ˆ0ˆ1ˆ1

ˆ1ˆ0ˆ0

10

10

10

xxMxxxxxy

xxMxxxxxy

xxMxxxxxy

CaMCaMCaMC

MCaCaCaC

MCaCaCaC

Maaa ˆ,ˆˆ ,1,0

Estimation of the Impulse Response function, aj

(with pre-whitening).

Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation:

ts

stst NXaY

0

Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with

{Xt : t T}.

In addition assume that {Xt : t T}, the weakly stationary time series has been identified as an ARMA(p,q) series, estimated and found to satisfy the following equation:

(B)Xt = (B)ut

where {ut : t T} is a white noise time series. Then

[(B)]-1(B)Xt = ut

transforms the Time series {Xt : t T} into the white noise time series{ut : t T}.

This process is called Pre-whitening the Input series.

Applying this transformation to the Output series {Yt : t T} yeilds:

tYBB )()]([ 1

ts

sts NBBXBBa )()]([)()]([ 1

0

1

ts

stst nuay

0

or

tt YBBy )()]([ 1

tt NBBn )()]([ 1

where

and

In this case the equations for the impulse response function - a0, a1, ... ,aM - become (assuming that for s > M, as = 0):

0

01

00

1

0

uuMuy

uuuy

uuuy

aM

a

a

0

ˆ and 0

or uu

uyk

uu

uyk C

kCa

ka

Summary

Identification and Estimation of

Box-Jenkins transfer model

To identify the series we need to determine

b, r and s.

The first step is to compute

1. the ACF’s and the cross CF’s

Cxx(h) and Cxy(h)

2. Estimate the impulse response function using

0ˆ1ˆˆ

1ˆ0ˆ1ˆ1

ˆ1ˆ0ˆ0

10

10

10

xxMxxxxxy

xxMxxxxxy

xxMxxxxxy

CaMCaMCaMC

MCaCaCaC

MCaCaCaC

The Impulse response function {at}

0 1 2 3 5 6 7 8

Exponentially decaying Transfer Function

s-r+1b zeroes

as

b s- r + 1 Pattern of an rth order difference equation

3. Determine the value of b, r and s from the pattern of the impulse response function

3. Determine preliminary estimates of the Box-Jenkins transfer function parameters using:i. for j > b+s. .

aj = 1aj-1 + 2aj-2+...+ r aj-r

ii. for b ≤ j ≤ b+s

aj = 1j-1 + 2aj-2+...+ r aj-r+ j

4. Determine preliminary estimates of the ARMA parameters of the input time series {xt}

5. Determine preliminary estimates of the ARIMA parameters of the noise time series {t}

1 bt t ty B B B x

bt t tB y B B x B

1 1t t r t ry y y

1 1t b t b s t b sx x x 1 1 1 1t t t

1 1 1 1t t t r t r t b t b s t b sy y y x x x 1 1 1 1t t

Maximum Likelihood estimation of the parameters of

the Box-Jenkins Transfer function model

The Box- Jenkins model is written

1 bt tB B B x

t ta B x 0

t s t s ts

y a x

21 2where ... r

rB I B B B 2

0 1 2and ... ssB I B B B

The parameters of the model are:

1 2 0 1 2 , ,..., and , , ,...,r s In addition

1. the ARMA parameters of the input series {xt}

2. The ARIMA parameters of the noise series {t}

The model for the noise {t}series can be written

1

t tB B u

21 2where ... p

pB I B B B

21 2and ... q

qB I B B B

Given starting values for {yt}, {xt}, and tu

0 0 0, and x y u

and the parameters of the transfer function model and the noise model , and δ ω β α

We can calculate successively: 0 0 0, , , , ,t tu u δ ω β α x y u

The maximum likelihood estimates are the values ˆ ˆˆ ˆ, and δ ω β α

that minimize: 2

0 , , tt

S u δ ω β α

Fitting a transfer function model

Example: Monthly Sales (Y) and Monthly Advertising expenditures

Mon Adver Sales Mon Adver Sales Mon Adver Sales Mon Adver Sales Mon Adver Sales

1 78.54 1315 51 102.18 1763 101 147.35 2143 151 81.03 1400 201 120.08 916

2 134.82 1304 52 87.95 1840 102 118.26 1909 152 84.61 1348 202 116.6 1024

3 106.38 1507 53 81.08 1679 103 112.15 2106 153 120.19 1658 203 121.53 1441

4 105.31 1580 54 93.78 1756 104 123.8 1997 154 65 1641 204 92.48 1601

5 122.18 1662 55 127.12 1766 105 96.46 2452 155 89.86 1512 205 145.82 1996

6 120.54 2044 56 126.21 1730 106 147.96 2298 156 91.53 1466 206 117.38 2109

7 153.02 1940 57 118.74 1498 107 104.87 2089 157 89 1782 207 135.48 2158

8 154.58 2015 58 109.54 1597 108 109.97 2200 158 80.6 1597 208 119.35 1854

9 163.61 2080 59 134.59 2019 109 112.95 1924 159 89.29 1553 209 138.61 2260

10 146.6 2070 60 107.67 2261 110 75.54 2354 160 58.2 1576 210 122.75 2215

11 157.49 2544 61 119.26 2110 111 94.58 2029 161 71.36 1603 211 102.4 2312

12 150.65 2598 62 115.73 2074 112 91.04 2205 162 71.83 1556 212 115.84 2248

13 155.82 2913 63 107.76 2219 113 59.15 1900 163 73.04 1554 213 119.65 2394

14 134.51 2758 64 102.35 2172 114 67.83 1647 164 70.84 1275 214 106.22 2316

15 145.09 2855 65 87 2136 115 93.02 1671 165 82.82 1214 215 103.06 1962

16 93.54 2671 66 92.78 1978 116 78.14 1623 166 60.55 1263 216 97 2104

17 189.59 2755 67 83.73 2020 117 70.19 1295 167 89.51 1344 217 78.26 2018

18 135.28 2632 68 64.12 1977 118 91.23 1312 168 52.83 1270 218 78.22 2014

19 165.66 2559 69 85.27 1671 119 61.48 1372 169 21.57 1437 219 56.17 1876

20 164 2105 70 111.13 1631 120 60.77 1437 170 34.75 1229 220 42.87 1861

21 119.11 2849 71 115.95 1606 121 69.03 1446 171 56.88 1342 221 58.93 1600

22 156.23 2559 72 107.55 1303 122 50.16 1418 172 36.69 1254 222 59.8 1413

23 133.32 2787 73 107.99 1449 123 48.42 1340 173 65.53 727 223 66.4 1185

24 134.64 2998 74 104.28 1721 124 65.86 1069 174 45.23 641 224 82.62 977

25 100.12 2459 75 103.91 2016 125 64.19 1248 175 47.97 804 225 53.57 905

26 112.62 2659 76 76.51 1896 126 40.46 1035 176 53.66 735 226 64.77 1103

27 113.36 2454 77 79.89 2023 127 48.73 976 177 22.47 1011 227 73.24 1131

28 104.49 2538 78 44.64 1863 128 45.87 1049 178 77.5 937 228 78.88 1334

The Data

Using SAS

Available in the Arts computer lab

The Start up window for SAS

To import data - Choose File -> Import data

The following window appears

Browse for the file to be imported

Identify the file in SAS

The next screen (not important) click Finish

The finishing screen

• You can now run analysis by typing code into the Edit window or selecting the analysis form the menu

• To fit a transfer function model we need to identify the model– Determine the order of differencing to achieve

Stationarity– Determine the value of b, r and s.

• To determine the degree of differencing we look at ACF’s and PACF’s for various order of differencing

To produce the ACF, PACF – type the following commands into the Editor window- Press Run button

• To identify the transfer function model we need to estimate the impulse response function using:

0ˆ1ˆˆ

1ˆ0ˆ1ˆ1

ˆ1ˆ0ˆ0

10

10

10

xxMxxxxxy

xxMxxxxxy

xxMxxxxxy

CaMCaMCaMC

MCaCaCaC

MCaCaCaC

• For this we need the ACF of the input series and the cross ACF of the input with the output

To produce the Cross correlation function – type the following commands into the Editor window

• the impulse response function using can be determined using some other package (i.e. Excel)

-2

0

2

4

6

8

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

b = 4

r,s = 1

To Estimate the transfer function model – type the following commands into the Editor window

To estimate the following model

1 bt t ty B B B x

21 2where ... r

rB I B B B

20 1 2and ... s

sB I B B B

Use

input=( b $ ( -lags ) / ( -lags) x)

In SAS

20 1 2and ... s

sB B B B

The Output

The Output