stad57 time series analysissdamouras/courses/stad57h3_w13/lecture 21... · time domain in time...
TRANSCRIPT
![Page 1: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/1.jpg)
STAD57 Time Series
Analysis
Lecture 21
1
![Page 2: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/2.jpg)
GARCH Model Extensions
There are many variations on GARCH models
geared towards different data characteristics
E.g. Integrated / Seasonal / Fractionally Integrated
GARCH model, Exponential GARCH model, etc
One common extension is to model the
conditional mean of a series with ARMA, and
its conditional variance with GARCH
E.g. ARMA with GARCH(r,s) errors
2 2 2 2
0 1 1
, ~ (0,1)( ) ( ) , where
iid
t t t t
p qt t
t j t j k t kj k
Z NB X B Z
Z
![Page 3: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/3.jpg)
Example
Fit ARMA(1,2) w/ GARCH(3,2) errors in R
3
Estimate Std. Error t value Pr(>|t|)
mu -1.244e-04 6.574e-05 -1.893 0.058365 .
ar1 7.513e-01 8.771e-02 8.565 < 2e-16 ***
ma1 -7.223e-01 9.055e-02 -7.977 1.55e-15 ***
ma2 -8.961e-02 3.135e-02 -2.859 0.004255 **
omega 3.388e-06 1.373e-06 2.467 0.013626 *
alpha1 1.000e-08 2.645e-02 0.000 1.000000
alpha2 6.892e-02 3.126e-02 2.205 0.027480 *
alpha3 1.362e-01 3.704e-02 3.678 0.000235 ***
beta1 1.082e-01 1.637e-01 0.661 0.508556
beta2 6.601e-01 1.542e-01 4.281 1.86e-05 ***
AIC BIC SIC HQIC
-6.566089 -6.516973 -6.566287 -6.547421
garchFit(~arma(1,2) + garch(3,2),spr)
model specification
![Page 4: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/4.jpg)
Example
4
Forecasts for ARMA(1,2) w/ GARCH(3,2) errors
![Page 5: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/5.jpg)
Time Series Analysis: Time &
Frequency Domain
In general, there are two complementary
ways to analyze time series:
Time Domain Analysis
Describing how process evolves in time
Frequency Domain Analysis
Describing process as (random) combination of
periodic functions with different frequencies
Two ways are equivalent for stationary series,
but each is more appropriate for different tasks
5
![Page 6: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/6.jpg)
Time Domain
In time domain, goal is to model evolution of
series in time, as function of its past
Already seen SARIMA models, for which function
F() is linear. More generally, can extended
method to nonlinear functions
Best suited for modeling auto-/cross-
correlations, and especially for forecasting
Commonly used for statistical, business, financial,
and economic applications 6
1 1, , , , ,t t t p t t qX F X X W W
![Page 7: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/7.jpg)
Frequency Domain
In frequency domain, goal is to model series
as random composition of periodic functions
with different frequencies
Best suited for modeling series with strong
periodic behavior over several frequencies
E.g. radio signals, vibrations, sounds etc.
Commonly used in engineering & physical sciences 7
( ), is RV
where: is fn w/ freq.
k
t k kkk k
UX U f t
f
![Page 8: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/8.jpg)
Frequency Domain
Purpose of frequency domain analysis is to
decompose process (a.k.a. signal) into its
constituent frequencies
Given , want to find frequencies
ωk (of functions fk) and the magnitude of their
associated RV’s Uk
E.g. Voice recognition application: want to map
speech (sound signals) into words, by analyzing
each syllable’s characteristic frequencies
8
( )t k kkX U f t
![Page 9: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/9.jpg)
Example
Speech signal (syllable aaa…hhh sampled at
10,000 points/sec)
9 0 200 400 600 800 1000
01
00
02
00
03
00
04
00
0
![Page 10: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/10.jpg)
Example
Note: complicated ACF/PACF, would require very
large SARIMA model in time domain analysis 10
0 50 100 150 200 250 300
-0.5
0.0
0.5
1.0
0 50 100 150 200 250 300
-0.5
0.0
0.5
Pa
rtia
l A
CF
ACF PACF
![Page 11: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/11.jpg)
Periodic Signals
The building blocks of frequency domain are
the trigonometric functions (sines & cosines)
Periodic functions in given by:
Look at trigonometric fn’s because any periodic fn
can be written as linear combination of sines &
cosines w/ different frequencies (Fourier series) 11
( )
( )
1 2
cos( ) cos( ) cos( ) sin( ) sin( )
( ) cos(2 )
cos(2 ) sin(2 ), for
k k k
k k k k
a b a b a b
f t U t
U t U t
( )k kkU f t
1
2
cos( )
sin( )
k k
k k
U U
U U
![Page 12: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/12.jpg)
Periodic Signals
For function define:
ω = frequency (# cycles per unit time)
1/ω = period (# unit times per cycle)
U = amplitude (max height of cosine)
φ = phase (start of cosine function)
In terms of , we have:
amplitude:
phase:
12
cos(2 )U t
1 2cos(2 ) sin(2 )U t U t 2 2
1 2U U U
1
2 1tan /U U
1
2
cos( )
sin( )from
U U
U U
![Page 13: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/13.jpg)
0.0 0.5 1.0
-2-1
01
2
Example
13
t
( ) 2 cos(2 2 ) 2 sin(2 2 )f t t t
phase φ
amplitude period
![Page 14: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/14.jpg)
Periodic Series
Consider process defined as:
Process is a sum of fixed periodic functions at
frequencies ωk, with random independent weights
The variances σk2 control the magnitude of the
weights Uk1 & Uk2 at frequencies ωk
14
1 21cos(2 ) sin(2 )
q
t k k k kkX U t U t
1 2
2
1 2 1 2[ ] [ ] 0 & [ ] [ ]
where, at frequency , RV's , are independent
with
k k k
k k k k k
U U
E U E U Var U Var U
![Page 15: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/15.jpg)
Example
Series with q=1, ω1=.06, σk2=1
15 0 20 40 60 80 100
-0.5
0.0
0.5
![Page 16: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/16.jpg)
Example
Series with q=2,
16
2
1 1
22 2
.06 1,
.1 2
0 20 40 60 80 100
-2-1
01
2
t
0 20 40 60 80 100
-3-1
13
Xt (sum)
ω2, σ22
ω1, σ12
![Page 17: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/17.jpg)
0 20 40 60 80 100
-2-1
01
2
t
0 20 40 60 80 100
-4-2
02
Example
Series with q=3,
17
2
11
2
2 2
23 3
1.06
.1 , 2
.4 .5
ω2, σ2
2
ω1, σ12
Xt (sum)
ω3, σ32
![Page 18: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/18.jpg)
Autocovariance
For
with independent 0-mean & σk2-var Uk1,Uk2,
the autocovariance function is given by:
Proof:
18
1 21cos(2 ) sin(2 )
q
t k k k kkX U t U t
2 2
1 1( ) cos(2 ) (0)
q q
k k kk kh h
![Page 19: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/19.jpg)
19
![Page 20: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/20.jpg)
Example
Series with q=3,
Find theoretical ACF of series
20
2
11
2
2 2
23 3
1.06
.1 , 2
.4 .5
![Page 21: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/21.jpg)
0 10 20 30 40
-0.5
0.0
0.5
1.0
Series X
0 10 20 30 40
-0.5
0.0
0.5
1.0
AC
F/A
CF
[1]
0 20 40 60 80 100
-4-2
02
Example (cont’d)
21
Series:
Sample
ACF
Theoretical
ACF
![Page 22: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/22.jpg)
Periodic Series
Can actually represent any 0-mean stationary
process as superposition of (possibly infinite)
trigonometric functions with different ωk & σk2
E.g. To generate White Noise {Wt}, t=1,…,n, with
Var[Wt]=σw2, use q=n random components with:
Obviously:
Can also show that
22
22& , 1, ,w
k k
kk n
n n
2 2
1(0)
n
k wk
( ) 0, 1, , 1h h n
![Page 23: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/23.jpg)
Example
23
0 50 100 150 200
-2-1
01
23
2 1& , 1, ,k k
kk n
n n
generated
WN(0,1)
series
(n=200)
0 5 10 15 20
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
5 10 15 20
-0.1
5-0
.05
0.0
50
.15
Pa
rtia
l A
CF
ACF PACF
![Page 24: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/24.jpg)
Scaled Periodogram
Given series data (X1,…,Xn), how can we find
frequency domain representation of {Xt}?
Don’t know ωk or σk2 beforehand
Idea: look at large set of frequencies ωj and
try to estimate their corresponding σj2
Since we only have data for t=1,…,n, look at
periods of 1/ωj=1,…,n → ωj=1/n,…,n/n=1
For each frequency ωj=j/n, j=1,…,n, estimate Uj1,
Uj2 using regression, and σj2 by their squares’ sum
24
![Page 25: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/25.jpg)
Scaled Periodogram
For fixed j in (1,…,n), we have
Regression model becomes:
This is like bivariate regression without intercept
From orthogonality of cos & sin, estimates are
given by correlations of Xt with cos & sin:
25
/j j n
1 2( / )cos(2 / ) ( / )sin(2 / )tX j n tj n j n tj n
11 12
1
12 12
1
cos(2 / ) 2ˆ ( / ) cos(2 / )cos (2 / )
sin(2 / ) 2ˆ ( / ) sin(2 / )sin (2 / )
n
ntttn t
t
n
ntttn t
t
X tj nj n X tj n
ntj n
X tj nj n X tj n
ntj n
0, / 2,j n n
![Page 26: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/26.jpg)
Scaled Periodogram
For j=n/2:
For j=0,n:
Scaled Periodogram, P(j/n), is defined as:
P(j/n) measures magnitude of sin/cos coefficients
Can be seen as estimate of 2σj2 at freq. ωj=j/n
P(j/n) is symmetric around P(1/2)
26
1 21
ˆ ˆ(1/ 2) ( 1) / , & (1/ 2) 0n t
ttX n
1 1 2 21
ˆ ˆ ˆ ˆ(0) (1) / , & (0) (1) 0n
ttX n
2 2
1 2ˆ ˆ( / ) ( / ) ( / ), 1, ,P j n j n j n j n
/ 1 / , 0,1, ,P j n P j n j n
![Page 27: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/27.jpg)
Periodogram
There is an easy way to calculate P(j/n)
without running all n/2 regressions, using the
Discrete Fourier Transform (DFT):
d(j/n) is complex-valued weighted average of {Xt}
27
( )
1
1 1
/ exp 2 / /
cos 2 / sin 2 / /
n
tt
n n
t tt t
d j n X i tj n n
X tj n i X tj n n
( )exp cos( ) sin( )ix x i x
![Page 28: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/28.jpg)
Periodogram
Periodogram, I(j/n), is defined as squared
modulus of DFT coefficients:
Can calculate P(j/n) as:
Fast Fourier Transform (FFT) algorithm offers
highly efficient way of calculating d(j/n) →
I(j/n) → P(j/n) 28
2
2 2
1 1
( / ) ( / )
1 1cos 2 / sin 2 /
n n
t tt t
I j n d j n
X tj n X tj nn n
4
/ /P j n I j nn
![Page 29: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/29.jpg)
Example
Series with q=3,
29
2
11
2
2 2
23 3
1.06
.1 , 2
.4 .5
Scaled
Periodogram
0.0 0.2 0.4 0.6 0.8 1.0
01
23
4
![Page 30: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/30.jpg)
Example
To calculate P(j/n) in R, use fft() function
30
> n=length(X)
> P = abs( 2 * fft(X) / n )^2
> Fr = (0:(n-1))/n
> plot(Fr, P, type='o')
series length
Fast Fourier transform
Scaled periodogram P(j/n)
corresponding frequencies (ωj=j/n)
![Page 31: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/31.jpg)
Example
P(j/n) for speech signal
31 0 2000 4000 6000 8000 10000
02
00
00
40
00
06
00
00
Frequncy in Hz (cycles/sec)
human voice frequency
ranges ~ 300-3500Hz
![Page 32: STAD57 Time Series Analysissdamouras/courses/STAD57H3_W13/Lecture 21... · Time Domain In time domain, goal is to model evolution of series in time, as function of its past Already](https://reader034.vdocuments.site/reader034/viewer/2022042301/5ecc1f57283e586628533515/html5/thumbnails/32.jpg)
Example
White Noise series
32 0.0 0.2 0.4 0.6 0.8 1.0
0.0
00
0.0
05
0.0
10
0.0
15
0.0
20
P(j/n) almost
uniform across
all frequencies