introduction to time series analysishalweb.uc3m.es/esp/personal/personas/jtena/eng/... ·...

96
Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B [email protected] 1 Introduction to time series (2013 )

Upload: others

Post on 19-Sep-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

Introduction to Time Series

Analysis

Juan de Dios Tena Horrillo

Office. 10.1.02B

[email protected]

1

Intro

ductio

n to

time series (2

013 )

Page 2: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

OUTLINE

Chapter 1. Univariate ARIMA models.

Chapter 2. Model fitting and checking.

Chapter 3. Prediction and model selection.

Chapter 4. Outliers and influential

observations.

Chapter 5. Vector Autoregressive and Vector

Error Correction Models.

Chapter 6. Nonlinear Time Series Modelling.

2

Intro

ductio

n to

time series (2

013 )

Page 3: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

TEXTBOOKS

Peña, D., Tiao, G.C. and Tsay, R.S. (2000). A Course in Time Series Analysis. Wiley & sons.

Hamilton, J.D. (1994). Time Series Analysis. Princeton University Press.

Box, G.E.P., Jenkins, G.M. and Reinsel, G. (1994). Time Series Analysis: Forecasting and Control, 3rd. Ed. Prentice-Hall, Englewood Cliffs, NJ.

Enders, W. (2004). Applied Econometric Time Series . Wiley & sons.

Lütkepohl, H. and Krätzig, M. (2004). Applied Time Series Econometrics. Cambridge University Press.

Franses, P.H. and van Dick, D. (2000). Non-linear time series models in empirical finance. Cambridge University Press.

3

Intro

ductio

n to

time series (2

013 )

Page 4: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

GRADING

Final exam (70%).

Class participation and empirical project (30%).

4

Intro

ductio

n to

time series (2

013 )

Page 6: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION

Time series: sequence of observations taken at

regular intervals of time

Data in bussines, engineering, enviroment,

medicine, etc.

6

Intro

ductio

n to

time series (2

013 )

Page 7: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

7

Intro

ductio

n to

time series (2

013 )

SUICIDES IN SPAIN, TOTAL NUMBER

Source: INE

100

150

200

250

300

350

95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10

SUICIDES

Page 8: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

8

Intro

ductio

n to

time series (2

013 )

CO2 EMISSIONS IN THE USA

Source: World Databank

2,800,000

3,200,000

3,600,000

4,000,000

4,400,000

4,800,000

5,200,000

5,600,000

6,000,000

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005

CO2

Page 9: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

9

Intro

ductio

n to

time series (2

013 )

INDUSTRIAL PRODUCTION INDEX (SPAIN)

Source: ine

30

40

50

60

70

80

90

100

110

120

1975 1980 1985 1990 1995 2000 2005 2010

IPI

Page 10: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

10

Intro

ductio

n to

time series (2

013 )

Page 11: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

11

Intro

ductio

n to

time series (2

013 )

OVERNIGHT STAYS IN SPANISH HOTELS

Source: ine

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

7,000,000

8,000,000

9,000,000

10,000,000

11,000,000

99 00 01 02 03 04 05 06 07 08 09 10 11 12

TURISTS

Page 12: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

Intro

ductio

n to

time series (2

013 )

12

CONSUMER PRICE INDEX IN GERMANY AND SPAIN

Source: Eurostats

70

80

90

100

110

120

130

1996 1998 2000 2002 2004 2006 2008 2010 2012

GERMAN_CPI SPANISH_CPI

Page 13: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

Intro

ductio

n to

time series (2

013 )

13

CONSUMER PRICE INDEX FOR FOOD AND NON-ALCOHOLIC

BEVERAGES IN GERMANY AND SPAIN

Source: Eurostats

7 0

8 0

9 0

1 0 0

1 1 0

1 2 0

1 9 9 6 1 9 9 8 2 0 0 0 2 0 0 2 2 0 0 4 2 0 0 6 2 0 0 8 2 0 1 0 2 0 1 2

S P A N IS H _ F O O D G E R M A N _ F O O D

Page 14: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

Intro

ductio

n to

time series (2

013 )

14

Source: Instituto Flores de Lemus

-3

-2

-1

1

2

3

4

5

6

2008 2009 2010 2011 2012 2013 2014

YEAR-ON-YEAR RATE OF INFLATION IN SPAIN AND CONTRIBUTIONS OF MAIN COMPONENTS

ENERGÍA ALIMENTOS NO ELABORADOS

INFLACIÓN SUBYACENTE IPC TOTAL

Page 15: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

Intro

ductio

n to

time series (2

013 )

15

Source: Instituto Flores de Lemus

-2.5

-1.5

-0.5

0.5

1.5

2.5

3.5

4.5

5.5

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

INFLATION IN MADRIDAnnual rates of growth

80% 60% 40% 20%Confidence intervals:

Average inflation(1998-2011): 2,7%

Page 16: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

16

Intro

ductio

n to

time series (2

013 )

Page 17: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

17

Intro

ductio

n to

time series (2

013 )

Page 18: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION.

Knowledge of the dynamic structure will help to

produce accurate forecasts of future observations

and design optimal control schemes.

18

Intro

ductio

n to

time series (2

013 )

Page 19: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

MOTIVATION

Main objectives

Understanding the dynamic structure of the

observations in a single series.

Ascertain the leading, lagging and feedback

relationships among several variables.

Analyzing the impact of other variables that

can be used for policy decisions.

What to do when there are breaks in the

evolution of the series. 19

Intro

ductio

n to

time series (2

013 )

Page 20: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

Chapter 1

Univariate ARIMA Models

Recommended readings: chapters 1 to 3 of Peña et al. (2000)

20

Intro

ductio

n to

time series (2

013 )

Page 21: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

CHAPTER 1. CONTENTS.

1.1. Introduction.

Definitions, examples,

1.2. Properties of univariate ARMA.

ARMA weights.

Stationarity and covariance structure.

Autocorrelation function.

Partial autocorrelación function.

1.3. Model especification.

1.4. Examples.

21

Intro

ductio

n to

time series (2

013 )

Page 22: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

22

Intro

ductio

n to

time series (2

013 )

Stochastic Process. Real-valued random

variable 𝑍𝑡 that follows a distribution 𝑓𝑡(𝑍𝑡) .

The T-dimensional variable 𝑍𝑡1, 𝑍𝑡2

, … , 𝑍𝑡𝑇

will have a joint distribution that depends on

𝑡1, … , 𝑡𝑇 .

Page 23: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

23

Intro

ductio

n to

time series (2

013 )

Time series 𝑧𝑡1, 𝑧𝑡2

, … , 𝑧𝑡𝑇 will denote a

particular realization of the stochastic process.

To simplify notation 𝑧1 , 𝑧2 , … , 𝑧𝑇 .

Page 24: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

24

Intro

ductio

n to

time series (2

013 )

Assumptions

1. The process is stationary.

2. The joint distribution of 𝑧1 , 𝑧2 , … , 𝑧𝑇 is a

multivariate normal distribution.

Page 25: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Implications of stationarity

The joint distribution remains constant over time

It is then true that

25

Intro

ductio

n to

time series (2

013 ) ),...,,(),...,,( 2121 kTkkT zzzfzzzf

;ztEz zt VVz

Page 26: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Stationarity implies a constant mean and bounded

deviations from it.

Strong requirement, few actual economic series will

satisfy it.

26

Intro

ductio

n to

time series (2

013 )

Page 27: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

27

Intro

ductio

n to

time series (2

013 )

Page 28: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

28

Intro

ductio

n to

time series (2

013 )

Page 29: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

29

Intro

ductio

n to

time series (2

013 )

Page 30: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Transformations to achieve stationarity

Constant variance: log/level plus outlier correction.

Stationary in mean: differencing.

30

Intro

ductio

n to

time series (2

013 )

Page 31: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

31

Intro

ductio

n to

time series (2

013 )

Page 32: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

32

Intro

ductio

n to

time series (2

013 )

Page 33: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Differencing

Let B be the backward operator

We shall use the operators:

Regular difference

Seasonal difference

33

Intro

ductio

n to

time series (2

013 )

jttj zzB

B 1s

s B 1

Page 34: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

34

Intro

ductio

n to

time series (2

013 )

If 𝑧𝑡 is a deterministic trend (𝑧𝑡 = 𝑎 + 𝑏𝑡)

∇𝑧𝑡 = 𝑏

∇2𝑧𝑡 = 0

where

∇2𝑧𝑡 = ∇(∇𝑧𝑡)

In general, ∇d will reduce a polynomial of degree d

to a constant.

Page 35: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

35

Intro

ductio

n to

time series (2

013 )

Example: for a quarterly series 𝑧𝑡 , ∇4𝑧𝑡 = 𝑧𝑡 − 𝑧𝑡−4

will cancel a constant, but it will also cancel other

deterministic periodic functions.

Page 36: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

36

Intro

ductio

n to

time series (2

013 )

Homogeneous difference equation

∇4𝑧𝑡 = (1 − 𝐵4)𝑧𝑡 = 𝑧𝑡 − 𝑧𝑡−4

Characteristic equation: 𝑟4 − 1 = 0

Solution is given by 𝑟 = 14

(four roots on the unit

circle)

𝑟1 = 1, 𝑟2 = −1, 𝑟3 = 𝑖, 𝑟4 = −𝑖

Two real roots and two complex conjugates with

modulus 1 and frequency 𝜔 = 𝜋/2.

Page 37: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Therefore,

1. One in the zero frequency (trend)

2. One in the twice-a-year seasonality

3. Associated with once-a-year seasonality

37

Intro

ductio

n to

time series (2

013 )

)1)(1)(1( 24 BBB

2/

Page 38: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

38

Intro

ductio

n to

time series (2

013 )

To see this just recall that the complementary function to

the difference equation 𝑧𝑡 = 𝑏𝑧𝑡−1 + 𝑐𝑧𝑡−2 will eventually be

of the form:

𝑧𝑡 = 𝑟𝑡 𝐴 cos 𝜃𝑡 + 𝐵 sin 𝜃𝑡

where 𝑟 is a positive constant and 𝜃 is an angle measured

in radians. 𝐴 and 𝐵 are arbitrary constants to enable the

solution to satisfy any starting point of 𝑧𝑡 .

Hence, when the two solutions of the characteristic

equations are 𝑖 and – 𝑖, sin 𝜃 = 1 and 𝜃 =𝜋

2.

Page 39: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

UNIVARIATE ARMA MODELS

“Building block” is the white noise process

Implications:

39

Intro

ductio

n to

time series (2

013 )

),0( 2Niidat

0,...),,/()( 321 ttttt aaaaEaE

0),(),( jttjtt aaCovaaE

221 ,...),/(),( atttjtt aaaVaraaVar

Page 40: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

40

Intro

ductio

n to

time series (2

013 )

General ARMA(p,q) processes:

𝜙 𝐵 𝑧𝑡 = 𝑐 + 𝜃(𝐵)𝑎𝑡

𝑧𝑡 is the observable time series.

𝑎𝑡 is sequence of white noise.

𝑐 is the constant term.

Page 41: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Autoregressive polynomial

Moving-average polynomial

The AR and MA polynomials are assumed to have no

common factor

41

Intro

ductio

n to

time series (2

013 )

ppBBBB ...1)( 2

21

qqBBBB ...1)( 2

21

Page 42: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

42

Intro

ductio

n to

time series (2

013 )

Stationarity implies that all zeros of 𝜙 𝐵 are

restricted to lie outside the unit circle and in this case:

𝑐 = 1 − 𝜙1 − ⋯− 𝜙𝑝 𝜇

where 𝜇 is the mean of the series.

Overall behavior of the series remains the same

over time.

Page 43: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

43

Intro

ductio

n to

time series (2

013 )

Overall behavior of the series remains the same

over time.

In real world, however, time series data often

exhibits a drifting behavior. Nonstationarity can be

modeled allowing some of the zeroes in 𝜙 𝐵 to be

equal to one. Thus,

𝜙 𝐵 (1 − 𝐵𝑑)𝑧𝑡 = 𝑐 + 𝜃(𝐵)𝑎𝑡

This is known as the ARIMA (p,d,q) model.

Page 44: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Some special cases of ARIMA(p,d,q):

AR(1)

MA(1)

ARMA(1,1)

IMA(1,1)

ARIMA(0,1,1)(0,1,1)

44

Intro

ductio

n to

time series (2

013 )

Page 45: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

45

Intro

ductio

n to

time series (2

013 )

AR(1) to MA(∞) by recursive substitution.

𝑧𝑡 = 𝜙𝑧𝑡−1 + 𝑎𝑡

𝑧𝑡 = 𝜙 𝜙𝑧𝑡−2 + 𝑎𝑡−1 + 𝑎𝑡 = 𝜙2𝑧𝑡−2 + 𝜙𝑎𝑡−1 + 𝑎𝑡

𝑧𝑡 = 𝜙𝑘𝑧𝑡−𝑘 + 𝜙𝑘−1𝑎𝑡−𝑘+1 + ⋯ + 𝜙2𝑎𝑡−2 + 𝜙𝑎𝑡−1 + 𝑎𝑡

Page 46: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

46

Intro

ductio

n to

time series (2

013 )

If 𝜙 < 1, then

𝑧𝑡 = 𝜙𝑗𝑎𝑡−𝑗

𝑗 =0

which is a MA(∞)

Page 47: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

INTRODUCTION

Some considerations:

ARMA models are not unique (variety of possible

representations.

Parsimony is always a desirable property.

AR representations are easiest to estimate (OLS) and

to be interpretated.

47

Intro

ductio

n to

time series (2

013 )

Page 48: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

For simplicity, we will assume zero mean and a starting point m. The ARIMA (p,d,q)

Can be rewritten as:

where,

are r=max(p,q) initial values.

48

Intro

ductio

n to

time series (2

013 )

tt aBzB )()(

aDzD

)',....,( tm zzz )',...,( tm aaa

)'0,...,0,,...,( 1 rmm ww

1,..., rmm ww

Page 49: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

49

Intro

ductio

n to

time series (2

013 )

1.........

.........

......

.........

...

1

1

1

p

p

D

1.........

.........

......

.........

...

1

1

1

q

q

D

)1)(1( mtmt

)1)(1( mtmt

Page 50: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

ARMA weights

Where,

50

Intro

ductio

n to

time series (2

013 )

11 DaDDz

1...

...

1

1

11

mt

DD

Page 51: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

The -weights can be obtained by equating

coefficients of powers of B from the relations:

where

51

Intro

ductio

n to

time series (2

013 )

)()()( BBB

...1)( 221 BBB

Page 52: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

The ARIMA (p,d,q) model can then be rewritten in the

MA form as:

52

Intro

ductio

n to

time series (2

013 )

mt

hhthtt aaz

1

Page 53: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

In the same way,

Where,

53

Intro

ductio

n to

time series (2

013 )

aDzDD 11

1...

...

1

1

11

mt

DD

Page 54: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

With

The AR form of the model is then,

54

Intro

ductio

n to

time series (2

013 )

).()()( BBB

t

mt

hhtht azz

1

Page 55: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

These two expressions are of fundamental importance in understanding the nature of the model.

The MA form with the weights, shows how the observation is affected by current and past shocks or innovations.

The AR form with the weigths, indicates how the observation is related to its own past values.

55

Intro

ductio

n to

time series (2

013 )

tz

Page 56: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

Examples : AR(1) model

MA representation:

AR representation:

56

Intro

ductio

n to

time series (2

013 )

....22

11 tttt aaaz

ttt azz 1

Page 57: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

Examples : MA(1) model

MA representation:

AR representation:

57

Intro

ductio

n to

time series (2

013 ) tjt

jtt azzz .......1

1 ttt aaz

Page 58: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

Examples : IMA(1,1) model

MA representation:

AR representation:

58

Intro

ductio

n to

time series (2

013 )

ttttt azzzz ...)1()1()1( 32

21

....))(1( 21 tttt aaaz

Page 59: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

STATIONARITY CONDITION

Consider,

Since the innovations are assumed normally

distributed, it follows that the observations are also

normally distributed.

It is seen that, if in the characteristic equation:

59

Intro

ductio

n to

time series (2

013 )

mt

hhthtt aaz

1

kpp

kk rArA 0011 ...

Page 60: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

STATIONARITY CONDITION

(Where the A’s denote polinomials in k and the r’s are

the distinct zeroes of )

Then as we have that

60

Intro

ductio

n to

time series (2

013 )

)(B

,1jrpoj ,...,1

, mt

,0)( tzE

0

2),cov(h

khhaktt zz

Page 61: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

STATIONARITY CONDITION

So that will be stationary in this asymptotic sense.

This is the stationarity condition of an ARMA model,

and is equivalent to require that the zeroes of

are lying outside the unit circle.

61

Intro

ductio

n to

time series (2

013 )

tz

)(B

Page 62: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

LAG K AUTOCOVARIANCE

Let us denote,

For an alternative expressions of the autocovariance

function in terms of the ARMA parameters,

62

Intro

ductio

n to

time series (2

013 )

)(),cov(),cov()( kzzzzk kttktt

)...( 11 ptpttkt zzzz

)...( 11 qtqttkt aaaz

Page 63: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

LAG K AUTOCOVARIANCE

By taking expectations on both sides and using the

MA form, we obtain, for

Where,

63

Intro

ductio

n to

time series (2

013 )

k

p

hh ghkk

1

)()(

qk

kg

kq

hkhha

k

0

,...2,1,00

2

,0k

Page 64: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

AUTOCORRELATION FUNCTION

The autocorrelation function is defined as

By substitution of , we obtain that

64

Intro

ductio

n to

time series (2

013 )

,...2,1,0,)0(

)()( k

kk

)(k

0)0(

)()(1

kg

hkkp

h

kh

Page 65: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

AUTOCORRELATION FUNCTION

When , in a MA(q) model, then,

That is, the autocorrelation function cuts off after lag

q.

65

Intro

ductio

n to

time series (2

013 )

0)( B

qk

qkk qq

,0

,)...1()(

1221

Page 66: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

Intuition

AR(1):

AR(2):

66

Intro

ductio

n to

time series (2

013 )

tttt zzzz 123

tttt zzzz 123

Page 67: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

The autocorrelation function only takes into account

that and are related in both cases.

But if we want to measure the direct relationship

(without the intermediate ), we found that it is

zero for the AR(1) and different from zero for the

AR(2) model

67

Intro

ductio

n to

time series (2

013 )

tz 2tz

1tz

Page 68: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

The partial autocorrelation function is, therefore, a

measure of the linear relation among observations k-

periods apart, independently of the intermediate

values.

68

Intro

ductio

n to

time series (2

013 )

Page 69: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

Consider first an stationary AR(p) model. The

autoregressive coefficients are related to the

autocorrelations by the Yule-Walker equations,

where, is the matrix,

69

Intro

ductio

n to

time series (2

013 )

pppG

pG )( pxp

Page 70: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

70

Intro

ductio

n to

time series (2

013 )

1)1(...)2()1(

)1(1)2(

)2(1)1(

)1()2(...)1(1

pp

p

p

pp

Gp

Page 71: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

Regarding this as system of p equations and p

unknowns (the coefficients), the solution is, for p>1,

the ratio of two determinants

71

Intro

ductio

n to

time series (2

013 )

ppp GH /

Page 72: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

72

Intro

ductio

n to

time series (2

013 )

)()1(...)2()1(

)1(1)2(

)2(1)1(

)1()2(...)1(1

ppp

pp

p

H p

where

Page 73: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

73

Intro

ductio

n to

time series (2

013 )

•A pxp matrix, and for p=1. This

leads to define, for any stationary model

•Which is known as the Partial Autocorrelation

Function.

)1( p

1/

1)(

kGH

kk

kk

k

Page 74: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PARTIAL AUTOCORRELATION FUNCTION

74

Intro

ductio

n to

time series (2

013 )

It has the property that, for a stationary AR(p)

model,

In other words, vanishes for k>p when the

model is AR(p).

pk

pkk

k0

k

Page 75: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

PROPERTIES

75

Intro

ductio

n to

time series (2

013 )

SAF PAF

AR(P) slow decay

towards zero

p different

from zero

MA(q) q different

from zero

slow decay

towards zero

ARMA(p,q) Slow decay

towards zero

Slow decay

towards zero

Page 76: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

AR(1) model

Model:

Variance:

76

Intro

ductio

n to

time series (2

013 )

ttt azz 1

2222azz

2

22

1

a

z

Page 77: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

Autocovariance function:

77

Intro

ductio

n to

time series (2

013 )

1)1(

11

0

)(2

2

2

kk

k

k

k a

z

Page 78: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

Autocorrelation function:

Partial autocorrelation:

78

Intro

ductio

n to

time series (2

013 )

1

1

01

)(

k

k

k

kk

)1()1(

Page 79: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

AR(2) model

Model:

Variance:

79

Intro

ductio

n to

time series (2

013 )

tttt azzz 2211

21

22

2

2

22

)1(1

1

a

z

Page 80: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

Autocorrelation function:

80

Intro

ductio

n to

time series (2

013 )

21

11

)(

2

21

2

2

1

k

k

k

22211 kkkk

Page 81: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

MA(1) model

Model:

Variance:

81

Intro

ductio

n to

time series (2

013 )

1 ttt aaz

2222aaz

)1( 222 az

Page 82: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

Autocovariance function:

82

Intro

ductio

n to

time series (2

013 )

10

1

0

)( 2

2

k

k

k

k a

z

Page 83: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

EXAMPLES

Autocorrelation function:

83

Intro

ductio

n to

time series (2

013 )

10

11

01

)(2

k

k

k

k

Page 84: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

The class of ARMA(p,q) models is extensive.

Guidelines are needed in selecting a member of the

class to represent the time series data at hand

84

Intro

ductio

n to

time series (2

013 )

Page 85: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Box and Jenkins (1976) proposed an iterative model

building strategy.

Tentative specification or identification of a model.

Efficient estimation of model parameters.

Diagnostic checking of fitted model for further

improvement.

85

Intro

ductio

n to

time series (2

013 )

Page 86: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Tentative specification. The aim is to employ statistics

that:

1. Can be readily calculated from the

data.

2. Allow the user to tentatively select a

model .

86

Intro

ductio

n to

time series (2

013 )

Page 87: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Three methods:

The Sample Autocorrelation Function

(SAF)

The Sample Partial Autocorrelation

Function (SPAF)

Use of diagnostic tools (AIC,BIC) to

find the best model (TRAMO-

automatic).

87

Intro

ductio

n to

time series (2

013 )

Page 88: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Sample Autocorrelation Function. The SACF of are defined as

with

and the sample mean of the n available observations

88

Intro

ductio

n to

time series (2

013 )

tz

,...2,1/ˆ 0 kCCkk

))((1

zzzzC jt

jn

ttj

z

Page 89: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Properties:

1.For stationary models,

2. When there exists a unit root in the AR polynomial

89

Intro

ductio

n to

time series (2

013 )

nkk ̂

1ˆ p

k

Page 90: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

If the SACF of the original series is persistently close

to 1 as k increases, one forms the first difference ,

, and studies its SACF to determine whether further

differencing is called for.

90

Intro

ductio

n to

time series (2

013 )

tz

Page 91: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Once stationary is achieved, a cutting off pattern after,

say a lag “q”, in the SACF will then lead to tentative

specification of a MA(q) model.

91

Intro

ductio

n to

time series (2

013 )

Page 92: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

The Sample Partial Autocorrelation Function:

Are obtained by replacing the in

by their sample estimates

92

Intro

ductio

n to

time series (2

013 )

,...2,1ˆ kk

k

1/

1)(

kGH

kk

kk

k

Page 93: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Properties:

1.For stationary models,

2. The are asymptotically normally distributed

3. For a stationary AR(p) model

93

Intro

ductio

n to

time series (2

013 )

nkk ̂

pknVar k 1)ˆ(

Page 94: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Properties 1, 2 and 3 make SPACF a convenient tool

for specifying the order or a stationary AR model

(cutting off pattern after lag p)

Not valid for non-stationary models.

94

Intro

ductio

n to

time series (2

013 )

Page 95: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Weakness of the SACF and SPACF.

1. Subjective judgment is often required to decide on

the order of differencing.

2. For stationary ARMA models, both SACF and

SPACF tend to exhibit a gradual “tapering off”

behavior, making the specification very difficult.

95

Intro

ductio

n to

time series (2

013 )

Page 96: Introduction to Time Series Analysishalweb.uc3m.es/esp/Personal/personas/jtena/eng/... · Introduction to Time Series Analysis Juan de Dios Tena Horrillo Office. 10.1.02B jtena@est-econ.uc3m.es

SPECIFICATION

Diagnostic Tools

The program (TRAMO), in an automatic way,

specifies a set of possible models, estimate them and

select the best one based on AIC and BIC criteria.

However an accurate jugment is always necessary to

interprect the output of TRAMO.

96

Intro

ductio

n to

time series (2

013 )