forecasting using r - rob j. hyndman · fit3

Forecasting using R 1

Forecasting using R

Rob J Hyndman

1.4 Exponential smoothing

Outline

1 Simple exponential smoothing

2 Trend methods

3 Lab session 6

4 Seasonal methods

5 Lab session 7

6 Taxonomy of exponential smoothing methods

Forecasting using R Simple exponential smoothing 2

Simple methodsTime series y1, y2, . . . , yT.Random walk forecasts

yT+h|T = yT

Average forecasts

yT+h|T =1T

T∑t=1

yt

Want something in between that weights most recentdata more highly.Simple exponential smoothing uses a weightedmoving average with weights that decreaseexponentially.

Forecasting using R Simple exponential smoothing 3

Simple methodsTime series y1, y2, . . . , yT.Random walk forecasts

yT+h|T = yT

Average forecasts

yT+h|T =1T

T∑t=1

yt

Want something in between that weights most recentdata more highly.Simple exponential smoothing uses a weightedmoving average with weights that decreaseexponentially.

Forecasting using R Simple exponential smoothing 3

Simple methodsTime series y1, y2, . . . , yT.Random walk forecasts

yT+h|T = yT

Average forecasts

yT+h|T =1T

T∑t=1

yt

Want something in between that weights most recentdata more highly.Simple exponential smoothing uses a weightedmoving average with weights that decreaseexponentially.

Forecasting using R Simple exponential smoothing 3

Simple Exponential Smoothing

Forecast equationyT+1|T = αyT + α(1− α)yT−1 + α(1− α)2yT−2 + · · ·

where 0 ≤ α ≤ 1.

Weights assigned to observations for:Observation α = 0.2 α = 0.4 α = 0.6 α = 0.8

yT 0.2 0.4 0.6 0.8yT−1 0.16 0.24 0.24 0.16yT−2 0.128 0.144 0.096 0.032yT−3 0.1024 0.0864 0.0384 0.0064yT−4 (0.2)(0.8)4 (0.4)(0.6)4 (0.6)(0.4)4 (0.8)(0.2)4yT−5 (0.2)(0.8)5 (0.4)(0.6)5 (0.6)(0.4)5 (0.8)(0.2)5

Forecasting using R Simple exponential smoothing 4

Simple Exponential Smoothing

Forecast equationyT+1|T = αyT + α(1− α)yT−1 + α(1− α)2yT−2 + · · ·

where 0 ≤ α ≤ 1.

Weights assigned to observations for:Observation α = 0.2 α = 0.4 α = 0.6 α = 0.8

yT 0.2 0.4 0.6 0.8yT−1 0.16 0.24 0.24 0.16yT−2 0.128 0.144 0.096 0.032yT−3 0.1024 0.0864 0.0384 0.0064yT−4 (0.2)(0.8)4 (0.4)(0.6)4 (0.6)(0.4)4 (0.8)(0.2)4yT−5 (0.2)(0.8)5 (0.4)(0.6)5 (0.6)(0.4)5 (0.8)(0.2)5

Forecasting using R Simple exponential smoothing 4

Simple Exponential SmoothingComponent form

Forecast equation yt+h|t = `tSmoothing equation `t = αyt + (1− α)`t−1

`t is the level (or the smoothed value) of the series attime t.yt+1|t = αyt + (1− α)yt|t−1Iterate to get exponentially weighted moving averageform.

Weighted average form

yT+1|T =T−1∑j=0α(1− α)jyT−j + (1− α)T`0

Forecasting using R Simple exponential smoothing 5

Optimisation

Need to choose value for α and `0Similarly to regression — we choose α and `0 byminimising SSE:

SSE =T∑t=1(yt − yt|t−1)2.

Unlike regression there is no closed form solution —use numerical optimization.

Forecasting using R Simple exponential smoothing 6

Example: Oil production

oil %>% ses(PI=FALSE) %>% autoplot

100

200

300

400

500

1980 2000 2020Time

y

Forecasts from Simple exponential smoothing

Forecasting using R Simple exponential smoothing 7

Outline

1 Simple exponential smoothing

2 Trend methods

3 Lab session 6

4 Seasonal methods

5 Lab session 7

6 Taxonomy of exponential smoothing methods

Forecasting using R Trend methods 8

Holt’s linear trendComponent form

Forecast yt+h|t = `t + hbtLevel `t = αyt + (1− α)(`t−1 + bt−1)Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

Two smoothing parameters α and β∗ (0 ≤ α, β∗ ≤ 1).`t level: weighted average between yt one-step aheadforecast for time t, (`t−1 + bt−1 = yt|t−1)bt slope: weighted average of (`t − `t−1) and bt−1,current and previous estimate of slope.Choose α, β∗, `0, b0 to minimise SSE.

Forecasting using R Trend methods 9

Holt’s linear trendComponent form

Forecast yt+h|t = `t + hbtLevel `t = αyt + (1− α)(`t−1 + bt−1)Trend bt = β∗(`t − `t−1) + (1− β∗)bt−1,

Two smoothing parameters α and β∗ (0 ≤ α, β∗ ≤ 1).`t level: weighted average between yt one-step aheadforecast for time t, (`t−1 + bt−1 = yt|t−1)bt slope: weighted average of (`t − `t−1) and bt−1,current and previous estimate of slope.Choose α, β∗, `0, b0 to minimise SSE.

Forecasting using R Trend methods 9

Holt’s method in R

window(ausair, start=1990, end=2004) %>%holt(h=5, PI=FALSE) %>% autoplot

20

30

40

50

1990 1995 2000 2005Time

y

Forecasts from Holt's method

Forecasting using R Trend methods 10

Damped trend method

Component formyt+h|t = `t + (φ + φ2 + · · · + φh)bt`t = αyt + (1− α)(`t−1 + φbt−1)bt = β∗(`t − `t−1) + (1− β∗)φbt−1.

Damping parameter 0 < φ < 1.If φ = 1, identical to Holt’s linear trend.As h→∞, yT+h|T → `T + φbT/(1− φ).Short-run forecasts trended, long-run forecastsconstant.

Forecasting using R Trend methods 11

Damped trend method

Component formyt+h|t = `t + (φ + φ2 + · · · + φh)bt`t = αyt + (1− α)(`t−1 + φbt−1)bt = β∗(`t − `t−1) + (1− β∗)φbt−1.

Damping parameter 0 < φ < 1.If φ = 1, identical to Holt’s linear trend.As h→∞, yT+h|T → `T + φbT/(1− φ).Short-run forecasts trended, long-run forecastsconstant.

Forecasting using R Trend methods 11

Example: Air passengers

window(ausair, start=1990, end=2004) %>%holt(damped=TRUE, h=5, PI=FALSE) %>% autoplot

20

30

40

1990 1995 2000 2005Time

y

Forecasts from Damped Holt's method

Forecasting using R Trend methods 12

Example: Sheep in Asia

livestock2 <- window(livestock, start=1970,end=2000)

fit1 <- ses(livestock2)fit2 <- holt(livestock2)fit3 <- holt(livestock2, damped = TRUE)

accuracy(fit1, livestock)accuracy(fit2, livestock)accuracy(fit3, livestock)

Forecasting using R Trend methods 13

Example: Sheep in AsiaSES Linear trend Damped trend

α 1.00 0.98 0.98β∗ 0.00 0.00φ 0.98`0 263.92 258.88 253.69b0 5.03 5.70Training RMSE 14.77 13.92 14.00Test RMSE 25.46 11.88 15.50Test MAE 20.38 10.67 13.95Test MAPE 4.60 2.53 3.21Test MASE 2.26 1.18 1.55

300

350

400

450

1970 1980 1990 2000 2010Year

Live

stoc

k, s

heep

in A

sia

(mill

ions

)

Data

SES

Holt's

Damped trend

Forecasting using R Trend methods 14

Outline

1 Simple exponential smoothing

2 Trend methods

3 Lab session 6

4 Seasonal methods

5 Lab session 7

6 Taxonomy of exponential smoothing methods

Forecasting using R Lab session 6 15

Lab Session 6

Forecasting using R Lab session 6 16

Outline

1 Simple exponential smoothing

2 Trend methods

3 Lab session 6

4 Seasonal methods

5 Lab session 7

6 Taxonomy of exponential smoothing methods

Forecasting using R Seasonal methods 17

Holt-Winters additive methodHolt and Winters extended Holt’s method to captureseasonality.Component form

yt+h|t = `t + hbt + st−m+h+m`t = α(yt − st−m) + (1− α)(`t−1 + bt−1)bt = β∗(`t − `t−1) + (1− β∗)bt−1st = γ(yt − `t−1 − bt−1) + (1− γ)st−m,

h+m = b(h− 1) mod mc + 1 = largest integer not greater than(h− 1) mod m. Ensures estimates from the final year are usedfor forecasting.Parameters: 0 ≤ α ≤ 1, 0 ≤ β∗ ≤ 1, 0 ≤ γ ≤ 1− α andm =period of seasonality (e.g. m = 4 for quarterly data).

Forecasting using R Seasonal methods 18

Holt-Winters additive method

Seasonal component is usually expressed asst = γ∗(yt − `t) + (1− γ∗)st−m.Substitute in for `t:st = γ∗(1− α)(yt − `t−1 − bt−1) + [1− γ∗(1− α)]st−mWe set γ = γ∗(1− α).The usual parameter restriction is 0 ≤ γ∗ ≤ 1, whichtranslates to 0 ≤ γ ≤ (1− α).

Forecasting using R Seasonal methods 19

Holt-Winters multiplicativeFor when seasonal variations are changing proportional to thelevel of the series.Component form

yt+h|t = (`t + hbt)st−m+h+m.

`t = αyt

st−m+ (1− α)(`t−1 + bt−1)

bt = β∗(`t − `t−1) + (1− β∗)bt−1st = γ

yt(`t−1 + bt−1)

+ (1− γ)st−m

With additive method st is in absolute terms:within each year

∑i si ≈ 0.

With multiplicative method st is in relative terms:within each year

∑i si ≈ m.

Forecasting using R Seasonal methods 20

Example: Visitor Nights

aust <- window(austourists,start=2005)fit1 <- hw(aust,seasonal="additive")fit2 <- hw(aust,seasonal="multiplicative")

30

40

50

60

2005 2007 2009 2011 2013Year

Inte

rnat

iona

l vis

itor

nigh

t in

Aus

tral

ia (

mill

ions

)

Data

HW additive forecasts

HW multiplicative forecasts

Forecasting using R Seasonal methods 21

Estimated components

35

40

45

0.6372

0.6374

0.6376

0.6378

−10

−5

0

5

10

levelslope

season

2006 2008 2010Year

Additive states

32

36

40

44

0.6

0.7

0.8

0.9

1.0

1.1

0.8

0.9

1.0

1.1

1.2

levelslope

season

2006 2008 2010Year

Multiplicative states

Forecasting using R Seasonal methods 22

Holt-Winters damped method

Often the single most accurate forecasting method forseasonal data:

yt+h|t = [`t + (φ + φ2 + · · · + φh)bt]st−m+h+m`t = α(yt/st−m) + (1− α)(`t−1 + φbt−1)bt = β∗(`t − `t−1) + (1− β∗)φbt−1st = γ

yt(`t−1 + φbt−1)

+ (1− γ)st−m

Forecasting using R Seasonal methods 23

Outline

1 Simple exponential smoothing

2 Trend methods

3 Lab session 6

4 Seasonal methods

5 Lab session 7

6 Taxonomy of exponential smoothing methods

Forecasting using R Lab session 7 24

Lab Session 7

Forecasting using R Lab session 7 25

Outline

1 Simple exponential smoothing

2 Trend methods

3 Lab session 6

4 Seasonal methods

5 Lab session 7

6 Taxonomy of exponential smoothing methods

Forecasting using R Taxonomy of exponential smoothing methods 26

Exponential smoothing methodsSeasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) (N,N) (N,A) (N,M)A (Additive) (A,N) (A,A) (A,M)Ad (Additive damped) (Ad,N) (Ad,A) (Ad,M)

(N,N): Simple exponential smoothing(A,N): Holt’s linear method(Ad,N): Additive damped trend method(A,A): Additive Holt-Winters’ method(A,M): Multiplicative Holt-Winters’ method(Ad,M): Damped multiplicative Holt-Winters’ method

There are also multiplicative trend methods (not recommended).Forecasting using R Taxonomy of exponential smoothing methods 27

Recursive formulae

Forecasting using R Taxonomy of exponential smoothing methods 28

7/ exponential smoothing 145

Trend SeasonalN A M

yt+h|t = `t yt+h|t = `t + st−m+h+m

yt+h|t = `tst−m+h+m

N `t = αyt + (1−α)`t−1 `t = α(yt − st−m) + (1−α)`t−1 `t = α(yt/st−m) + (1−α)`t−1st = γ(yt − `t−1) + (1−γ)st−m st = γ(yt/`t−1) + (1−γ)st−m

yt+h|t = `t + hbt yt+h|t = `t + hbt + st−m+h+m

yt+h|t = (`t + hbt)st−m+h+m

A `t = αyt + (1−α)(`t−1 + bt−1) `t = α(yt − st−m) + (1−α)(`t−1 + bt−1) `t = α(yt/st−m) + (1−α)(`t−1 + bt−1)bt = β∗(`t − `t−1) + (1− β∗)bt−1 bt = β∗(`t − `t−1) + (1− β∗)bt−1 bt = β∗(`t − `t−1) + (1− β∗)bt−1

st = γ(yt − `t−1 − bt−1) + (1−γ)st−m st = γ(yt/(`t−1 − bt−1)) + (1−γ)st−myt+h|t = `t +φhbt yt+h|t = `t +φhbt + st−m+h+

myt+h|t = (`t +φhbt)st−m+h+

m

Ad `t = αyt + (1−α)(`t−1 +φbt−1) `t = α(yt − st−m) + (1−α)(`t−1 +φbt−1) `t = α(yt/st−m) + (1−α)(`t−1 +φbt−1)bt = β∗(`t − `t−1) + (1− β∗)φbt−1 bt = β∗(`t − `t−1) + (1− β∗)φbt−1 bt = β∗(`t − `t−1) + (1− β∗)φbt−1

st = γ(yt − `t−1 −φbt−1) + (1−γ)st−m st = γ(yt/(`t−1 −φbt−1)) + (1−γ)st−myt+h|t = `tb

ht yt+h|t = `tb

ht + st−m+h+

myt+h|t = `tb

ht st−m+h+

m

M `t = αyt + (1−α)`t−1bt−1 `t = α(yt − st−m) + (1−α)`t−1bt−1 `t = α(yt/st−m) + (1−α)`t−1bt−1bt = β∗(`t/`t−1) + (1− β∗)bt−1 bt = β∗(`t/`t−1) + (1− β∗)bt−1 bt = β∗(`t/`t−1) + (1− β∗)bt−1

st = γ(yt − `t−1bt−1) + (1−γ)st−m st = γ(yt/(`t−1bt−1)) + (1−γ)st−m

yt+h|t = `tbφht yt+h|t = `tb

φht + st−m+h+

myt+h|t = `tb

φht st−m+h+

m

Md `t = αyt + (1−α)`t−1bφt−1 `t = α(yt − st−m) + (1−α)`t−1b

φt−1 `t = α(yt/st−m) + (1−α)`t−1b

φt−1

bt = β∗(`t/`t−1) + (1− β∗)bφt−1 bt = β∗(`t/`t−1) + (1− β∗)bφt−1 bt = β∗(`t/`t−1) + (1− β∗)bφt−1st = γ(yt − `t−1b

φt−1) + (1−γ)st−m st = γ(yt/(`t−1b

φt−1)) + (1−γ)st−m

Table 7.8: Formulae for recursivecalculations and point forecasts. Ineach case, `t denotes the series levelat time t, bt denotes the slope at timet, st denotes the seasonal componentof the series at time t, and m denotesthe number of seasons in a year; α, β∗,γ and φ are smoothing parameters,φh = φ + φ2 + · · · + φh and h+

m =b(h− 1) mod mc+ 1.

Method Initial values

(N,N) l0 = y1

(A,N) (Ad,N) l0 = y1, b0 = y2 − y1

(M,N) (Md,N) l0 = y1, b0 = y2/y1

(A,A) (Ad,A) l0 = 1m (y1 + · · ·+ ym)

b0 = 1m

[ym+1−y1

m + · · ·+ ym+m−ymm

]

s0 = y1 − `0, s−1 = y2 − `0, . . . , s−m+1 = ym − `0

(A,M) (Ad,M) l0 = 1m (y1 + · · ·+ ym)

b0 = 1m

[ym+1−y1

m + · · ·+ ym+m−ymm

]

s0 = y1/`0, s−1 = y2/`0, . . . , s−m+1 = ym/`0

Table 7.9: Initialisation strategies forsome of the more commonly usedexponential smoothing methods.

R functions

Simple exponential smoothing: no trend.ses(y)Holt’s method: linear trend.holt(y)Damped trend method.holt(y, damped=TRUE)Holt-Winters methodshw(y, damped=TRUE, seasonal="additive")hw(y, damped=FALSE, seasonal="additive")hw(y, damped=TRUE, seasonal="multiplicative")hw(y, damped=FALSE, seasonal="multiplicative")

Forecasting using R Taxonomy of exponential smoothing methods 29