forecasting using r - rob j. hyndman · 2016. 10. 7. · forecasting using r some simple...
TRANSCRIPT
![Page 1: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/1.jpg)
Forecasting using R 1
Forecasting using R
Rob J Hyndman
1.2 The forecaster’s toolbox
![Page 2: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/2.jpg)
Outline
1 Forecasting
2 Some simple forecasting methods
3 Forecasting residuals
4 Lab session 3
5 Evaluating forecast accuracy
6 Lab session 4
Forecasting using R Forecasting 2
![Page 3: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/3.jpg)
Forecasting
Forecasting is estimating how the sequence ofobservations will continue into the future.
We usually think probabilistically about future samplepathsWhat range of values covers the possible samplepaths with 80% probability?
Forecasting using R Forecasting 3
![Page 4: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/4.jpg)
Australian beer production
Forecasting using R Forecasting 4
![Page 5: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/5.jpg)
Australian beer production
400
450
500
1995 2000 2005 2010Time
y
level
80
95
Forecasts from ETS(M,N,M)
Forecasting using R Forecasting 5
![Page 6: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/6.jpg)
Outline
1 Forecasting
2 Some simple forecasting methods
3 Forecasting residuals
4 Lab session 3
5 Evaluating forecast accuracy
6 Lab session 4
Forecasting using R Some simple forecasting methods 6
![Page 7: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/7.jpg)
Some simple forecasting methods
400
450
500
1995 2000 2005Year
meg
alitr
es
Australian quarterly beer production
Forecasting using R Some simple forecasting methods 7
Can you think of any forecasting methods for these data?
![Page 8: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/8.jpg)
Some simple forecasting methods
400
450
500
1995 2000 2005Year
meg
alitr
es
Australian quarterly beer production
Forecasting using R Some simple forecasting methods 7
Can you think of any forecasting methods for these data?
![Page 9: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/9.jpg)
Some simple forecasting methods
80
90
100
110
1990 1991 1992 1993 1994 1995Year
thou
sand
s
Number of pigs slaughtered in Victoria
Forecasting using R Some simple forecasting methods 8
How would you forecast these data?
![Page 10: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/10.jpg)
Some simple forecasting methods
80
90
100
110
1990 1991 1992 1993 1994 1995Year
thou
sand
s
Number of pigs slaughtered in Victoria
Forecasting using R Some simple forecasting methods 8
How would you forecast these data?
![Page 11: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/11.jpg)
Some simple forecasting methods
3600
3700
3800
3900
4000
0 50 100 150 200 250 300Day
Dow−Jones index
Forecasting using R Some simple forecasting methods 9
How would you forecast these data?
![Page 12: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/12.jpg)
Some simple forecasting methods
3600
3700
3800
3900
4000
0 50 100 150 200 250 300Day
Dow−Jones index
Forecasting using R Some simple forecasting methods 9
How would you forecast these data?
![Page 13: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/13.jpg)
Some simple forecasting methodsAverage method
Forecast of all future values is equal to mean of historicaldata {y1, . . . , yT}.Forecasts: yT+h|T = y = (y1 + · · · + yT)/T
Naïve methodForecasts equal to last observed value.Forecasts: yT+h|T = yT.Consequence of efficient market hypothesis.
Seasonal naïve methodForecasts equal to last value from same season.Forecasts: yT+h|T = yT+h−km wherem = seasonal period andk = b(h− 1)/mc+1.
Forecasting using R Some simple forecasting methods 10
![Page 14: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/14.jpg)
Some simple forecasting methodsAverage method
Forecast of all future values is equal to mean of historicaldata {y1, . . . , yT}.Forecasts: yT+h|T = y = (y1 + · · · + yT)/T
Naïve methodForecasts equal to last observed value.Forecasts: yT+h|T = yT.Consequence of efficient market hypothesis.
Seasonal naïve methodForecasts equal to last value from same season.Forecasts: yT+h|T = yT+h−km wherem = seasonal period andk = b(h− 1)/mc+1.
Forecasting using R Some simple forecasting methods 10
![Page 15: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/15.jpg)
Some simple forecasting methodsAverage method
Forecast of all future values is equal to mean of historicaldata {y1, . . . , yT}.Forecasts: yT+h|T = y = (y1 + · · · + yT)/T
Naïve methodForecasts equal to last observed value.Forecasts: yT+h|T = yT.Consequence of efficient market hypothesis.
Seasonal naïve methodForecasts equal to last value from same season.Forecasts: yT+h|T = yT+h−km wherem = seasonal period andk = b(h− 1)/mc+1.
Forecasting using R Some simple forecasting methods 10
![Page 16: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/16.jpg)
Some simple forecasting methods
400
450
500
1995 2000 2005 2010Year
Meg
alitr
es
Forecast Method
Mean
Naïve
Seasonal Naïve
Forecasts for quarterly beer production
Forecasting using R Some simple forecasting methods 11
![Page 17: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/17.jpg)
Some simple forecasting methods
3600
3700
3800
3900
4000
0 100 200 300Day
Forecast Method
Mean
Naïve
Dow Jones Index (daily ending 15 Jul 94)
Forecasting using R Some simple forecasting methods 12
![Page 18: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/18.jpg)
Some simple forecasting methods
Mean: meanf(y, h=20)Naïve: naive(y, h=20)Seasonal naïve: snaive(y, h=20)Linear trend: forecast(tslm(y ~ trend))Linear trend with seasonal dummies:forecast(tslm(y ~ trend + season))
Forecasting using R Some simple forecasting methods 13
![Page 19: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/19.jpg)
Outline
1 Forecasting
2 Some simple forecasting methods
3 Forecasting residuals
4 Lab session 3
5 Evaluating forecast accuracy
6 Lab session 4
Forecasting using R Forecasting residuals 14
![Page 20: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/20.jpg)
Forecasting residualsResiduals in forecasting: difference between observedvalue and its forecast based on all previous observations:et = yt − yt|t−1.
Assumptions
1 {et} uncorrelated. If they aren’t, then information left inresiduals that should be used in computing forecasts.
2 {et} have mean zero. If they don’t, then forecasts arebiased.
Useful properties (for prediction intervals)
3 {et} have constant variance.4 {et} are normally distributed.
Forecasting using R Forecasting residuals 15
![Page 21: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/21.jpg)
Forecasting residualsResiduals in forecasting: difference between observedvalue and its forecast based on all previous observations:et = yt − yt|t−1.
Assumptions
1 {et} uncorrelated. If they aren’t, then information left inresiduals that should be used in computing forecasts.
2 {et} have mean zero. If they don’t, then forecasts arebiased.
Useful properties (for prediction intervals)
3 {et} have constant variance.4 {et} are normally distributed.
Forecasting using R Forecasting residuals 15
![Page 22: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/22.jpg)
Forecasting residualsResiduals in forecasting: difference between observedvalue and its forecast based on all previous observations:et = yt − yt|t−1.
Assumptions
1 {et} uncorrelated. If they aren’t, then information left inresiduals that should be used in computing forecasts.
2 {et} have mean zero. If they don’t, then forecasts arebiased.
Useful properties (for prediction intervals)
3 {et} have constant variance.4 {et} are normally distributed.
Forecasting using R Forecasting residuals 15
![Page 23: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/23.jpg)
Example: Dow-Jones index
3600
3700
3800
3900
4000
0 50 100 150 200 250Day
series
Data
Fitted
Dow Jones Index (daily ending 15 Jul 94)
Forecasting using R Forecasting residuals 16
![Page 24: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/24.jpg)
Example: Dow-Jones index
Naïve forecast:
yt|t−1 = yt−1
et = yt − yt−1
Note: et are one-step-forecast residuals
Forecasting using R Forecasting residuals 17
![Page 25: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/25.jpg)
Example: Dow-Jones index
Naïve forecast:
yt|t−1 = yt−1
et = yt − yt−1
Note: et are one-step-forecast residuals
Forecasting using R Forecasting residuals 17
![Page 26: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/26.jpg)
Example: Dow-Jones index
Naïve forecast:
yt|t−1 = yt−1
et = yt − yt−1
Note: et are one-step-forecast residuals
Forecasting using R Forecasting residuals 17
![Page 27: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/27.jpg)
Example: Dow-Jones index
−100
−50
0
50
0 50 100 150 200 250Day
Residuals from naive method
Forecasting using R Forecasting residuals 18
![Page 28: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/28.jpg)
Example: Dow-Jones index
0
10
20
30
40
50
−100 −50 0 50res
coun
t
Histogram of residuals
Forecasting using R Forecasting residuals 19
![Page 29: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/29.jpg)
Example: Dow-Jones index
−0.10
−0.05
0.00
0.05
0.10
0 5 10 15 20Lag
AC
F
ACF of residuals
Forecasting using R Forecasting residuals 20
![Page 30: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/30.jpg)
Example: Dow-Jones index
fc <- naive(dj)
res <- residuals(fc)
autoplot(res)
hist(res,breaks="FD")
ggAcf(res,main="")
Forecasting using R Forecasting residuals 21
![Page 31: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/31.jpg)
ACF of residuals
We assume that the residuals are white noise(uncorrelated, mean zero, constant variance). If theyaren’t, then there is information left in the residualsthat should be used in computing forecasts.So a standard residual diagnostic is to check the ACFof the residuals of a forecasting method.We expect these to look like white noise.
Forecasting using R Forecasting residuals 22
![Page 32: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/32.jpg)
Portmanteau testsConsider a whole set of rk values, and develop a test to seewhether the set is significantly different from a zero set.Box-Pierce test
Q = Th∑k=1
r2kwhere h is max lag being considered and T is number ofobservations.
My preferences: h = 10 for non-seasonal data, h = 2mfor seasonal data.If each rk close to zero, Q will be small.If some rk values large (positive or negative), Q will belarge.
Forecasting using R Forecasting residuals 23
![Page 33: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/33.jpg)
Portmanteau testsConsider a whole set of rk values, and develop a test to seewhether the set is significantly different from a zero set.Box-Pierce test
Q = Th∑k=1
r2kwhere h is max lag being considered and T is number ofobservations.
My preferences: h = 10 for non-seasonal data, h = 2mfor seasonal data.If each rk close to zero, Q will be small.If some rk values large (positive or negative), Q will belarge.
Forecasting using R Forecasting residuals 23
![Page 34: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/34.jpg)
Portmanteau testsConsider a whole set of rk values, and develop a test to seewhether the set is significantly different from a zero set.Ljung-Box test
Q∗ = T(T + 2)h∑k=1
(T − k)−1r2kwhere h is max lag being considered and T is number ofobservations.
My preferences: h = 10 for non-seasonal data, h = 2mfor seasonal data.Better performance, especially in small samples.
Forecasting using R Forecasting residuals 24
![Page 35: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/35.jpg)
Portmanteau testsIf data are WN, Q∗ has χ2 distribution with (h− K) degreesof freedom where K = no. parameters in model.When applied to raw data, set K = 0.For the Dow-Jones example,
res <- residuals(naive(dj))
# lag=h and fitdf=K
> Box.test(res, lag=10, fitdf=0)Box-Pierce test
X-squared = 14.0451, df = 10, p-value = 0.1709> Box.test(res, lag=10, fitdf=0, type="Lj")
Box-Ljung testX-squared = 14.4615, df = 10, p-value = 0.153
Forecasting using R Forecasting residuals 25
![Page 36: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/36.jpg)
Outline
1 Forecasting
2 Some simple forecasting methods
3 Forecasting residuals
4 Lab session 3
5 Evaluating forecast accuracy
6 Lab session 4
Forecasting using R Lab session 3 26
![Page 37: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/37.jpg)
Lab Session 3
Forecasting using R Lab session 3 27
![Page 38: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/38.jpg)
Outline
1 Forecasting
2 Some simple forecasting methods
3 Forecasting residuals
4 Lab session 3
5 Evaluating forecast accuracy
6 Lab session 4
Forecasting using R Evaluating forecast accuracy 28
![Page 39: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/39.jpg)
Measures of forecast accuracyLet yt denote the tth observation and yt|t−1 denote itsforecast based on all previous data, where t = 1, . . . , T.Then the following measures are useful.
MAE = T−1T∑t=1|yt − yt|t−1|
MSE = T−1T∑t=1
(yt − yt|t−1)2 RMSE =
√√√√√T−1T∑t=1
(yt − yt|t−1)2
MAPE = 100T−1T∑t=1|yt − yt|t−1|/|yt|
MAE, MSE, RMSE are all scale dependent.MAPE is scale independent but is only sensible ifyt � 0 for all t, and y has a natural zero.
Forecasting using R Evaluating forecast accuracy 29
![Page 40: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/40.jpg)
Measures of forecast accuracyLet yt denote the tth observation and yt|t−1 denote itsforecast based on all previous data, where t = 1, . . . , T.Then the following measures are useful.
MAE = T−1T∑t=1|yt − yt|t−1|
MSE = T−1T∑t=1
(yt − yt|t−1)2 RMSE =
√√√√√T−1T∑t=1
(yt − yt|t−1)2
MAPE = 100T−1T∑t=1|yt − yt|t−1|/|yt|
MAE, MSE, RMSE are all scale dependent.MAPE is scale independent but is only sensible ifyt � 0 for all t, and y has a natural zero.
Forecasting using R Evaluating forecast accuracy 29
![Page 41: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/41.jpg)
Measures of forecast accuracyMean Absolute Scaled Error
MASE = T−1T∑t=1|yt − yt|t−1|/Q
where Q is a stable measure of the scale of the time series{yt}.
Proposed by Hyndman and Koehler (IJF, 2006).For non-seasonal time series,
Q = (T − 1)−1T∑t=2|yt − yt−1|
works well. Then MASE is equivalent to MAE relative to anaïve method.
Forecasting using R Evaluating forecast accuracy 30
![Page 42: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/42.jpg)
Measures of forecast accuracyMean Absolute Scaled Error
MASE = T−1T∑t=1|yt − yt|t−1|/Q
where Q is a stable measure of the scale of the time series{yt}.
Proposed by Hyndman and Koehler (IJF, 2006).For seasonal time series,
Q = (T −m)−1T∑
t=m+1|yt − yt−m|
works well. Then MASE is equivalent to MAE relative to aseasonal naïve method.
Forecasting using R Evaluating forecast accuracy 31
![Page 43: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/43.jpg)
Measures of forecast accuracy
400
450
500
1995 2000 2005Year
Meg
alitr
es
Forecast Method
Mean
Naive
SeasonalNaive
Forecasts for quarterly beer production
Forecasting using R Evaluating forecast accuracy 32
![Page 44: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/44.jpg)
Measures of forecast accuracy
beer3 <- window(ausbeer, start=2006)accuracy(beerfit1, beer3)accuracy(beerfit2, beer3)accuracy(beerfit3, beer3)
RMSE MAE MAPE MASE
Mean method 38.95 34.46 8.33 2.35Naïve method 70.80 63.10 15.71 4.29Seasonal naïve method 13.59 12.20 2.95 0.83
Forecasting using R Evaluating forecast accuracy 33
![Page 45: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/45.jpg)
Training and test sets
Available data
Training set Test set(e.g., 80%) (e.g., 20%)
The test set must not be used for any aspect of modeldevelopment or calculation of forecasts.Forecast accuracy is based only on the test set.
Forecasting using R Evaluating forecast accuracy 34
![Page 46: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/46.jpg)
Training and test sets
beer3 <- window(ausbeer,start=1992,end=c(2005,4))beer4 <- window(ausbeer,start=2006)
fit1 <- meanf(beer3,h=20)fit2 <- naive(beer3,h=20)
accuracy(fit1,beer4)accuracy(fit2,beer4)
In-sample accuracy (one-step forecasts)
accuracy(fit1)accuracy(fit2)
Forecasting using R Evaluating forecast accuracy 35
![Page 47: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/47.jpg)
Training and test sets
beer3 <- window(ausbeer,start=1992,end=c(2005,4))beer4 <- window(ausbeer,start=2006)
fit1 <- meanf(beer3,h=20)fit2 <- naive(beer3,h=20)
accuracy(fit1,beer4)accuracy(fit2,beer4)
In-sample accuracy (one-step forecasts)
accuracy(fit1)accuracy(fit2)
Forecasting using R Evaluating forecast accuracy 35
![Page 48: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/48.jpg)
Beware of over-fittingA model which fits the data well does not necessarilyforecast well.A perfect fit can always be obtained by using a modelwith enough parameters. (Compare R2)Over-fitting a model to data is as bad as failing toidentify the systematic pattern in the data.Problems can be overcome by measuring trueout-of-sample forecast accuracy. That is, total datadivided into “training” set and “test” set. Training setused to estimate parameters. Forecasts are made fortest set.Accuracy measures computed for errors in test setonly.
Forecasting using R Evaluating forecast accuracy 36
![Page 49: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/49.jpg)
Poll: true or false?
1 Good forecast methods should have normallydistributed residuals.
2 A model with small residuals will give good forecasts.3 The best measure of forecast accuracy is MAPE.4 If your model doesn’t forecast well, you should make
it more complicated.5 Always choose the model with the best forecast
accuracy as measured on the test set.
Forecasting using R Evaluating forecast accuracy 37
![Page 50: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/50.jpg)
Outline
1 Forecasting
2 Some simple forecasting methods
3 Forecasting residuals
4 Lab session 3
5 Evaluating forecast accuracy
6 Lab session 4
Forecasting using R Lab session 4 38
![Page 51: Forecasting using R - Rob J. Hyndman · 2016. 10. 7. · Forecasting using R Some simple forecasting methods 10. Some simple forecasting methods. 400 450 500 1995 2000 2005 2010](https://reader035.vdocuments.site/reader035/viewer/2022062416/6123eb4a1b21cd00f901499b/html5/thumbnails/51.jpg)
Lab Session 4
Forecasting using R Lab session 4 39