estimating sd errors

8/12/2019 Estimating Sd Errors

1/70

Intro to EstimatingSDErrors.xls.This workbook demonstrates three facts about the estimation of the SD of the errors:

1) Estimating the SD of the Errors with just the SD of the Residuals gives a biased estimato2) Estimating the SD of the Errors with the RMSE (the SD of the Residuals adjusted by a de

is better than using just the SD of the Residuals.

3) Estimating the SD of the Errors with the RMSE gives a biased, but consistent estimator.

Because the RMSE is a consistent estimator of the SD of the errors, it is commonly used as an estimat

The data generation process is the classical econometric box model in the bivariate case:

Y = b0+ b1X + e

where eis iid, the Xs are fixed in repeated sampling and independent of the errors.

The Data sheet implements this DGP and can be used to run Monte Carlo simulations with the MCSimThe n sheets have Monte Carlo results.

The Q&A sheet has a few practice problems.


2/70

.grees of freedom factor)

of the SD of the errors.

add-in.


3/70

DGP is CEM: Y = b0+ b1X + e In order to compare ap

e~N(0,SD Error), iid chosen, the SD of X reb0 0 you can compare the e

b1 5 n 10

SD Error 5

1 1.000

X Error Y

0.174 -2.178 -1.307 slope 5.59 -2.72 intercept0.522 -11.201 -8.589 est SE 1.76 3.53 est SE0.870 -1.568 2.784 R square 0.56 5.568 RMSE1.219 1.896 7.989 F 10.10 8 df 1.567 0.324 8.157 Reg SS 313.00 248.01 SSR1.915 9.267 18.8422.263 -5.920 5.3952.611 -0.870 12.186

2.959 -3.415 11.3813.307 -3.213 13.324

Predicted Y = b0+ b1X

y = 5.59x - 2.72

-10

-5

0

5

10

15

20

25

0 1 2 3


4/70

ples to apples, no matter the value of n

mains constant. With the SD of X fixed,

ffect of n on the SE of the sample slope.

5.568 RMSE by formula4.980 SD Residuals See ExponentialDist.xls in the \Basic Tools\RandomNu

Residuals to learn about the distribution of the errors associated wi

0.442 To return to drawing errors from the Normal Distribution-8.788 click on the Set N button at cell D2.0.6383.8942.115

10.852-4.5430.300

-2.452-2.457


5/70

ber folder

h this button.


6/70


7/70


8/70


9/70


10/70


11/70


12/70


13/70


14/70

t


15/70

Results of Monte Carlo SimulationSample

Number

Data!$F$

11 Data!$J$7

1 1.481 4.683 1000 repetitions2 1.142 3.613 4 secs

3 1.319 4.170 3.1623Average 1.520

4 1.622 5.128 10SD 0.3776

5 1.786 5.649 Max 2.852

6 1.325 4.189 Min 0.576

7 1.500 4.744

8 1.182 3.737

9 1.144 3.617

10 1.175 3.716

11 0.888 2.80712 1.852 5.858

13 0.899 2.841

14 0.700 2.214

15 1.431 4.525

16 1.443 4.562

17 1.515 4.791 1

18 2.081 6.580

19 0.576 1.822

20 1.465 4.632

21 1.371 4.33622 1.390 4.394

23 2.135 6.751

24 2.242 7.089

25 1.208 3.820

26 1.213 3.836

27 0.973 3.076

28 1.253 3.963

29 0.947 2.993

30 1.066 3.370

31 1.590 5.02732 1.963 6.209

33 1.480 4.680

34 2.033 6.430

35 0.803 2.538

36 1.639 5.182

37 1.135 3.591

Data!$F$11

Simulation Stats

0.5 2.5

Histog


16/70

38 1.683 5.323

39 1.730 5.471

40 1.757 5.555

41 1.415 4.474

42 1.480 4.679

43 1.173 3.711

44 1.639 5.184

45 1.409 4.457

46 1.756 5.554

47 1.114 3.523

48 1.610 5.091

49 1.713 5.416

50 1.619 5.119

51 1.487 4.702

52 1.492 4.718

53 1.652 5.22354 0.907 2.868

55 1.091 3.450

56 1.670 5.280

57 2.245 7.099

58 1.639 5.184

59 0.910 2.878

60 1.819 5.753

61 1.294 4.092

62 1.174 3.712

63 0.963 3.04764 2.240 7.084

65 1.579 4.992

66 1.171 3.702

67 2.062 6.521

68 1.157 3.660

69 1.181 3.733

70 1.537 4.862

71 1.447 4.576

72 1.705 5.393

73 1.215 3.84474 1.237 3.912

75 1.012 3.199

76 1.397 4.417

77 1.517 4.796

78 1.133 3.583

79 1.640 5.186


17/70

80 1.659 5.246

81 1.692 5.352

82 1.450 4.586

83 1.178 3.724

84 1.225 3.873

85 1.668 5.273

86 1.435 4.537

87 1.182 3.736

88 1.155 3.654

89 1.265 4.001

90 1.832 5.795

91 1.421 4.493

92 1.518 4.800

93 1.315 4.158

94 1.781 5.632

95 1.184 3.74396 1.285 4.062

97 1.983 6.270

98 1.404 4.441

99 1.863 5.890

100 1.991 6.297

Only the first 100 repetitions are displayed on this worksheet.


18/70

Average 4.808n = 10

SD 1.1942

Max 9.018

Min 1.822

Data!$J$7 Notes

4.5 6.5 8.5

am of Data!$F$11 And Data!$J$7

Data!$F$11

Data!$J$7


19/70


20/70


21/70


22/70


23/70


24/70


25/70


26/70


27/70

Data!$F$1 Data!$J$7

0.5 0.5 0 0

0.75 0.5 10 01 0.75 10 0

1.25 0.75 52 0

1.5 1 52 0

1.75 1 196 0

2 1.25 196 0

2.25 1.25 248 0

2.5 1.5 248 0

2.75 1.5 230 0

3 1.75 230 0

1.75 146 32 146 3

2 84 4

2.25 84 4

2.25 25 8

2.5 25 8

2.5 6 11

2.75 6 11

2.75 3 22

3 3 22

3 0 303.25 30

3.25 49

3.5 49

3.5 73

3.75 73

3.75 71

4 71

4 77

4.25 77

4.25 874.5 87

4.5 74

4.75 74

4.75 77

5 77

5 80


28/70

5.25 80

5.25 60

5.5 60

5.5 67

5.75 67

5.75 56

6 56

6 25

6.25 25

6.25 26

6.5 26

6.5 32

6.75 32

6.75 24

7 24

7 177.25 17

7.25 11

7.5 11

7.5 4

7.75 4

7.75 3

8 3

8 4

8.25 4

8.25 18.5 1

8.5 1

8.75 1

8.75 2

9 2

9 1

9.25 1

9.25 0


29/70


Number

Data!$F$

11 Data!$J$7


3 1.299 5.810 Average 1.091

4 1.045 4.675 SD 0.1829

5 1.274 5.697 Max 1.620

6 1.268 5.670 Min 0.634

7 1.420 6.349

8 1.313 5.871

9 1.208 5.403

10 1.155 5.163

11 1.194 5.34112 0.930 4.161

13 1.256 5.618

14 1.017 4.546

15 1.152 5.150

16 1.065 4.761

17 0.731 3.270 1

18 1.204 5.383

19 0.918 4.104

20 1.463 6.543

21 1.260 5.63622 1.223 5.472

23 1.151 5.147

24 0.944 4.220

25 0.957 4.282

26 1.087 4.861

27 1.277 5.711

28 1.196 5.347

29 1.235 5.524

30 1.098 4.910

31 1.042 4.66232 0.690 3.086

33 0.894 3.998

34 0.900 4.023

35 0.771 3.446

36 1.132 5.061

37 1.020 4.561

Data!$F$11

Simulation Stats

0.6 1.6

Histog


30/70

38 1.176 5.261

39 1.056 4.724

40 1.100 4.917

41 1.060 4.742

42 1.150 5.143

43 1.112 4.971

44 1.299 5.809

45 0.940 4.204

46 1.293 5.782

47 1.087 4.861

48 1.175 5.257

49 0.809 3.619

50 0.946 4.230

51 1.297 5.798

52 1.066 4.769

53 0.784 3.50454 1.169 5.230

55 1.229 5.498

56 1.297 5.802

57 0.949 4.246

58 1.238 5.536

59 0.735 3.287

60 0.962 4.301

61 1.345 6.017

62 1.120 5.010

63 0.896 4.00964 0.878 3.927

65 0.875 3.913

66 1.230 5.500

67 1.154 5.161

68 0.898 4.018

69 1.092 4.884

70 1.332 5.959

71 1.391 6.220

72 0.867 3.879

73 0.812 3.63274 0.946 4.231

75 1.041 4.654

76 0.996 4.453

77 1.003 4.487

78 1.167 5.220

79 1.254 5.606


31/70

80 0.868 3.883

81 1.080 4.831

82 1.237 5.533

83 1.003 4.485

84 1.239 5.541

85 1.149 5.140

86 1.185 5.301

87 1.213 5.424

88 1.116 4.993

89 1.106 4.946

90 1.148 5.133

91 1.442 6.449

92 1.179 5.274

93 1.156 5.170

94 1.079 4.826

95 0.982 4.39496 1.298 5.807

97 0.977 4.368

98 1.171 5.238

99 1.215 5.435

100 0.639 2.858



32/70


33/70

Average 4.881

SD 0.8179

Max 7.246

Min 2.835

Data!$J$7 Notes

.6 3.6 4.6 5.6 6.6


Data!$F$11

Data!$J$7


34/70


35/70


36/70


37/70


38/70


39/70


40/70


41/70


42/70


43/70


44/70


45/70

Data!$F$1 Data!$J$7

0.6 0.6 0 0

0.7 0.6 14 00.8 0.7 14 0

0.9 0.7 38 0

1 0.8 38 0

1.1 0.8 95 0

1.2 0.9 95 0

1.3 0.9 176 0

1.4 1 176 0

1.5 1 220 0

1.6 1.1 220 0

1.7 1.1 176 01.2 176 0

1.2 157 0

1.3 157 0

1.3 70 0

1.4 70 0

1.4 37 0

1.5 37 0

1.5 14 0

1.6 14 0

1.6 3 01.7 3 0

1.7 0 0

1.8 0

1.8 0

1.9 0

1.9 0

2 0

2 0

2.1 0

2.1 02.2 0

2.2 0

2.3 0

2.3 0

2.4 0

2.4 0


46/70

2.5 0

2.5 0

2.6 0

2.6 0

2.7 0

2.7 0

2.8 0

2.8 3

2.9 3

2.9 2

3 2

3 8

3.1 8

3.1 5

3.2 5

3.2 83.3 8

3.3 8

3.4 8

3.4 8

3.5 8

3.5 12

3.6 12

3.6 19

3.7 19

3.7 153.8 15

3.8 19

3.9 19

3.9 25

4 25

4 41

4.1 41

4.1 24

4.2 24

4.2 43

4.3 43

4.3 47

4.4 47

4.4 49

4.5 49

4.5 45


47/70

4.6 45

4.6 50

4.7 50

4.7 46

4.8 46

4.8 55

4.9 55

4.9 48

5 48

5 29

5.1 29

5.1 51

5.2 51

5.2 34

5.3 34

5.3 35

5.4 35

5.4 39

5.5 39

5.5 48

5.6 48

5.6 25

5.7 25

5.7 28

5.8 28

5.8 21

5.9 21

5.9 13

6 13

6 23

6.1 23

6.1 14

6.2 14

6.2 9

6.3 9

6.3 10

6.4 10

6.4 12

6.5 12

6.5 8

6.6 8

6.6 4


48/70

6.7 4

6.7 6

6.8 6

6.8 2

6.9 2

6.9 4

7 4

7 0

7.1 0

7.1 4

7.2 4

7.2 1

7.3 1

7.3 0


49/70


Number

Data!$F$

11 Data!$J$7


3 0.852 5.391 Average 0.784

4 0.818 5.176 SD 0.0867

5 0.708 4.479 Max 1.072

6 0.846 5.353 Min 0.524

7 0.764 4.829

8 0.768 4.859

9 0.729 4.614

10 0.780 4.932

11 0.790 4.99412 0.729 4.608

13 0.900 5.695

14 0.853 5.392

15 0.680 4.304

16 0.777 4.914

17 0.718 4.542 1

18 0.923 5.836

19 0.848 5.365

20 0.704 4.453

21 0.737 4.66022 0.653 4.128

23 0.865 5.470

24 0.862 5.449

25 0.799 5.054

26 0.835 5.279

27 0.801 5.065

28 0.780 4.935

29 0.954 6.031

30 0.600 3.795

31 0.718 4.54132 0.747 4.725

33 0.823 5.207

34 0.705 4.462

35 0.727 4.598

36 0.879 5.558

37 0.749 4.735

Data!$F$11

Simulation Stats

0.5 1.5

Histog


50/70

38 0.727 4.596

39 0.827 5.232

40 0.772 4.880

41 0.614 3.885

42 0.796 5.032

43 0.680 4.299

44 0.793 5.014

45 0.710 4.493

46 0.747 4.726

47 0.832 5.263

48 0.761 4.812

49 0.706 4.464

50 0.778 4.919

51 0.654 4.134

52 0.764 4.830

53 0.722 4.56654 0.853 5.395

55 0.758 4.795

56 0.726 4.590

57 0.794 5.021

58 0.738 4.667

59 0.722 4.566

60 0.681 4.308

61 0.800 5.058

62 0.920 5.819

63 0.666 4.21364 0.807 5.105

65 0.612 3.871

66 0.718 4.541

67 0.753 4.759

68 0.720 4.551

69 0.801 5.067

70 0.686 4.341

71 0.677 4.284

72 0.792 5.007

73 0.908 5.74274 0.794 5.024

75 0.849 5.373

76 0.819 5.182

77 0.752 4.759

78 0.772 4.881

79 0.764 4.831


51/70

80 0.754 4.770

81 0.754 4.766

82 0.809 5.118

83 0.717 4.537

84 0.799 5.051

85 0.724 4.580

86 0.873 5.521

87 0.744 4.707

88 0.823 5.204

89 0.770 4.872

90 0.830 5.249

91 0.761 4.813

92 0.660 4.177

93 0.657 4.155

94 0.790 4.997

95 0.718 4.54096 0.929 5.877

97 0.855 5.410

98 0.742 4.695

99 0.993 6.282

100 0.904 5.716



52/70


53/70

Average 4.960

SD 0.5486

Max 6.778

Min 3.314

Data!$J$7 Notes

2.5 3.5 4.5 5.5 6.5


Data!$F$11

Data!$J$7


54/70


55/70


56/70


57/70


58/70


59/70


60/70


61/70


62/70


63/70


64/70


65/70

Data!$F$1 Data!$J$7

0.5 0.5 0 0

0.6 0.5 8 00.7 0.6 8 0

0.8 0.6 151 0

0.9 0.7 151 0

1 0.7 420 0

1.1 0.8 420 0

0.8 321 0

0.9 321 0

0.9 92 0

1 92 0

1 8 01.1 8 0

1.1 0 0

1.2 0

1.2 0

1.3 0

1.3 0

1.4 0

1.4 0

1.5 0

1.5 01.6 0

1.6 0

1.7 0

1.7 0

1.8 0

1.8 0

1.9 0

1.9 0

2 0

2 02.1 0

2.1 0

2.2 0

2.2 0

2.3 0

2.3 0


66/70

2.4 0

2.4 0

2.5 0

2.5 0

2.6 0

2.6 0

2.7 0

2.7 0

2.8 0

2.8 0

2.9 0

2.9 0

3 0

3 0

3.1 0

3.1 03.2 0

3.2 0

3.3 0

3.3 2

3.4 2

3.4 0

3.5 0

3.5 0

3.6 0

3.6 13.7 1

3.7 6

3.8 6

3.8 9

3.9 9

3.9 18

4 18

4 18

4.1 18

4.1 24

4.2 24

4.2 30

4.3 30

4.3 41

4.4 41

4.4 57


67/70

4.5 57

4.5 63

4.6 63

4.6 67

4.7 67

4.7 80

4.8 80

4.8 60

4.9 60

4.9 66

5 66

5 62

5.1 62

5.1 67

5.2 67

5.2 64

5.3 64

5.3 58

5.4 58

5.4 39

5.5 39

5.5 41

5.6 41

5.6 29

5.7 29

5.7 31

5.8 31

5.8 20

5.9 20

5.9 15

6 15

6 12

6.1 12

6.1 6

6.2 6

6.2 6

6.3 6

6.3 2

6.4 2

6.4 0

6.5 0

6.5 3


68/70


69/70

Q&A for EstimatingSDErrors.xls

1) Run a 100-observation Monte Carlo simulation of the RMSE and SD of the Residuals.Take a picture of your results and paste them in a Word document.

2) Compare your results from Question 1 to Figures 15.5.2 and 15.5.3.What do these three Monte Carlo simulations suggest about the RMSE as an estimator of the SD of th

Let us see how the estimated SE of the sample slope performs as an estimator of the exact SE.3) First, setn=10 in the Data sheet and compute the exact SE (using the usual formula).

Show your work.

4) Now, run a Monte Carlo simulation with n = 10 in which you track both the estimated SE of the samp

How does the estimated SE of the sample slope perform? How does the RMSE perform? What is thebetween the two statistics?

Be sure to note the number of observations in the Monte Carlo results sheet.

5) Repeat the same process, changing n to 20 and then 40, and tracking the estimated SE of the slope

Did things improve? What do you conclude about the effects of increasing the sample size on

the estimated SE of the sample slope?

6) Demonstrate that the RMSE squared is an unbiased estimator of the Variance of the errors.On the Data sheet compute the RMSE squared. Then run a Monte Carlo experiment in which you trackthe value of RMSE squared. Set n = 10, and run 10,000 repetitions. Use both the Normal distribution

and the Exponential Distribution for the error terms. Comment on your results.

7) If we know that the RMSE is biased toward being too small, why can we not apply some adjustment ffix the bias? To answer this question, compare the sampling distribution for the RMSE with normal errosampling distribution for the RMSE with exponential errors. Why can't we use a single adjustment factodepends only on n and k?


70/70

box?

le slope and the RMSE.

elationship

and RMSE.

actor tors versus ther which

estimating sd errors

Documents