estimating sd errors
Post on 03-Jun-2018
232 Views
Preview:
TRANSCRIPT
-
8/12/2019 Estimating Sd Errors
1/70
Intro to EstimatingSDErrors.xls.This workbook demonstrates three facts about the estimation of the SD of the errors:
1) Estimating the SD of the Errors with just the SD of the Residuals gives a biased estimato2) Estimating the SD of the Errors with the RMSE (the SD of the Residuals adjusted by a de
is better than using just the SD of the Residuals.
3) Estimating the SD of the Errors with the RMSE gives a biased, but consistent estimator.
Because the RMSE is a consistent estimator of the SD of the errors, it is commonly used as an estimat
The data generation process is the classical econometric box model in the bivariate case:
Y = b0+ b1X + e
where eis iid, the Xs are fixed in repeated sampling and independent of the errors.
The Data sheet implements this DGP and can be used to run Monte Carlo simulations with the MCSimThe n sheets have Monte Carlo results.
The Q&A sheet has a few practice problems.
-
8/12/2019 Estimating Sd Errors
2/70
.grees of freedom factor)
of the SD of the errors.
add-in.
-
8/12/2019 Estimating Sd Errors
3/70
DGP is CEM: Y = b0+ b1X + e In order to compare ap
e~N(0,SD Error), iid chosen, the SD of X reb0 0 you can compare the e
b1 5 n 10
SD Error 5
1 1.000
X Error Y
0.174 -2.178 -1.307 slope 5.59 -2.72 intercept0.522 -11.201 -8.589 est SE 1.76 3.53 est SE0.870 -1.568 2.784 R square 0.56 5.568 RMSE1.219 1.896 7.989 F 10.10 8 df 1.567 0.324 8.157 Reg SS 313.00 248.01 SSR1.915 9.267 18.8422.263 -5.920 5.3952.611 -0.870 12.186
2.959 -3.415 11.3813.307 -3.213 13.324
Predicted Y = b0+ b1X
y = 5.59x - 2.72
-10
-5
0
5
10
15
20
25
0 1 2 3
-
8/12/2019 Estimating Sd Errors
4/70
ples to apples, no matter the value of n
mains constant. With the SD of X fixed,
ffect of n on the SE of the sample slope.
5.568 RMSE by formula4.980 SD Residuals See ExponentialDist.xls in the \Basic Tools\RandomNu
Residuals to learn about the distribution of the errors associated wi
0.442 To return to drawing errors from the Normal Distribution-8.788 click on the Set N button at cell D2.0.6383.8942.115
10.852-4.5430.300
-2.452-2.457
-
8/12/2019 Estimating Sd Errors
5/70
ber folder
h this button.
-
8/12/2019 Estimating Sd Errors
6/70
-
8/12/2019 Estimating Sd Errors
7/70
-
8/12/2019 Estimating Sd Errors
8/70
-
8/12/2019 Estimating Sd Errors
9/70
-
8/12/2019 Estimating Sd Errors
10/70
-
8/12/2019 Estimating Sd Errors
11/70
-
8/12/2019 Estimating Sd Errors
12/70
-
8/12/2019 Estimating Sd Errors
13/70
-
8/12/2019 Estimating Sd Errors
14/70
t
-
8/12/2019 Estimating Sd Errors
15/70
Results of Monte Carlo SimulationSample
Number
Data!$F$
11 Data!$J$7
1 1.481 4.683 1000 repetitions2 1.142 3.613 4 secs
3 1.319 4.170 3.1623Average 1.520
4 1.622 5.128 10SD 0.3776
5 1.786 5.649 Max 2.852
6 1.325 4.189 Min 0.576
7 1.500 4.744
8 1.182 3.737
9 1.144 3.617
10 1.175 3.716
11 0.888 2.80712 1.852 5.858
13 0.899 2.841
14 0.700 2.214
15 1.431 4.525
16 1.443 4.562
17 1.515 4.791 1
18 2.081 6.580
19 0.576 1.822
20 1.465 4.632
21 1.371 4.33622 1.390 4.394
23 2.135 6.751
24 2.242 7.089
25 1.208 3.820
26 1.213 3.836
27 0.973 3.076
28 1.253 3.963
29 0.947 2.993
30 1.066 3.370
31 1.590 5.02732 1.963 6.209
33 1.480 4.680
34 2.033 6.430
35 0.803 2.538
36 1.639 5.182
37 1.135 3.591
Data!$F$11
Simulation Stats
0.5 2.5
Histog
-
8/12/2019 Estimating Sd Errors
16/70
38 1.683 5.323
39 1.730 5.471
40 1.757 5.555
41 1.415 4.474
42 1.480 4.679
43 1.173 3.711
44 1.639 5.184
45 1.409 4.457
46 1.756 5.554
47 1.114 3.523
48 1.610 5.091
49 1.713 5.416
50 1.619 5.119
51 1.487 4.702
52 1.492 4.718
53 1.652 5.22354 0.907 2.868
55 1.091 3.450
56 1.670 5.280
57 2.245 7.099
58 1.639 5.184
59 0.910 2.878
60 1.819 5.753
61 1.294 4.092
62 1.174 3.712
63 0.963 3.04764 2.240 7.084
65 1.579 4.992
66 1.171 3.702
67 2.062 6.521
68 1.157 3.660
69 1.181 3.733
70 1.537 4.862
71 1.447 4.576
72 1.705 5.393
73 1.215 3.84474 1.237 3.912
75 1.012 3.199
76 1.397 4.417
77 1.517 4.796
78 1.133 3.583
79 1.640 5.186
-
8/12/2019 Estimating Sd Errors
17/70
80 1.659 5.246
81 1.692 5.352
82 1.450 4.586
83 1.178 3.724
84 1.225 3.873
85 1.668 5.273
86 1.435 4.537
87 1.182 3.736
88 1.155 3.654
89 1.265 4.001
90 1.832 5.795
91 1.421 4.493
92 1.518 4.800
93 1.315 4.158
94 1.781 5.632
95 1.184 3.74396 1.285 4.062
97 1.983 6.270
98 1.404 4.441
99 1.863 5.890
100 1.991 6.297
Only the first 100 repetitions are displayed on this worksheet.
-
8/12/2019 Estimating Sd Errors
18/70
Average 4.808n = 10
SD 1.1942
Max 9.018
Min 1.822
Data!$J$7 Notes
4.5 6.5 8.5
am of Data!$F$11 And Data!$J$7
Data!$F$11
Data!$J$7
-
8/12/2019 Estimating Sd Errors
19/70
-
8/12/2019 Estimating Sd Errors
20/70
-
8/12/2019 Estimating Sd Errors
21/70
-
8/12/2019 Estimating Sd Errors
22/70
-
8/12/2019 Estimating Sd Errors
23/70
-
8/12/2019 Estimating Sd Errors
24/70
-
8/12/2019 Estimating Sd Errors
25/70
-
8/12/2019 Estimating Sd Errors
26/70
-
8/12/2019 Estimating Sd Errors
27/70
Data!$F$1 Data!$J$7
0.5 0.5 0 0
0.75 0.5 10 01 0.75 10 0
1.25 0.75 52 0
1.5 1 52 0
1.75 1 196 0
2 1.25 196 0
2.25 1.25 248 0
2.5 1.5 248 0
2.75 1.5 230 0
3 1.75 230 0
1.75 146 32 146 3
2 84 4
2.25 84 4
2.25 25 8
2.5 25 8
2.5 6 11
2.75 6 11
2.75 3 22
3 3 22
3 0 303.25 30
3.25 49
3.5 49
3.5 73
3.75 73
3.75 71
4 71
4 77
4.25 77
4.25 874.5 87
4.5 74
4.75 74
4.75 77
5 77
5 80
-
8/12/2019 Estimating Sd Errors
28/70
5.25 80
5.25 60
5.5 60
5.5 67
5.75 67
5.75 56
6 56
6 25
6.25 25
6.25 26
6.5 26
6.5 32
6.75 32
6.75 24
7 24
7 177.25 17
7.25 11
7.5 11
7.5 4
7.75 4
7.75 3
8 3
8 4
8.25 4
8.25 18.5 1
8.5 1
8.75 1
8.75 2
9 2
9 1
9.25 1
9.25 0
-
8/12/2019 Estimating Sd Errors
29/70
Results of Monte Carlo SimulationSample
Number
Data!$F$
11 Data!$J$7
1 1.114 4.980 1000 repetitions2 1.311 5.865 8 secs
3 1.299 5.810 Average 1.091
4 1.045 4.675 SD 0.1829
5 1.274 5.697 Max 1.620
6 1.268 5.670 Min 0.634
7 1.420 6.349
8 1.313 5.871
9 1.208 5.403
10 1.155 5.163
11 1.194 5.34112 0.930 4.161
13 1.256 5.618
14 1.017 4.546
15 1.152 5.150
16 1.065 4.761
17 0.731 3.270 1
18 1.204 5.383
19 0.918 4.104
20 1.463 6.543
21 1.260 5.63622 1.223 5.472
23 1.151 5.147
24 0.944 4.220
25 0.957 4.282
26 1.087 4.861
27 1.277 5.711
28 1.196 5.347
29 1.235 5.524
30 1.098 4.910
31 1.042 4.66232 0.690 3.086
33 0.894 3.998
34 0.900 4.023
35 0.771 3.446
36 1.132 5.061
37 1.020 4.561
Data!$F$11
Simulation Stats
0.6 1.6
Histog
-
8/12/2019 Estimating Sd Errors
30/70
38 1.176 5.261
39 1.056 4.724
40 1.100 4.917
41 1.060 4.742
42 1.150 5.143
43 1.112 4.971
44 1.299 5.809
45 0.940 4.204
46 1.293 5.782
47 1.087 4.861
48 1.175 5.257
49 0.809 3.619
50 0.946 4.230
51 1.297 5.798
52 1.066 4.769
53 0.784 3.50454 1.169 5.230
55 1.229 5.498
56 1.297 5.802
57 0.949 4.246
58 1.238 5.536
59 0.735 3.287
60 0.962 4.301
61 1.345 6.017
62 1.120 5.010
63 0.896 4.00964 0.878 3.927
65 0.875 3.913
66 1.230 5.500
67 1.154 5.161
68 0.898 4.018
69 1.092 4.884
70 1.332 5.959
71 1.391 6.220
72 0.867 3.879
73 0.812 3.63274 0.946 4.231
75 1.041 4.654
76 0.996 4.453
77 1.003 4.487
78 1.167 5.220
79 1.254 5.606
-
8/12/2019 Estimating Sd Errors
31/70
80 0.868 3.883
81 1.080 4.831
82 1.237 5.533
83 1.003 4.485
84 1.239 5.541
85 1.149 5.140
86 1.185 5.301
87 1.213 5.424
88 1.116 4.993
89 1.106 4.946
90 1.148 5.133
91 1.442 6.449
92 1.179 5.274
93 1.156 5.170
94 1.079 4.826
95 0.982 4.39496 1.298 5.807
97 0.977 4.368
98 1.171 5.238
99 1.215 5.435
100 0.639 2.858
Only the first 100 repetitions are displayed on this worksheet.
-
8/12/2019 Estimating Sd Errors
32/70
-
8/12/2019 Estimating Sd Errors
33/70
Average 4.881
SD 0.8179
Max 7.246
Min 2.835
Data!$J$7 Notes
.6 3.6 4.6 5.6 6.6
am of Data!$F$11 And Data!$J$7
Data!$F$11
Data!$J$7
-
8/12/2019 Estimating Sd Errors
34/70
-
8/12/2019 Estimating Sd Errors
35/70
-
8/12/2019 Estimating Sd Errors
36/70
-
8/12/2019 Estimating Sd Errors
37/70
-
8/12/2019 Estimating Sd Errors
38/70
-
8/12/2019 Estimating Sd Errors
39/70
-
8/12/2019 Estimating Sd Errors
40/70
-
8/12/2019 Estimating Sd Errors
41/70
-
8/12/2019 Estimating Sd Errors
42/70
-
8/12/2019 Estimating Sd Errors
43/70
-
8/12/2019 Estimating Sd Errors
44/70
-
8/12/2019 Estimating Sd Errors
45/70
Data!$F$1 Data!$J$7
0.6 0.6 0 0
0.7 0.6 14 00.8 0.7 14 0
0.9 0.7 38 0
1 0.8 38 0
1.1 0.8 95 0
1.2 0.9 95 0
1.3 0.9 176 0
1.4 1 176 0
1.5 1 220 0
1.6 1.1 220 0
1.7 1.1 176 01.2 176 0
1.2 157 0
1.3 157 0
1.3 70 0
1.4 70 0
1.4 37 0
1.5 37 0
1.5 14 0
1.6 14 0
1.6 3 01.7 3 0
1.7 0 0
1.8 0
1.8 0
1.9 0
1.9 0
2 0
2 0
2.1 0
2.1 02.2 0
2.2 0
2.3 0
2.3 0
2.4 0
2.4 0
-
8/12/2019 Estimating Sd Errors
46/70
2.5 0
2.5 0
2.6 0
2.6 0
2.7 0
2.7 0
2.8 0
2.8 3
2.9 3
2.9 2
3 2
3 8
3.1 8
3.1 5
3.2 5
3.2 83.3 8
3.3 8
3.4 8
3.4 8
3.5 8
3.5 12
3.6 12
3.6 19
3.7 19
3.7 153.8 15
3.8 19
3.9 19
3.9 25
4 25
4 41
4.1 41
4.1 24
4.2 24
4.2 43
4.3 43
4.3 47
4.4 47
4.4 49
4.5 49
4.5 45
-
8/12/2019 Estimating Sd Errors
47/70
4.6 45
4.6 50
4.7 50
4.7 46
4.8 46
4.8 55
4.9 55
4.9 48
5 48
5 29
5.1 29
5.1 51
5.2 51
5.2 34
5.3 34
5.3 35
5.4 35
5.4 39
5.5 39
5.5 48
5.6 48
5.6 25
5.7 25
5.7 28
5.8 28
5.8 21
5.9 21
5.9 13
6 13
6 23
6.1 23
6.1 14
6.2 14
6.2 9
6.3 9
6.3 10
6.4 10
6.4 12
6.5 12
6.5 8
6.6 8
6.6 4
-
8/12/2019 Estimating Sd Errors
48/70
6.7 4
6.7 6
6.8 6
6.8 2
6.9 2
6.9 4
7 4
7 0
7.1 0
7.1 4
7.2 4
7.2 1
7.3 1
7.3 0
-
8/12/2019 Estimating Sd Errors
49/70
Results of Monte Carlo SimulationSample
Number
Data!$F$
11 Data!$J$7
1 0.753 4.763 1000 repetitions2 1.005 6.355 15 secs
3 0.852 5.391 Average 0.784
4 0.818 5.176 SD 0.0867
5 0.708 4.479 Max 1.072
6 0.846 5.353 Min 0.524
7 0.764 4.829
8 0.768 4.859
9 0.729 4.614
10 0.780 4.932
11 0.790 4.99412 0.729 4.608
13 0.900 5.695
14 0.853 5.392
15 0.680 4.304
16 0.777 4.914
17 0.718 4.542 1
18 0.923 5.836
19 0.848 5.365
20 0.704 4.453
21 0.737 4.66022 0.653 4.128
23 0.865 5.470
24 0.862 5.449
25 0.799 5.054
26 0.835 5.279
27 0.801 5.065
28 0.780 4.935
29 0.954 6.031
30 0.600 3.795
31 0.718 4.54132 0.747 4.725
33 0.823 5.207
34 0.705 4.462
35 0.727 4.598
36 0.879 5.558
37 0.749 4.735
Data!$F$11
Simulation Stats
0.5 1.5
Histog
-
8/12/2019 Estimating Sd Errors
50/70
38 0.727 4.596
39 0.827 5.232
40 0.772 4.880
41 0.614 3.885
42 0.796 5.032
43 0.680 4.299
44 0.793 5.014
45 0.710 4.493
46 0.747 4.726
47 0.832 5.263
48 0.761 4.812
49 0.706 4.464
50 0.778 4.919
51 0.654 4.134
52 0.764 4.830
53 0.722 4.56654 0.853 5.395
55 0.758 4.795
56 0.726 4.590
57 0.794 5.021
58 0.738 4.667
59 0.722 4.566
60 0.681 4.308
61 0.800 5.058
62 0.920 5.819
63 0.666 4.21364 0.807 5.105
65 0.612 3.871
66 0.718 4.541
67 0.753 4.759
68 0.720 4.551
69 0.801 5.067
70 0.686 4.341
71 0.677 4.284
72 0.792 5.007
73 0.908 5.74274 0.794 5.024
75 0.849 5.373
76 0.819 5.182
77 0.752 4.759
78 0.772 4.881
79 0.764 4.831
-
8/12/2019 Estimating Sd Errors
51/70
80 0.754 4.770
81 0.754 4.766
82 0.809 5.118
83 0.717 4.537
84 0.799 5.051
85 0.724 4.580
86 0.873 5.521
87 0.744 4.707
88 0.823 5.204
89 0.770 4.872
90 0.830 5.249
91 0.761 4.813
92 0.660 4.177
93 0.657 4.155
94 0.790 4.997
95 0.718 4.54096 0.929 5.877
97 0.855 5.410
98 0.742 4.695
99 0.993 6.282
100 0.904 5.716
Only the first 100 repetitions are displayed on this worksheet.
-
8/12/2019 Estimating Sd Errors
52/70
-
8/12/2019 Estimating Sd Errors
53/70
Average 4.960
SD 0.5486
Max 6.778
Min 3.314
Data!$J$7 Notes
2.5 3.5 4.5 5.5 6.5
am of Data!$F$11 And Data!$J$7
Data!$F$11
Data!$J$7
-
8/12/2019 Estimating Sd Errors
54/70
-
8/12/2019 Estimating Sd Errors
55/70
-
8/12/2019 Estimating Sd Errors
56/70
-
8/12/2019 Estimating Sd Errors
57/70
-
8/12/2019 Estimating Sd Errors
58/70
-
8/12/2019 Estimating Sd Errors
59/70
-
8/12/2019 Estimating Sd Errors
60/70
-
8/12/2019 Estimating Sd Errors
61/70
-
8/12/2019 Estimating Sd Errors
62/70
-
8/12/2019 Estimating Sd Errors
63/70
-
8/12/2019 Estimating Sd Errors
64/70
-
8/12/2019 Estimating Sd Errors
65/70
Data!$F$1 Data!$J$7
0.5 0.5 0 0
0.6 0.5 8 00.7 0.6 8 0
0.8 0.6 151 0
0.9 0.7 151 0
1 0.7 420 0
1.1 0.8 420 0
0.8 321 0
0.9 321 0
0.9 92 0
1 92 0
1 8 01.1 8 0
1.1 0 0
1.2 0
1.2 0
1.3 0
1.3 0
1.4 0
1.4 0
1.5 0
1.5 01.6 0
1.6 0
1.7 0
1.7 0
1.8 0
1.8 0
1.9 0
1.9 0
2 0
2 02.1 0
2.1 0
2.2 0
2.2 0
2.3 0
2.3 0
-
8/12/2019 Estimating Sd Errors
66/70
2.4 0
2.4 0
2.5 0
2.5 0
2.6 0
2.6 0
2.7 0
2.7 0
2.8 0
2.8 0
2.9 0
2.9 0
3 0
3 0
3.1 0
3.1 03.2 0
3.2 0
3.3 0
3.3 2
3.4 2
3.4 0
3.5 0
3.5 0
3.6 0
3.6 13.7 1
3.7 6
3.8 6
3.8 9
3.9 9
3.9 18
4 18
4 18
4.1 18
4.1 24
4.2 24
4.2 30
4.3 30
4.3 41
4.4 41
4.4 57
-
8/12/2019 Estimating Sd Errors
67/70
4.5 57
4.5 63
4.6 63
4.6 67
4.7 67
4.7 80
4.8 80
4.8 60
4.9 60
4.9 66
5 66
5 62
5.1 62
5.1 67
5.2 67
5.2 64
5.3 64
5.3 58
5.4 58
5.4 39
5.5 39
5.5 41
5.6 41
5.6 29
5.7 29
5.7 31
5.8 31
5.8 20
5.9 20
5.9 15
6 15
6 12
6.1 12
6.1 6
6.2 6
6.2 6
6.3 6
6.3 2
6.4 2
6.4 0
6.5 0
6.5 3
-
8/12/2019 Estimating Sd Errors
68/70
-
8/12/2019 Estimating Sd Errors
69/70
Q&A for EstimatingSDErrors.xls
1) Run a 100-observation Monte Carlo simulation of the RMSE and SD of the Residuals.Take a picture of your results and paste them in a Word document.
2) Compare your results from Question 1 to Figures 15.5.2 and 15.5.3.What do these three Monte Carlo simulations suggest about the RMSE as an estimator of the SD of th
Let us see how the estimated SE of the sample slope performs as an estimator of the exact SE.3) First, setn=10 in the Data sheet and compute the exact SE (using the usual formula).
Show your work.
4) Now, run a Monte Carlo simulation with n = 10 in which you track both the estimated SE of the samp
How does the estimated SE of the sample slope perform? How does the RMSE perform? What is thebetween the two statistics?
Be sure to note the number of observations in the Monte Carlo results sheet.
5) Repeat the same process, changing n to 20 and then 40, and tracking the estimated SE of the slope
Did things improve? What do you conclude about the effects of increasing the sample size on
the estimated SE of the sample slope?
6) Demonstrate that the RMSE squared is an unbiased estimator of the Variance of the errors.On the Data sheet compute the RMSE squared. Then run a Monte Carlo experiment in which you trackthe value of RMSE squared. Set n = 10, and run 10,000 repetitions. Use both the Normal distribution
and the Exponential Distribution for the error terms. Comment on your results.
7) If we know that the RMSE is biased toward being too small, why can we not apply some adjustment ffix the bias? To answer this question, compare the sampling distribution for the RMSE with normal errosampling distribution for the RMSE with exponential errors. Why can't we use a single adjustment factodepends only on n and k?
-
8/12/2019 Estimating Sd Errors
70/70
box?
le slope and the RMSE.
elationship
and RMSE.
actor tors versus ther which
top related