econometrics - regression powerpoint

7/30/2019 Econometrics - Regression Powerpoint

1/18

Multiple Regression Analysis

OLS AsymptoticsCh. 5


2/18

uXy

]xxiX k 1[

0)()|( X|xx k 1 u E u E

Earlier we covered finite sample (or exact) properties of the OLS estimators. We had the following results:

Assumption MLR.1 : the model of interest is linear inparameters,

Assumption MLR.2: Our data are a random sample of n

observationsAssumption MLR.3: Matrix of regressorshas full column rankAssumption MLR.4: Zero conditional mean

Given assumptions MLR1-4, OLS estimator is unbiased:E(b|X)= . This is an exact property, because it holds for any sample size (including small).


3/18

22 )|()|( XX ii u E uV

jiuuCov ji ,0)|,( X

Assumption MLR.5: Spherical disturbances

=> homoskedasticity

With this additional assumption, OLS can be shownto be the best linear unbiased estimator (Gauss-Markov theorem).

This is also an exact property of the OLS.

=> non-autocorrelation


4/18

Assumption MLR.6 : conditional on X, errors are normallydistributed with mean 0 and constant variance 2:

.

This assumption allows making a stronger case aboutOLS estimator: it is the best among all (not just linear)unbiased estimators.

As above, this is an exact property of the OLS.

In addition to making OLS the best unbiased estimator this last assumption also allows us constructing the exactsampling distribution of b which leads to the t and Fdistribution for t and F statistics.


5/18

What will happens to our results if Assumption MLR.6 does not hold?

Fortunately, given all the other assumptions we made, OLShas good large sample properties (or asymptoticproperties).

In particular, t and F statistic have approximately t and F

distribution, at least in large samples.


6/18

Consistency of the OLS

Sometimes an estimator fails to be unbiased. In thosecases it has to be at least consistent:

nasb P n 0)|(|

As sample size growths, the probability of being far fromthe true parameter is approaching zero.

Consistency is a minimal requirement for an estimator.


7/18

Theorem : Under assumptions MLR1-4 OLS estimator isunbiased (as we know) and also consistent.

112

1

1

12

1

1

1 )(

),(

)(1

)(1

)(

)(

xVar

u xCov

x xn

u x xn

x x

y x x

i

n

i

ii

n

i

i

n

i

ii

n

i

0),( u xCov

For the case of simple regression model:

In this derivation we used the fact that

which is a weaker assumption than SLR4 (x and u areindependent).


8/18

nn p

nn p p

nn

uXXX

uXXXb

uXXXuXXXuXXXXyXXXb

11

1111

''lim

''lim)lim(

''')'()(')'(')'(

consistentisOLS 0'

limIf

'lim

''lim)lim(

'thatassumesLet'

1

nlim

n

p

n p

nn p p

n

uX

uXQ

uXXXb

QXX

1

A little bit of matrix algebra to deal with the multiple linear regression case:


9/18

00lim')|('1|'V2.

0lim0)|('1

|'

E1.

2

2 QXX

XXuXXuX

XuXXuX

V nn

V nn

E E nn

0

'

lim

n p

uX

0QuXXX

b1

1''lim)lim(nn

p p

consistentisOLS


10/18

Previously we have seen that if , OLSestimator is not unbiased.

Now we can add that if any of the independent variablesare correlated with the error term, the OLS will also be

inconsistent!

This is very bad: there is bias in small samples and it doesnot go away as we increase sample size.

0)|( Xu E


11/18

Hypothesis testing without MLR 6

Target: Assumption MLR 6

Assumption MLR 6 plays crucial role in hypothesis testing:

What happens to the distribution of t and F statistics whenthe errors are not normal?

)1,(2)1(

2

)1(2)1(1

~)1/(

/

~)1/()(

)][,(~),(~

k n J k n

J

k n

k n j

j j

F k n

J F

t k n

SN be s

b

XX'NbI0Nu 22


12/18

If errors are not normally distributed, b-OLS will not be

normally distributed, and b\SE(b) will not have t-distributionand F-statistic will not have exact F distribution.

Theorem: Even thought b-OLS is not normal if assumptionMLR6 does not hold, it is asymptotically normal =>approximately normal in large enough samples.


13/18

QXX

uXXXbuXXXbuXXXb

111

n

nnn

' thatassumption same theuse We

'')(')'(')'(

),(' 2Q0uX N n

d

More matrix algebra:

Using the central limit theorem it can be shown that

.


14/18

)][,()][,(),(1

),(),(''

)(

?]'[

121212

1221

12

XX

1

QbQ0Q0b

Q0Q0QuXXX

b

s

aa

d

n N n N N n

N N nn

n

2

22 )lim( s p

1XX )'(2 s

It can be shown that s2 is a consistent estimator of

(in addition to being unbiased. HW problem).

Also, since XX -> nQ, we can use

as an appropriate estimator of the covariance matrix.

as an


15/18

From these results it follows that

)1,0()(/)())var(,( N b sebb N ba

j j j j j

a

j

Where as before:

And the R-squared is from regressing xj on all the other independent variables. Now: Take the square root out of the asymptotic variance of bj Estimate with s

And we have the estimated asymptotic standard error of bj:

s(bj).which can be used to do hypotheses testing. All our stepswill be as before, the difference is that now the results willbe asymptotic (valid for large samples).

n

i jij j

j j j x xSST RSST

bVar 1

22

2

)(,)1(

)(


16/18

From these results it follows that

)1,0()(/)( N se

d

j j j

(use t table for testing, since it shows critical values for standard normal when n is large).

How about the F-statistic? It has an approximate F-distribution in large samples. So, we use the same steps aswe did before to perform hypothesis testing.

To show the asymptotic normality of b-OLS we do need toassume homoskedasticity (same error variance for allobservations).The heteroskedasticity case will bediscussed in our next topic.


17/18

Finally, it can be shown that under assumptions MLR 1-5(Gauss-Markov assumptions) b-OLS is asympto t ica l ly efficient among a certain class of estimators, B:

To claim the asymptotic normality and efficiency of b-OLS we do need assumptions MLR1-5. In particular, weNEED to assume homoskedasticity (same error variancefor all observations).

The case of heteroskedasticity is what we will discussnext.

)()( BV AsymptoticbV Asymptotic OLS


18/18

OLS AsymptoticsMLR 1,2,3,4 => b-OLS is unbiased AND consistentMLR 1,2,3,4 + MLR 5 => b-OLS is BLUE

MLR 1,2,3,4 + MLR 5 + MLR 6 => b-OLS is BUE

b-OLS Unb, Lin Unbiased MLR 1-5 MLR 1-6

B1 Unb, Lin Unb, Lin b-OLS b-OLS

B2 Unb, Lin Unb, Lin b-OLS b-OLS

B3 Bsd, Nlin

B4 Unb, Nlin Unb, Nlin b-OLSB5 Unb, Lin Unb, Lin b-OLS b-OLS

B6 Bsd, Lin

B7 Unb, Nlin Unb, Nlin b-OLS

econometrics - regression powerpoint

Documents