chapter 10 random regressors and moment-based estimation

Principles of Econometrics, 4th Edition Chapter 10: Random Regressors and

Moment-Based Estimation

Chapter 10Random Regressors and


Walter R. Paczkowski Rutgers University



10.1 Linear Regression with Random x’s10.2 Cases in Which x and e are Correlated10.3 Estimators Based on the Method of

Moments10.4 Specification Tests

Chapter Contents



We relax the assumption that variable x is not random



10.1 Linear Regression with Random x’s



Modified simple regression assumptions:

10.1Linear Regression with Random x’s

A10.1 yi = β1 + β2xi + ei correctly describes the relationship

between yi and xi in the population, where β1 and β2 are unknown

(fixed) parameters and ei is an unobservable random error term.

A10.2 The data pairs (xi, yi), i = 1, …, N, are obtained by random

sampling. That is, the data pairs are collected from the same

population, by a process in which each pair is independent of every

other pair. Such data are said to be independent and identically

distributed.



Modified simple regression assumptions (Continued):


A10.3 The expected value of the error term e, conditional on the value

of x, is zero.

If E(e|x) = 0, then we can show that it is also true that x and e are uncorrelated, and that cov(x, e) = 0. Explanatory variables that are not correlated with the error term are called exogenous variables.

Conversely, if x and e are correlated, then cov(x, e) ≠ 0 and we can show that E(e|x) ≠ 0. Explanatory variables that are

correlated with the error term are called endogenous variables.

A10.4 In the sample, x must take at least two different values.



Modified simple regression assumptions (Continued):


A10.5 var(e|x) = σ2. The variance of the error term, conditional on any x, is a constant σ2.

A10.6 The distribution of the error term is normal.



Assumption A10.2 states that both y and x are obtained by a sampling process, and thus are random– This is the only one new assumption on our list




The result that under the classical assumptions, and fixed x’s, the least squares estimator is the best linear unbiased estimator, is a finite sample, or a small sample– This means is that the result does not depend on

the size of the sample


10.1.1The Small Sample Properties of the

Least Squares Estimators



Under assumptions A10.1–A10.6:1. The least squares estimator is unbiased2. The least squares estimator is the best linear

unbiased estimator of the regression parameters, and the usual estimator of σ2 is unbiased

3. The distributions of the least squares estimators, conditional upon the x’s, are normal, and their variances are estimated in the usual way • The usual interval estimation and hypothesis

testing procedures are valid






If x is random, as long as the data are obtained by random sampling and the other usual assumptions hold, no changes in our regression methods are required






For the purposes of a ‘‘large sample’’ analysis of the least squares estimator, it is convenient to replace assumption A10.3 by:

A10.3* E(e) = 0 and cov(x, e) = 0


10.1.2Large Sample

Properties of the Least Squares

Estimators



Now we can say: – Under assumptions A10.1, A10.2, A10.3*, A10.4, and A10.5, the least

squares estimators:1. Are consistent.

– They converge in probability to the true parameter values as N→∞.

2. Have approximate normal distributions in large samples, whether the errors are normally distributed or not. – Our usual interval estimators and test statistics are valid, if the

sample is large.3. If assumption A10.3* is not true, and in particular if cov(x,e) ≠ 0

so that x and e are correlated, then the least squares estimators are inconsistent. – They do not converge to the true parameter values even in very

large samples. – None of our usual hypothesis testing or interval estimation

procedures are valid.


10.1.2Large Sample

Properties of the Least Squares

Estimators




10.1.3Why Least Squares

Estimation Fails

FIGURE 10.1 (a) Correlated x and e





Estimation Fails

FIGURE 10.1 (b) Plot of data, true and fitted regression functions



The statistical consequences of correlation between x and e is that the least squares estimator is biased — and this bias will not disappear no matter how large the sample – Consequently the least squares estimator is

inconsistent when there is correlation between x and e



Estimation Fails



10.2 Cases in Which x and e are

Correlated



When an explanatory variable and the error term are correlated, the explanatory variable is said to be endogenous – This term comes from simultaneous equations

models• It means ‘‘determined within the system’’

– Using this terminology when an explanatory variable is correlated with the regression error, one is said to have an ‘‘endogeneity problem’’

10.2Cases in Which x

and e are Correlated



The errors-in-variables problem occurs when an explanatory variable is measured with error – If we measure an explanatory variable with

error, then it is correlated with the error term, and the least squares estimator is inconsistent



10.2.1Measurement Error



Let y = annual savings and x* = the permanent annual income of a person – A simple regression model is:

– Current income is a measure of permanent income, but it does not measure permanent income exactly.• It is sometimes called a proxy variable • To capture this feature, specify that:




*1 2i i iy x v Eq. 10.1

*i i ix x u Eq. 10.2



Substituting:




*1 2

1 2

1 2 2

1 2

iy x v

x u v

x v u

x e

Eq. 10.3



In order to estimate Eq. 10.3 by least squares, we must determine whether or not x is uncorrelated with the random disturbance e – The covariance between these two random

variables, using the fact that E(e) = 0, is:




*2

2 22 2

cov ,

0u

x e E xe E x u v u

E u

Eq. 10.4



The least squares estimator b2 is an inconsistent estimator of β2 because of the correlation between the explanatory variable and the error term– Consequently, b2 does not converge to β2 in

large samples– In large or small samples b2 is not

approximately normal with mean β2 and variance




22var b x x



Another situation in which an explanatory variable is correlated with the regression error term arises in simultaneous equations models– Suppose we write:



10.2.2Simultaneous

Equations Bias

1 2Q P e Eq. 10.5



There is a feedback relationship between P and Q– Because of this, which results because price and

quantity are jointly, or simultaneously, determined, we can show that cov(P, e) ≠ 0

– The resulting bias (and inconsistency) is called the simultaneous equations bias



10.2.2Simultaneous

Equations Bias



When an omitted variable is correlated with an included explanatory variable, then the regression error will be correlated with the explanatory variable, making it endogenous



10.2.3Omitted Variables



Consider a log-linear regression model explaining observed hourly wage:

–What else affects wages? What have we omitted?




21 2 3 4ln β β β βWAGE EDUC EXPER EXPER e Eq. 10.6



We might expect cov(EDUC, e) ≠ 0– If this is true, then we can expect that the least

squares estimator of the returns to another year of education will be positively biased,

E(b2) > β2, and inconsistent• The bias will not disappear even in very

large samples






Estimating our wage equation, we have:

–We estimate that an additional year of education increases wages approximately 10.75%, holding everything else constant • If ability has a positive effect on wages, then

this estimate is overstated, as the contribution of ability is attributed to the education variable



10.2.4Least Squares

Estimation of a Wage Equation

2ln 0.5220 0.1075 0.0416 0.0008

se 0.1986 0.0141 0.0132 0.0004

WAGE EDUC EXPER EXPER



10.3 Estimators Based on the Method of

Moments



When all the usual assumptions of the linear model hold, the method of moments leads to the least squares estimator – If x is random and correlated with the error

term, the method of moments leads to an alternative, called instrumental variables estimation, or two-stage least squares estimation, that will work in large samples

10.3Estimators Based on

the Method of Moments



The kth moment of a random variable Y is the expected value of the random variable raised to the kth power:

– The kth population moment in Eq. 10.7 can be estimated consistently using the sample (of size N) analog:

10.3.1Method of Moments

Estimation of a Population Mean

and Variance



th moment of kkE Y k Y Eq. 10.7

thˆ sample moment of k kk iE Y k Y y N Eq. 10.8



The method of moments estimation procedure equates m population moments to m sample moments to estimate m unknown parameters– Example:



and Variance



Eq. 10.9 22 2 2var Y E Y E Y



The first two population and sample moments of Y are:



and Variance



Eq. 10.10

1

2 22 2

Population Moments Sample Momentsˆ

ˆi

i

E Y y N

E Y y N



Solve for the unknown mean and variance parameters:

and



and Variance



Eq. 10.11 ˆ iy N y

22 2 22 2 2

2ˆ ˆ ii i y yy y Nyy

N N N

Eq. 10.12



In the linear regression model y = β1 + β2x + e, we usually assume:

– If x is fixed, or random but not correlated with e, then:


Estimation in the Simple Linear

Regression Model



Eq. 10.13

Eq. 10.14

1 20 0i i iE e E y x

1 20 0E xe E x y x



We have two equations in two unknowns:



Regression Model



Eq. 10.15

1 2

1 2

1 0

1 0

i i

i i i

y b b xN

x y b b xN



These are equivalent to the least squares normal equations and their solution is:

– Under "nice" assumptions, the method of moments principle of estimation leads us to the same estimators for the simple linear regression model as the least squares principle



Regression Model



Eq. 10.16

2 2

1 2

i i

i

x x y yb

x x

b y b x



Suppose that there is another variable, z, such that:1. z does not have a direct effect on y, and thus it

does not belong on the right-hand side of the model as an explanatory variable

2. z is not correlated with the regression error term e• Variables with this property are said to be

exogenous3. z is strongly [or at least not weakly] correlated

with x, the endogenous explanatory variableA variable z with these properties is called an instrumental variable

10.3.3Instrumental

Variables Estimation in the Simple Linear Regression Model





If such a variable z exists, then it can be used to form the moment condition:

– Use Eqs. 10.13 and 10.16, the sample moment conditions are:

10.3.3Instrumental




1 20 0E ze E z y x Eq. 10.16

1 2

1 2

1 ˆ ˆ 0

1 ˆ ˆ 0

i i

i i i

y xN

z y xN

Eq. 10.17



Solving these equations leads us to method of moments estimators, which are usually called the instrumental variable (IV) estimators:

10.3.3Instrumental




Eq. 10.18

2

1 2

ˆ

ˆ ˆ

i ii i i i

i i i i i i

z z y yN z y z yN z x z x z z x x

y x



These new estimators have the following properties:– They are consistent, if z is exogenous, with

E(ze) = 0 – In large samples the instrumental variable

estimators have approximate normal distributions • In the simple regression model:

10.3.3Instrumental




Eq. 10.19

2

2 2 22ˆ ~ ,

zx i

Nr x x



These new estimators have the following properties (Continued):– The error variance is estimated using the

estimator:

10.3.3Instrumental




2

1 22ˆ ˆ

ˆ2

i i

IV

y x

N



Note that we can write the variance of the instrumental variables estimator of β2 as:

– Because the variance of the instrumental variables estimator will always be larger than the variance of the least squares estimator, and thus it is said to be less efficient

10.3.3aThe Importance of

Using Strong Instruments



22

2 2 22

varˆvarzxzx i

brr x x

2 1zxr



To extend our analysis to a more general setting, consider the multiple regression model:

– Let xK be an endogenous variable correlated with the error term

– The first K - 1 variables are exogenous variables that are uncorrelated with the error term e - they are ‘‘included’’ instruments

10.3.4Instrumental

Variables Estimation in the Multiple

Regression Model



1 2 2β β βK Ky x x e



We can estimate this equation in two steps with a least squares estimation in each step

10.3.4Instrumental


Regression Model





The first stage regression has the endogenous variable xK on the left-hand side, and all exogenous and instrumental variables on the right-hand side– The first stage regression is:

– The least squares fitted value is:

10.3.4Instrumental


Regression Model



1 2 2 1 1 1 1K K K L L Kx x x z z v Eq. 10.20

1 2 2 1 1 1 1ˆ ˆˆ ˆ ˆˆK K K L Lx x x z z Eq. 10.21



The second stage regression is based on the original specification:

– The least squares estimators from this equation are the instrumental variables (IV) estimators

– Because they can be obtained by two least squares regressions, they are also popularly known as the two-stage least squares (2SLS) estimators • We will refer to them as IV or 2SLS or

IV/2SLS estimators

10.3.4Instrumental


Regression Model



Eq. 10.22 *1 2 2 ˆβ β βK Ky x x e



The IV/2SLS estimator of the error variance is based on the residuals from the original model:

10.3.4Instrumental


Regression Model



Eq. 10.23 2

1 2 22ˆ ˆ ˆβ β β

σi i K Ki

IV

y x x

N K



In the simple regression, if x is endogenous and we have L instruments:

– The two sample moment conditions are:

10.3.4aUsing Surplus Instruments in

Simple Regression



1 1 1ˆ ˆˆˆ L Lx z z

1 2

1 2

1 ˆ ˆβ β 0

1 ˆ ˆˆ β β 0

i i

i i i

y xN

x y xN



Solving using the fact that , we get:

10.3.4aUsing Surplus Instruments in

Simple Regression



2

1 2

ˆ ˆ ˆβ

ˆˆ ˆ

ˆ ˆβ β

i i i i

i ii i

x x y y x x y yx x x xx x x x

y x

x x



Sometimes we have more instrumental variables at our disposal than are necessary – Suppose we have L = 2 instruments, z1 and z2

– Then we have:

10.3.4bSurplus Moment

Conditions



2 2 1 2β β 0E z e E z y x



We have three sample moment conditions:

10.3.4bSurplus Moment

Conditions



1 2

1 1 2 2

2 1 2 3

1 ˆ ˆ ˆβ β 0

1 ˆ ˆ ˆβ β 0

1 ˆ ˆ ˆβ β 0

i i i

i i i

i i i

y x mN

z y x mN

z y x mN



The first stage regression is a key tool in assessing whether an instrument is ‘‘strong’’ or ‘‘weak’’ in the multiple regression setting

10.3.5Assessing

Instrument Strength Using the First Stage

Model





Suppose the first stage regression equation is:

– The key to assessing the strength of the instrumental variable z1 is the strength of its relationship to xK after controlling for the effects of all the other exogenous variables

10.3.5aOne Instrumental

Variable



1 2 2 1 1 1 1K K K Kx x x z v Eq. 10.24



Suppose the first stage regression equation is:

–We require that at least one of the instruments be strong

10.3.5bMore Than One

Instrumental Variable



Eq. 10.25 1 2 2 1 1 1 1K K K L L Kx x x z z v



Consider the model with an instrumental variable MOTHEREDUC:

10.3.6Instrumental

Variables Estimation of the Wage

Equation



29.7751 0.0489 0.0013 0.2677 se 0.4249 0.0417 0.0012 0.0311EDUC EXPER EXPER MOTHEREDUC

Eq. 10.26



To implement instrumental variables estimation using the two-stage least squares approach, we obtain the predicted values of education from the first stage equation and insert it into the log-linear wage equation to replace EDUC – Then estimate the resulting equation by least

squares

10.3.6Instrumental


Equation





The instrumental variables estimates of the log-linear wage equation are:

10.3.6Instrumental


Equation



2ln 0.1982 0.0493 0.0449 0.0009

se 0.4729 0.0374 0.0136 0.0004

WAGE EDUC EXPER EXPER



Using FATHEREDUC, the first stage equation is:

10.3.6Instrumental


Equation



21 2 3 1 2γ γ γ θ θEDUC EXPER EXPER MOTHEREDUC FATHEREDUC v



10.3.6Instrumental


Equation



Table 10.1 First-Stage Equation



The IV/2SLS estimates are:

10.3.6Instrumental


Equation



2ln 0.0481 0.0614 0.0442 0.0009

se 0.4003 0.0314 0.0134 0.0004

WAGE EDUC EXPER EXPER Eq. 10.27



In a multiple regression model, the coefficients are the effect of a unit change in an explanatory, independent, variable on the expected outcome, holding all other things constant– In calculus terminology, the coefficients are

partial derivatives

10.3.7Partial Correlation





We can net out or partial out the effects of explanatory variables– Regression coefficients can be thought of

measuring the effect of one variable on another after removing, or partialling out, the effects of all other variables

– The sample correlation between two residuals is called the partial correlation coefficient

10.3.7Partial Correlation





The multiple regression model, including all K variables, is:

10.3.8Instrumental

Variables Estimation in a General Model



1 2 2 1 1

G exogenous variables B endogenous variables

G G G G K Ky x x x x e Eq. 10.28



Think of G = Good explanatory variables, B = Bad explanatory variables and L = Lucky instrumental variables– It is a necessary condition for IV estimation that L ≥ B– If L = B then there are just enough instrumental variables

to carry out IV estimation • The model parameters are said to just identified or

exactly identified in this case• The term identified is used to indicate that the model

parameters can be consistently estimated – If L > B then we have more instruments than are

necessary for IV estimation, and the model is said to be overidentified

10.3.8Instrumental






Consider the B first-stage equations:

The predicted values are:

In the second stage of estimation we apply least squares to:

10.3.8Instrumental




1 2 2 1 1 ,

1, ,G j j j Gj G j Lj L jx x x z z v

j B

Eq. 10.29

1 2 2 1 1ˆ ˆˆ ˆ ˆˆ ,

1, ,G j j j Gj G j Lj Lx x x z z

j B

*1 2 2 1 1ˆ ˆG G G G K Ky x x x x e Eq. 10.30



Consider the model with B = 2:

– The first-stage equations are:

10.3.8aAssessing

Instrument Strength in a General Model



Eq. 10.31 1 2 2 1 1 1 1G G G G G Gy x x x x e

1 11 21 2 1 11 1 21 2 1

2 12 22 2 2 12 1 22 2 2

γ γ γ θ θγ γ γ θ θ

G G G

G G G

x x x z z vx x x z z v



When testing the null hypothesis H0: βk = c, use of the test statistic is valid in large samples – It is common, but not universal, practice to use

critical values, and p-values, based on the distribution rather than the more strictly appropriate N(0,1) distribution

– The reason is that tests based on the t-distribution tend to work better in samples of data that are not large

10.3.8bHypothesis Testing with Instrumental

Variables Estimates



ˆ ˆsek kt c



When testing a joint hypothesis, such as H0: β2 = c2, β3 = c3, the test may be based on the chi-square distribution with the number of degrees of freedom equal to the number of hypotheses (J) being tested – The test itself may be called a “Wald” test, or a

likelihood ratio (LR) test, or a Lagrange multiplier (LM) test

– These testing procedures are all asymptotically equivalent

10.3.8bHypothesis Testing with Instrumental

Variables Estimates





Unfortunately R2 can be negative when based on IV estimates– Therefore the use of measures like R2 outside

the context of the least squares estimation should be avoided

10.3.8cGoodness-of-Fit

with Instrumental Variables Estimates





10.4 Specification Tests



1. Can we test for whether x is correlated with the error term?

– This might give us a guide of when to use least squares and when to use IV estimators

2. Can we test if our instrument is valid, and uncorrelated with the regression error, as required?

10.4Specification Tests



The null hypothesis is H0: cov(x, e) = 0 against the alternative H1: cov(x, e) ≠ 0


10.4.1The Hausman Test for Endogeneity



If null hypothesis is true, both the least squares estimator and the instrumental variables estimator are consistent

– Naturally if the null hypothesis is true, use the more efficient estimator, which is the least squares estimator

If the null hypothesis is false, the least squares estimator is not consistent, and the instrumental variables estimator is consistent

– If the null hypothesis is not true, use the instrumental variables estimator, which is consistent





There are several forms of the test, usually called the Hausman test





Consider the model:

– Let z1 and z2 be instrumental variables for x.

1. Estimate the model by least squares, and obtain the residuals .

• If there are more than one explanatory variables that are being tested for endogeneity, repeat this estimation for each one, using all available instrumental variables in each regression


10.4.1The Hausman Test for Endogeneity 1 2y x e

1 1 1 2 2x z z v

1 1 1 2 2ˆ ˆˆv x z z



Consider the model (Continued):

2. Include the residuals computed in step 1 as an explanatory variable in the original regression,

– Estimate this "artificial regression" by least squares, and employ the usual t-test for the hypothesis of significance



1 2 ˆy x v e

0

1

: 0 no correlation between and : 0 correlation between and

H x eH x e



Consider the model (Continued):

3. If more than one variable is being tested for endogeneity, the test will be an F-test of joint significance of the coefficients on the included residuals





A test of the validity of the surplus moment conditions is:

1. Compute the IV estimates using all available instruments, including the G variables x1=1, x2, …, xG that are presumed to be exogenous, and the L instruments

2. Obtain the residuals

3. Regress on all the available instruments described in step 1


10.4.2Testing Instrument

Validity

ˆk

1 2 2ˆˆ .ˆ ˆ

K Ke y x x

e



A test of the validity of the surplus moment conditions is (Continued):

4. Compute NR2 from this regression, where N is the sample size and R2 is the usual goodness-of-fit measure

5. If all of the surplus moment conditions are valid, then

• If the value of the test statistic exceeds the 100(1−α)-percentile from the distribution, then we conclude that at least one of the surplus moment conditions restrictions is not valid


10.4.2Testing Instrument

Validity

2 2( ) .~ L BNR

2( )L B




10.4.3Specification Tests

for the Wage Equation

Table 10.2 Hausman Test Auxiliary Regression



Key Words



asymptotic propertiesconditional expectationendogenous variableserrors-in-variablesexogenous variablesfinite sample propertiesfirst stage regressionHausman test

Keywords

instrumental variableinstrumental variable estimatorjust identified equationslarge sample propertiesover identified equationspopulation momentsrandom sampling

reduced form equationsample momentssimultaneous equations biastest of surplus moment conditionstwo-stage least squares estimationweak instruments



Appendices



We can use the conditional pdf to compute the conditional mean of Y given X:

Similarly we can define the conditional variance of Y given X:

10AConditional and

Iterated Expectations

Eq. 10A.1

10A.1Conditional

Expectations

| | |y y

E Y X x yP Y y X x yf y x

2var | | |

yY X x y E Y X x f y x



The law of iterated expectations says that the expected value of the conditional expectation of Y given X:

10AConditional and


Eq. 10A.2

10A.2Iterated

Expectations

|XE Y E E Y X



We can now show:

10AConditional and


10A.2Iterated

Expectations

,

|

| [by changing order of summation]

|

|

y y x

y x

x y

x

X

E Y yf y y f x y

y f y x f x

yf y x f x

E Y X x f x

E E Y X



Two other results can be shown to be true:

10AConditional and


10A.2Iterated

Expectations

|XE XY E XE Y X

cov , |X XX Y E X E Y X

Eq. 10A.3

Eq. 10A.4



The following can be shown to hold:

10AConditional and


10A.3Regression

Model Application

Eq. 10A.5

Eq. 10A.6

| 0 0i x i i xE e E E e x E

| 0 0i i x i i i x iE x e E x E e x E x

cov , | 0 0i i x i x i i x i xx e E x E e x E x Eq. 10A.7



If E(e|x) = 0 it follows that E(e) = 0, E(xe) = 0, and cov(x, e) = 0– However, if E(e|x) ≠ 0 then cov(x, e) ≠ 0

10AConditional and


10A.3Regression

Model Application



This is an algebraic proof that the least squares estimator is not consistent when cov(x, e) ≠ 0 – The regression model is y = β1 + β2x + e. – Under Eq. A10.3, E(e) = 0, so that

E(y) = β1 + β2E(x)

10BThe

Inconsistency of the Least Squares

Estimator



Subtract this expectation from the original equation:

–Multiply both sides by x – E(x):

– Take expected values of both sides:

or

10BThe


Estimator

2i i i i iy E y x E x e

22i i i i i i i i ix E x y E y x E x x E x e

22i i i i i i i i iE x E x y E y E x E x E x E x e

2cov , var cov ,x y x x e



Solve for β2:

– If we assume cov(x, e) = 0, then:

– The least squares estimator can be expressed as:

10BThe


Estimator

2

cov , cov ,var var

x y x ex x

Eq. 10B.1

2

cov ,var

x yx

Eq. 10B.2

2 2 2

/ 1 cov( , )var( )/ 1

i i i i

i i

x x y y x x y y N x ybxx x x x N

Eq. 10B.3



The sample variance and covariance converge to the true variance and covariance as the sample size N increases, so that the least squares estimator converges to β2

– If cov(x, e) = 0, then:

– If cov(x, e) ≠ 0, then:

10BThe


Estimator

2 2

cov( , ) cov( , )var( )var( )

x y x ybxx

2

cov , cov ,var var

x y x ex x



The least squares estimator now converges to:

10BThe


Estimator

2 2 2

cov , cov ,

var varx y x e

bx x

Eq. 10B.4



The IV estimator can be expressed as:

– For large samples:

10CThe Consistency

of the IV Estimator

Eq. 10C.1

2

1 cov ,ˆ1 cov ,

i i

i i

z z y y N z yz z x x N z x

2

cov ,ˆcov ,

z yz x

Eq. 10C.2



Following the steps from Appendix 10B, we get:

If cov(x, e) = 0, then:

10CThe Consistency

of the IV Estimator

Eq. 10C.3

Eq. 10C.4

2

cov , cov ,cov , cov ,

z y z ez x z x

2 2

cov ,ˆcov ,

z yz x



Start with the simple regression model:

We can describe the relationship between an instrumental variable z, which must be correlated with x but uncorrelated with e, as:

10DThe Logic of the Hausman Test

Eq. 10D.1

Eq. 10D.2

1 2y x e

0 1x z v



We can divide x into two parts, a systematic part and a random part, as:

– Substituting:


Eq. 10D.3

Eq. 10D.4

x E x v

1 2 1 2

1 2 2

y x e E x v eE x v e



An estimated analog of Eq. 10D.3 is:

Substitute Eq. 10D.5 into the original Eq. 10D.1:


Eq. 10D.5

Eq. 10D.6

ˆ ˆx x v

1 2 1 2

1 2 2

ˆ ˆˆ ˆ

y x e x v ex v e



To reduce confusion, write:

– If we omit


Eq. 10D.7

Eq. 10D.8

1 2 ˆ ˆy x v e

1 2 ˆy x e

v



Carrying out the test is made simpler by playing a trick on Eq. 10D.7:


Eq. 10D.9

1 2 2 2

1 2 2

1 2

ˆ ˆ ˆ ˆ

ˆ ˆ ˆ

ˆ

y x v e v v

x v v e

x v e



Using canonical correlations there is a solution to the problem of identifying weak instruments when an equation has more than one endogenous variable– Canonical correlations are a generalization of

the usual concept of a correlation between two variables and attempt to describe the association between two sets of variables

10ETesting for Weak

Instruments



A test for weak identification, the situation that arises when the instruments are correlated with the endogenous regressors but only weakly, is based on the Cragg-Donald F-test statistic

10ETesting for Weak

Instruments

10E.1A Test for Weak Identification

2 2Cragg-Donald 1B BN G B L r r Eq. 10E.1



Two particular consequences of weak instruments:– Relative Bias: In the presence estimator can

become large– Rejection Rate (Test Size): When estimating a

model with endogenous regressors, testing hypotheses about the coefficients of the endogenous variables is frequently of interest

10ETesting for Weak

Instruments




10ETesting for Weak

Instruments


Table 10E.1 Critical Values for the Weak Instrument Test Basedon IV Test Size (5% level of significance)



10ETesting for Weak

Instruments


Table 10E.2 Critical Values for the Weak Instrument Test Basedon IV Relative Bias (5% level of significance)



Consider the following HOURS supply equation specification:

where

10ETesting for Weak

Instruments

10E.2Examples of

Testing for Weak Identification

1 2 3 4 5β β β β 6 βHOURS MTR EDUC KIDSL NWIFEINC e Eq. 10E.2

1000NWIFEINC FAMINC WAGE HOURS



Weak IV Example 1: Endogenous: MTR; Instrument: EXPER– The estimated first-stage equation for MTR is

Model (1) of Table 10E.3– The estimated coefficient of MTR in the

estimated HOURS supply equation in Model (1) of Table 10E.4 is negative and significant at the 5% level

10ETesting for Weak

Instruments

10E.2Examples of




10ETesting for Weak

Instruments

10E.2Examples of


Table 10E.3 First-stage Equations



10ETesting for Weak

Instruments

10E.2Examples of


Table 10E.4 IV Estimation of Hours Equation



Weak IV Example 2: Endogenous: MTR; Instruments: EXPER, EXPER2, LARGECITY– The first-stage equation estimates are reported

in Model (2) of Table 10E.3– The estimated coefficient of MTR in the

estimated HOURS supply equation in Model (2) of Table 10E.4 is negative and significant at the 5% level, although the magnitudes of all the coefficients are smaller in absolute value for this estimation than the model in Model (1)

10ETesting for Weak

Instruments

10E.2Examples of




Weak IV Example 3 Endogenous: MTR, EDUC; Instruments: MOTHEREDUC, FATHEREDUC– The first-stage equations for MTR and EDUC

are Model (3) and Model (4) of Table 10E.3– The estimates of the HOURS supply equation,

Model (3) of Table 10E.4, shows parameter estimates that are wildly different from those in Model (1) and Model (2), and the very small t-statistic values imply very large standard errors, another consequence for instrumental variables estimation in the presence of weak instruments

10ETesting for Weak

Instruments

10E.2Examples of




If instrumental variables are ‘‘weak,’’ then the instrumental variables, or two-stage least squares, estimator is unreliable When there is a single endogenous variable, the first-stage F-test of the joint significance of the external instruments is an indicator of instrument strengthIf there is more than one endogenous variable on the right-hand side of an equation, then the F-test statistics from the first stage equations do not provide reliable information about instrument strength

10ETesting for Weak

Instruments

10E.3Testing for Weak

Identification: Conclusions



We do two types of simulations1. We generate a sample of artificial data and

give numerical illustrations of the estimators and tests

2. We carry out a Monte Carlo simulation to illustrate the repeated sampling properties of the least squares and IV/2SLS estimators under various conditions

10FMonte Carlo Simulation



We create an artificial sample of y values by adding e to the systematic portion of the regression– The least squares estimates are

10F.1Illustrations

Using Simulated Data

ˆ 0.9789 1.7034

se 0.088 0.090LSy x




The IV estimates using z1 are:

The IV estimates using z2 are:

10F.1Illustrations


1_ˆ 1.1011 1.1924

se 0.109 0.195IV zy x

2_ˆ 1.3451 0.1724

se 0.256 0.797IV zy x




If we use instrumental variables estimation with the invalid instrument, we get:

10F.1Illustrations


3_ˆ 0.9640 1.7657

se 0.095 0.172IV zy x




The outcome of two-stage least squares estimation using the two instruments z1 and z2 where we first obtain the first-stage regression of x on the two instruments z1 and z2:

– The instrumental variables estimates are:

10F.1Illustrations


1 2ˆ 0.1947 0.5700 0.2068

se 0.079 0.089 0.077

x z z Eq. 10F.1

1 2_ ,ˆ 1.1376 1.0399

se 0.116 0.194IV z zy x

Eq. 10F.2




To implement the Hausman test, estimate the first-stage equation shown in Eq. 10F.1 using the instruments z1 and z2 – Compute the residuals:

– Include the residuals as an extra variable in the regression equation and apply least squares:

10F.1.1The Hausman

Test

1 2ˆ ˆ 0.1947 0.5700 0.2068v x x x z z

ˆ ˆ1.1376 1.0399 0.9957se 0.080 0.133 0.163

y x v




If we consider using just z1 as an instrument, the estimated first-stage equation is:

If we use just z2 as an instrument, the estimated first-stage equation is:

10F.1.2Test for Weak Instruments

1ˆ 0.2196 0.5711

t 6.24

x z

2ˆ 0.2140 0.2090

t 2.28

x z




If we use z1, z2, and z3 as instruments, there are two surplus moment conditions. – The IV estimates using these three instruments

are:

– Obtaining the residuals and regressing them on the instruments yields:

• The R2 from this regression is 0.1311 and NR2 = 13.11

10F.1.3Testing the Validity of Surplus

Instruments

1 2 3_ , ,ˆ 1.0626 1.3535IV z z zy x

1 2 3ˆ 0.0207 0.1033 0.2355 0.1798e z z z




10F.2The Repeated

Sampling Properties of

IV/2SLS

Table 10F.1 Monte Carlo Simulation Results10F

Monte Carlo Simulation



In the case in which ρ = 0.8 and π = 0.1, the mean square error for the least squares estimator is:

The IV estimator it is

10000 22 21

β 10000 0.6062mmb

10F.2The Repeated

Sampling Properties of

IV/2SLS

21000022 21

β β 10000 1.0088mm


chapter 10 random regressors and moment-based estimation

Documents