the classical two-variable regression model

8/11/2019 The Classical Two-Variable Regression Model

1/29

1/29

EC114 Introduction to Quantitative Economics15. The Classical Two-Variable Regression Model I

Marcus Chambers

Department of EconomicsUniversity of Essex

14/16 February 2012

EC114 Introduction to Quantitative Economics 15. The Classical Two-Variable Regression Model I
http://find/


2/29

2/29

Outline

1 Introduction

2 Assumptions of the Classical Regression Model

3 Properties of the OLS Estimators

Reference : R. L. Thomas, Using Statistics in Economics ,McGraw-Hill, 2005, sections 12.1 and 12.2.

http://find/


3/29

Introduction 3/29

We have dealt with the problem of estimating the unknown

population parameters, and , in the linear regressionmodelY = + X + ,

where Y is the dependent variable, X is the regressor, and is a random disturbance that causes Y to deviate from its

expected value, E (Y ) = + X .In order to estimate and we assume we have a sampleof n observations on Y and X , which satisfy

Y i = + X i + i, i = 1 , . . . , n,

i.e. the model holds at each point in the sample.

http://find/


4/29

Introduction 4/29

The Ordinary Least Squares (OLS) estimators of and are

b = xi yi x2i

, a = Y b X ,

respectively, where Y and X are the sample means of Y

and X respectively, yi = Y i Y and xi = X i X .The tted model is

Y i = a + bX i + ei, i = 1 , . . . , n,

where ei denotes the residual i.e. the deviation of Y i fromthe tted value Y i = a + bX i.

http://find/http://goback/


5/29

Introduction 5/29

We measure the goodness-of-t by the coefcient of

determination, 0 R2

1 , where

R2 = b2 x2i

y2i= 1

SSRSST

,

SSR = e2

i and SST = y2

i .A low R2 indicates a poor t, while a high R2 indicates agood t.But can we say anything more about our sampleregression? For example:

What are the properties of the estimators a and b? Are they good estimators of and ?

http://find/


6/29

Introduction 6/29

We have seen (Lecture 14) that two useful properties of anestimator, Q, of a parameter, , are:

unbiasedness i.e. E (Q) = ; and efciency i.e. no other estimator of has smallervariance.

In addition, a linear estimator is said to be BLUE (BestLinear Unbiased Estimator) if it is:

linear (L); unbiased (U); and no other linear unbiased estimator (LUE) has smaller

variance, so it is best (B).

Do the OLS estimators, a and b, of and , have suchproperties?If we wish to answer this question, we need to make somemore assumptions about X and . . .



7/29

A i f h Cl i l R i M d l 8/29


8/29

Assumptions of the Classical Regression Model 8/29

The rst set of assumptions concern the explanatory

variable, X :Assumptions concerning the explanatory variable X

IA (non-random X ): X is non-stochastic (non-random);

IB (xed X ): The values of X are xed in repeatedsamples.

Assumption IA (non-random X ) implies that X is not

determined by chance but is entirely non-random.One way of interpreting IA is that X is chosen by theinvestigator.


http://find/


9/29


But IA is hard to justify in Economics in which the X valuesare not chosen by the researcher in an experiment but areobserved in the real world.It is, however, a useful starting point for our analysis of theOLS estimators.It is interesting to note that the origins of IA lie in theClassical model having been developed for the physicalsciences in which the researcher can choose the X valuesin an experiment and then see what the resulting values for

Y are.




10/29


Assumption IB (xed X ) implies that if we could obtainmore than one sample then the same values of X would befound in each sample.Again, this is something that may be possible in

experiments in the physical sciences, but in Economics wetypically only have one sample.Hence, given the xed, non-random nature of X , the onlysource of randomness determining Y is , about which weshall make the following assumptions:


http://find/


11/29


Assumptions concerning the disturbances

IIA (zero mean): E ( i) = 0 , for all i;

IIB (constant variance): V ( i) = 2 = constant for all i;

IIC (zero covariance): Cov( i, j) = 0 for all i = j;

IID (normality): each i is normally distributed.

These assumptions govern the properties of the random part of the model.

Given that X is xed they govern the variation in Y inrepeated samples, as illustrated in the next diagram:


http://find/


12/29


The vertical lines correspond to the xed values of X .The dots on the lines show the values of Y that occur inrepeated samples these differ because different values of occur each time due to their randomness.


http://find/


13/29

p g

Assumption IIA (zero mean) implies that the average

effect of in repeated samples is zero.Another way of interpreting this is that, on average, theexpected value of Y is + X i.e.

E (Y i) = E ( + X i + i)= + X i + E ( i)= + X i, i = 1 , . . . , n,

because E ( i) = 0 under IIA.

Note that E (Y i) is not the same for each i but depends on X i which is not constant throughout the sample.


http://find/


14/29

p g

Assumption IIB (constant variance) tells us that alldisturbances come from distributions with the samevariance, 2 .This implies that the variance of Y around E (Y ) (= + X )is the same at all points in the sample.Note that this Assumption does not say anything about theform of the distribution of the i, just that the variances areequal across the sample.


http://find/


15/29

Assumption IIC (zero covariance) ensures that there is nosystematic tendency for i to be related to j ( j = i).

This is often a strong assumption for time series wherethere can be a high degree of correlation from one periodto the next.Recall that represents the deviation of Y from its average

value E (Y ), and can be regarded as the unexpectedcomponent of Y .For example, a sequence of unexpectedly cold monthsmight lead to an increased demand for gas for heatingpurposes.This unexpected rise in demand would be reected in asequence of consecutive positive shocks (the s) implyinga positive covariance structure, which is ruled out by IIC.


http://find/


16/29

Assumption IID (normality) builds on the previousassumptions by specifying that the distribution of each i isnormal.Combining IIA (zero mean), IIB (constant variance) and IID(normality) gives

i N (0 , 2 ), i = 1 , . . . , n.

Note that Y i E (Y i) = Y i X i = i; this implies that

V (Y i) = E (Y i E (Y i))2 = E ( 2i ) = V ( i) =

2

which in turn implies thatY i N ( + X i,

2 ), i = 1 , . . . , n.

We can illustrate these ideas in a diagram:


http://find/


17/29

Note that the distribution has the same shape (normal) atall points but the mean changes with the value of X :

E (Y i) = + X i, i = 1 , . . . , n.


Properties of the OLS Estimators 18/29
http://goforward/http://find/http://goback/


18/29

Under IA (non-random X ) and IB (xed X ) the OLSestimators are linear , meaning that they can be written asa linear function of the Y i.For b this means that

b = wiY i = w1 Y 1 + w2 Y 2 + . . . + wnY n ,

where the wi are constants. In fact,

b = xi yi x2i

= xi(Y i Y )

x2i

= xiY i

x2i

because Y xi = 0

= wiY i

where wi = xi/ x2i .




19/29

Under IA (non-random X ), IB (xed X ) and IIA (zero mean),

the OLS estimators are unbiased : E (a ) = and E (b) = .

This means that the means of the sampling distributions of

a and b coincide with the population parameters and ,respectively.Put another way, if we had repeated samples then a and bwould, on average, be equal to the population parameters.In practice, of course, we only have a single sample, but itis a useful property for the distributions of the estimators tobe centred at the population parameter values.


http://find/


20/29

It is also possible to show that the OLS estimators are bestout of all possible linear unbiased estimators i.e. the OLS

estimators are BLUE (best linear unbiased estimators).For this we need IA, IB, IIA, IIB and IIC.The proof of this result also known as the Gauss-MarkovTheorem is a bit complicated!

The variances of a and b which are the smallest out of allLUEs are given by

V (a ) = 2a = 2 X 2i

n x2iand V (b) = 2b =

2

x2i.

These are the variances of the sampling distributions of aand b; their square roots ( a and b) are the standard errors .


http://find/


21/29

If we now add Assumption IID (normality) it can be shownthat the OLS estimators are efcient among all unbiasedestimators (not just linear ones).The normality assumption, IID, also ensures that a and bare normally distributed:

a N (, 2a), b N (, 2

b).

These results concerning the distributions of a and benable us to conduct hypothesis tests more on this inLecture 16.

However, there is one parameter that remains unknown,which is 2 .We therefore need to estimate 2 , but how?


http://find/


22/29

Recall that 2 is the variance of each i.

Our usual sample variance estimator, for a sample X 1 , . . . , X n , is calculated as the sum of squared deviationsof observations from their mean, divided by n 1 :

s2 X =( X i X )2

n 1.

Problem: we dont observe the i so cant compute such anestimator.However, we can use the residuals ei instead, the obvious

estimator beings2 =

(ei e)2

n 1.


http://find/


23/29

But this estimator is biased:

E ( s2 ) = n 2

n 12 < 2 .

Instead, we use the unbiased estimator

s2 = n 1n 2

s2 = (ei e)2

n 2.

Note that the denominator involves n 2 and not n 1 .This is because in constructing the ei we have had toestimate two parameters, and , by a and b and havetherefore used up two degrees of freedom.


http://find/


24/29

Recall that the OLS normal equations imply that ei = 0

and, hencee =

ein

= 0

n= 0 .

We can therefore write s2 as

s2 = e2

in 2

.

It turns out that s2 is an unbiased estimator of 2 i.e. it ispossible to show that

E (s2 ) = 2 .




25/29

In order to compute s2 we need to nd e2i (=SSR).One way is to calculate ei for each i = 1 , . . . , n.There is, however, a simpler method based on quantitiesthat are calculated for the OLS estimators:

e2i = y2i b xi yi

where xi = X i X and yi = Y i Y as usual.


http://find/


26/29

The estimated variances of a and b are then

s2a = s2 X 2i

n x2i, s2b =

s2

x2i.

The estimated standard errors are then given by sa and sb ;these are the values computed by Stata when running alinear regression.For example, the regression output for the cross-section of30 countries money supply and GDP is as follows:




27/29

. regress m g

Source | SS df MS Number of obs = 30-------------+------------------------------ F( 1, 28) = 94.88

Model | 20.3862321 1 20.3862321 Prob > F = 0.0000Residual | 6.01600434 28 .214857298 R-squared = 0.7721

-------------+------------------------------ Adj R-squared = 0.7640Total | 26.4022364 29 .910421946 Root MSE = .46353

------------------------------------------------------------------------------m | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------g | .1748489 .0179502 9.74 0.000 .1380795 .2116182

_cons | .0212579 .1157594 0.18 0.856 -.2158645 .2583803------------------------------------------------------------------------------

The numbers in the column headed Std. Err. are calculatedusing the formulae on the previous slide.

As we shall see next week they are used in theconstruction of t -statistics for hypothesis tests concerning and .


http://find/


28/29

We can summarise the results for the OLS estimators asfollows:

Property AssumptionsLinearity IA, IBUnbiasedness IA, IB, IIABLUness IA, IB, IIA, IIB, IIC

Efciency IA, IB, IIA, IIB, IIC, IIDNormality IA, IB, IIA, IIB, IIC, IID

Notice that more assumptions are required in order toobtain stronger results this is a common situation.

We shall now look at how these results for a and b can beused for making inferences about the populationparameters and .

http://find/


29/29

the classical two-variable regression model

Documents