Download - Ie Slide02(1)
![Page 1: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/1.jpg)
Introductory Econometrics
ECON2206/ECON3209
Slides02
Lecturer: Minxian Yang
ie_Slides02 my, School of Economics, UNSW 1
![Page 2: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/2.jpg)
2. Simple Regression Model (Ch2)
2. Simple Regression Model
• Lecture plan
– Motivation and definitions
– ZCM assumption
– Estimation method: OLS
– Units of measurement
– Nonlinear relationships
– Underlying assumptions of simple regression model
– Expected values and variances of OLS estimators
– Regression with STATA
ie_Slides02 my, School of Economics, UNSW 2
![Page 3: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/3.jpg)
2. Simple Regression Model (Ch2)
• Motivation
– Example 1. Ceteris paribus effect of fertiliser on soybean yield
yield = β0 + β1ferti + u .
– Example 2. Ceteris paribus effect of education on wage
wage = β0 + β1educ + u .
– In general,
y = β0 + β1x + u,
where u represents factors other than x that affect y.
– We are interested in
• explaining y in terms of x,
• how y responds to changes in x,
holding other factors fixed.
ie_Slides02 my, School of Economics, UNSW 3
![Page 4: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/4.jpg)
2. Simple Regression Model (Ch2)
• Simple regression model
– Definition
y = β0 + β1x + u ,
• y : dependent variable (observable)
• x : independent variable (observable)
• β1 : slope parameter, “partial effect,” (to be estimated)
• β0 : intercept parameter (to be estimated)
• u : error term or disturbance (unobservable)
– The disturbance u represents all factors other than x.
– With the intercept β0, the population average of u can
always be set to zero (without losing anything)
E(u) = 0 . y = β0 + E(u) + β1x + u − E(u)
ie_Slides02 my, School of Economics, UNSW 4
![Page 5: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/5.jpg)
2. Simple Regression Model (Ch2)
• Zero conditional mean assumption
– If other factors in u are held fixed (Δu = 0), the ceteris
paribus effect of x on y is β1 :
Δy = β1 Δx .
– But under what condition u can be held fixed while x
changes?
• As x and u are treated as random variables,
“u is fixed while x varying” is described as
“the mean of u for any given x is the same (zero)”.
– The required condition is
E(u | x) = E(u) = 0 ,
known as zero-conditional-mean (ZCM) assumption.
ie_Slides02 my, School of Economics, UNSW 5
Δ = “change”
X = X1 X2 X3 ...
E(u |X) = 0 0 0 0
y = β0 + β1x + u
y + Δy = β0 + β1(x + Δx)
+ u + Δu
![Page 6: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/6.jpg)
2. Simple Regression Model (Ch2)
• Zero conditional mean assumption
– Example 2. wage = β0 + β1educ + u
Suppose u represents ability.
Then ZCM assumption amounts to
E(ability | educ) = 0 ,
ie, the average ability is the same irrespective of the
years of education.
This is not true
• if people choose the education level to suit their ability;
• or if more ability is associated with less (or more)
education.
In practice, we do not know if ZCM holds and have to
deal with this issue.ie_Slides02 my, School of Economics, UNSW 6
![Page 7: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/7.jpg)
2. Simple Regression Model (Ch2)
• Zero conditional mean assumption
– Taking the conditional expectations of
y = β0 + β1x + u
for given x, ZCM implies
E(y | x) = β0 + β1x ,
known as the population regression function
(PRF), which is a linear function of x.
– The distribution of y is centred about E(y | x).
Systematic part of y : E(y | x).
Unsystematic part of y : u.
ie_Slides02 my, School of Economics, UNSW 7
![Page 8: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/8.jpg)
2. Simple Regression Model (Ch2)
• Simple regression model
yi = β0 + β1xi + ui
ie_Slides02 my, School of Economics, UNSW 8
x
y
x1
E(y| x = x3)
= β0 + β1x3
distribution of y
for given x = x3
conditional mean of y given x
(population regression line)
x2 x3
E(y| x = x2)
E(y| x = x1)
u
distribution of u
![Page 9: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/9.jpg)
2. Simple Regression Model (Ch2)
• Observations on (x, y)
– A random sample is a set of independent
observations on (x, y), ie, {(xi , yi), i = 1,2,...,n}.
– At observation level, the model may be written as
yi = β0 + β1xi + ui , i = 1, 2, ..., n
where i is the observation index.
– Collectively,
– Matrix notation:
ie_Slides02 my, School of Economics, UNSW 9
.
1
1
1
or ,
1
1
1
2
1
1
02
1
2
1
2
1
2
1
10
2
1
nnnnnn u
u
u
x
x
x
y
y
y
u
u
u
x
x
x
y
y
y
. UBXY
![Page 10: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/10.jpg)
2. Simple Regression Model (Ch2)
• Estimate simple regression
– The model:
yi = β0 + β1xi + ui , i = 1, 2, ..., n
– Let be the estimates of (β0 , β1).
– Corresponding residual is
– The sum of squared residuals (SSR)
indicates the goodness of the estimates.
– Good estimates should make SSR small.
ie_Slides02 my, School of Economics, UNSW 10
)ˆ,ˆ( 10
.,...,,,ˆˆˆ nixyu iii 21 10
n
i
ii
n
i
i xyuSSR1
2
10
1
2 )ˆˆ(ˆ
![Page 11: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/11.jpg)
2. Simple Regression Model (Ch2)
• Ordinary least squares (OLS)
– The OLS estimates minimise the SSR:
– Choose to minimise SSR.
The first order conditions lead to
ie_Slides02 my, School of Economics, UNSW 11
SSR. of minimiser )ˆ,ˆ( 10
,)ˆˆ( 01
10
n
i
ii xy
)ˆ,ˆ( 10
)ˆ,ˆ( 10
.)ˆˆ( 01
10
n
i
iii xxy
mean residual = 0
covariance of
residual and x
= 0
![Page 12: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/12.jpg)
2. Simple Regression Model (Ch2)
• Ordinary least squares (OLS)
– Solving the two equations with two unknowns gives
where
– OLS requires the condition
ie_Slides02 my, School of Economics, UNSW 12
,ˆˆ ,
)(
))((ˆ xy
xx
yyxx
n
i
i
n
i
ii
10
1
2
11
.)(
n
i
i xx1
2 0
. ,
n
i
i
n
i
i xn
xyn
y11
11
![Page 13: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/13.jpg)
2. Simple Regression Model (Ch2)
• OLS regression line or SRF
– For any set of data {(xi , yi), i = 1,2,...,n} with n > 2,
OLS can always be carried out as long as
– Once OLS estimates are obtained,
is known as the fitted value of y when x = xi.
– By OLS regression line or sample regression
function (SRF), we refer to
which is an estimate of PRF E(y | x) = β0 + β1 x.
ie_Slides02 my, School of Economics, UNSW 13
.)(
n
i
i xx1
2 0
ii xy 10 ˆˆˆ
,ˆˆˆ xy 10
![Page 14: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/14.jpg)
2. Simple Regression Model (Ch2)
• Interpretation of OLS estimate
– In the SRF
the slope estimate is the change in when x
increases by one unit:
which is of primary interest in practice.
– The dependent variable y may be decomposed either
as the sum of the SRF and the residual
or as the sum of the PRF and the disturbance.
ie_Slides02 my, School of Economics, UNSW 14
,ˆˆˆ xy 10
1 y
,/ˆˆ xy 1
uyy ˆˆ
.)|( uxyEy
![Page 15: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/15.jpg)
2. Simple Regression Model (Ch2)
• PRF versus SRF– Hope: SRF = PRF “on average” or “when n goes to infinity”.
ie_Slides02 my, School of Economics, UNSW 15
population
regression
line β0+ β1x
sample
regression
line
(xi, yi)
ui
x
y
iii uxy 10
residual
x10 ˆˆ
![Page 16: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/16.jpg)
2. Simple Regression Model (Ch2)
• OLS example
– Example 2. (regress wage educ)
• Population : workforce in 1976
• y = wage : hourly earnings (in $)
• x = educ : years of education
• OLS SRF : n = 526
• Interpretation
– Slope 0.54 : each additional year of schooling increases
the wage by $0.54.
– Intercept -0.90 : “fitted wage of a person with educ = 0”?
SRF does poorly at low levels of education.
• Predicted wage for a person with educ = 10?
ie_Slides02 my, School of Economics, UNSW 16
,..ˆ educegwa 540900 0 5 10 15
05
10
15
20
25
educ
wa
ge
![Page 17: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/17.jpg)
2. Simple Regression Model (Ch2)
• Properties of OLS
– The first order conditions:
imply that
• the sum of residuals is zero.
• the sample covariance of x and the residual is zero.
• the mean point is always on the SRF (or OLS
regression line).
ie_Slides02 my, School of Economics, UNSW 17
,)ˆˆ( 01
10
n
i
ii xy
01
10
n
i
iii xxy )ˆˆ(
),( yx
![Page 18: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/18.jpg)
2. Simple Regression Model (Ch2)
• Sums of squares
– Each yi may be decomposed into
– Measure variations from :
• Total sum of squares (total variation in yi ):
• Explained sum of squares (variation in ):
• sum of squared Residuals (variation in ):
• It can be shown that SST = SSE + SSR .
ie_Slides02 my, School of Economics, UNSW 18
y
.ˆˆiii uyy
,)(
n
i i yySST1
2
,)ˆ(
n
i i yySSE1
2
.ˆ
n
i iuSSR1
2
iy
iu
![Page 19: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/19.jpg)
2. Simple Regression Model (Ch2)
• R-squared: a goodness-of-fit measure
– How well does x explain y?
or how well does the OLS regression line fit data?
– We may use the fraction of variation in y that is
explained by x (or by the SRF) to measure.
– R-squared (coefficient of determination):
• larger R2, better fit;
• 0 ≤ R2 ≤ 1.
eg. R2 = 0.165 for
16.5% of variation in wage is explained by educ.
ie_Slides02 my, School of Economics, UNSW 19
.SST
SSR
SST
SSER 12
,..ˆ educegwa 540900
Not advisable to put
too much weight on
R2 when evaluating
regression models.
![Page 20: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/20.jpg)
2. Simple Regression Model (Ch2)
• Effects of changing units of measurement
– If y is multiplied by a constant c, then the OLS
intercept and slope estimates are also multiplied by c.
– If x is multiplied by a constant c, then the OLS
intercept estimate is unchanged but the slope
estimate is multiplied by 1/c.
– The R2 does not change when varying the units of
measurement.
eg. When wage is in dollars,
If wage is in cents,
ie_Slides02 my, School of Economics, UNSW 20
...ˆ educegwa 540900
.ˆ educegwa 5490
![Page 21: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/21.jpg)
2. Simple Regression Model (Ch2)
• Nonlinear relationships between x and y
– The OLS only requires the regression model
y = β0 + β1x + u
to be linear in parameters.
– Nonlinear relationships between y and x can be easily
accommodated.
eg. Suppose a better description
is that each year of education
increases wage by a fixed
percentage. This leads to
log(wage) = β0 + β1 educ + u ,
with %Δwage = (100β1)Δeduc
when Δu= 0.
OLS:
ie_Slides02 my, School of Economics, UNSW 21
186008305840 2 .,..ˆ Reduceglwa
0 5 10 15
01
23
educlw
ag
e
![Page 22: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/22.jpg)
2. Simple Regression Model (Ch2)
• Nonlinear relationships between x and y
– Linear models are linear in parameters.
– OLS applies to linear models no matter how x and y
are defined.
– But be careful about the interpretation of β.
ie_Slides02 my, School of Economics, UNSW 22
![Page 23: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/23.jpg)
2. Simple Regression Model (Ch2)
• OLS estimators
– A random sample, containing independent draws
from the same population, is random.
• A data set is a realisation of the random sample.
– OLS “estimates” computed from a random
sample is random, called the OLS estimators.
– To make inference about the population parameters
(β0, β1), we need to understand the statistical
properties of the OLS estimators.
– In particular, we like to know the means and
variances of the OLS estimators.
– We find these under a set of assumptions about the
simple regression model.
ie_Slides02 my, School of Economics, UNSW 23
)ˆ,ˆ( 10
![Page 24: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/24.jpg)
2. Simple Regression Model (Ch2)
• Assumptions about simple regression model(SLR1 to SLR4)
1. (linear in parameters) In the population model, y is
related to x by y = β0 + β1 x + u, where (β0, β1) are
population parameters and u is disturbance.
2. (random sample) {(xi , yi), i = 1,2,...,n} with n > 2 is a
random sample drawn from the population model.
3. (sample variation) The sample outcomes on x are
not of the same value.
4. (zero conditional mean) The disturbance u satisfies
E(u | x) = 0 for any given value of x. For the random
sample, E(ui | xi) = 0 for i = 1,2,...,n.
ie_Slides02 my, School of Economics, UNSW 24
![Page 25: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/25.jpg)
2. Simple Regression Model (Ch2)
• Property 1 of OLS estimators
Theorem 2.1
Under SLR1 to SLR4, the OLS estimators are
unbiased:
Unbiased estimators
– they are “centred” around (β0, β1).
– they correctly estimate (β0, β1) on average.
It is useful to note that
ie_Slides02 my, School of Economics, UNSW 25
.)ˆ( ,)ˆ( 0011 EE
)ˆ,ˆ( 10
),()()( uuxxyy iii 1
.)(
))((ˆ
n
i i
n
i ii
xx
xxuu
1
2
111
The estimation error is
entirely driven by a linear
combination of ui with
weights dependent on x.
![Page 26: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/26.jpg)
2. Simple Regression Model (Ch2)
• Property 2 of OLS estimators
5. (SLR5, homoskedasticity)
Var(ui|xi) = σ2 for i = 1,2,...,n. (It implies Var(ui) = σ2.)
Theorem 2.2
Under SLR1 to SLR5, the variances of are:
– the larger is σ2, the greater are the variances.
– the larger the variation in x, the smaller the variances.
ie_Slides02 my, School of Economics, UNSW 26
)ˆ,ˆ( 10
.)(
)ˆ( ,)(
)ˆ(
n
i i
n
i i
n
i i xx
xnVar
xxVar
1
2
1
212
0
1
2
2
1
Strictly, Theorem 2.2 is about the variances
of OLS estimators, conditional on given x.
![Page 27: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/27.jpg)
2. Simple Regression Model (Ch2)
• Homoskedasticity and heteroskedasticity
ie_Slides02 my, School of Economics, UNSW 27
![Page 28: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/28.jpg)
2. Simple Regression Model (Ch2)
• Estimation of σ2
– As the residual approximates u, the estimator of σ2 is
– is known as the standard error of the
regression, useful in forming the standard errors of
.
Theorem 2.3 (unbiased estimator of σ2)
Under SLR1 to SLR5,
ie_Slides02 my, School of Economics, UNSW 28
.)ˆ( 22 E
.ˆ
ˆ22
1
2
2
n
u
n
SSRn
i i
2 ˆˆ
)ˆ,ˆ( 10
“2” is the number of
estimated coefficients
![Page 29: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/29.jpg)
2. Simple Regression Model (Ch2)
• OLS in STATA
ie_Slides02 my, School of Economics, UNSW 29
SSR
standard
error of
regression
![Page 30: Ie Slide02(1)](https://reader034.vdocuments.site/reader034/viewer/2022052317/563db8ad550346aa9a95e370/html5/thumbnails/30.jpg)
2. Simple Regression Model (Ch2)
• Summary
– What is a simple regression model?
– What is the ZCM assumption? Why is it crucial for model interpretation and OLS being unbiased?
– What is the OLS estimation principle?
– What are PRF, SRF, error term and residual?
– How is R-squared is related to SSR?
– Can we describe, in a simple linear regression model, the nonlinear relationship between x and y?
– What are Assumptions SLR1 to SLR5? Why do we need to understand them?
– What are the statistical properties of OLS estimators?
– How do you OLS in STATA? regress y x
ie_Slides02 my, School of Economics, UNSW 30