may 2004 prof.vuthieu 1 basic econometrics course leader prof. dr.sc vuthieu

May 2004Prof.VuThieu1

Basic EconometricsBasic Econometrics

Course Leader Prof. Dr.Sc VuThieu



Introduction: What is Econometrics?


IntroductionIntroduction What is EconometricsWhat is Econometrics??

Definition 1: Economic Measurement Definition 2: Application of the mathematical statistics to economic data in order to lend empirical support to the economic mathematical models and obtain numerical results (Gerhard Tintner, 1968)



Definition 3: The quantitative analysis of actual economic phenomena based on concurrent development of theory and observation, related by appropriate methods of inference (P.A.Samuelson, T.C.Koopmans and J.R.N.Stone, 1954)



Definition 4: The social science which applies economics, mathematics and statistical inference to the analysis of economic phenomena (By Arthur S. Goldberger, 1964) Definition 5: The empirical determination of economic laws (By H. Theil, 1971)



Definition 6: A conjunction of economic theory and actual measurements, using the theory and technique of statistical inference as a bridge pier (By T.Haavelmo, 1944)

And the others


Econometrics

Economic Theory

MathematicalEconomics

Economic Statistics

Mathematic Statistics


IntroductionIntroduction Why a separate Why a separate disciplinediscipline?? Economic theory makes statements that are mostly qualitative in nature, while econometrics gives empirical content to most economic theory

Mathematical economics is to express economic theory in mathematical form without empirical verification of the theory, while econometrics is mainly interested in the later


IntroductionIntroduction Why a separate disciplineWhy a separate discipline?? Economic Statistics is mainly concerned with collecting, processing and presenting economic data. It does not being concerned with using the collected data to test economic theories

Mathematical statistics provides many of tools for economic studies, but econometrics supplies the later with many special methods of quantitative analysis based on economic data


Econometrics

Economic Theory

MathematicalEconomics

Economic Statistics

Mathematic Statistics


IntroductionIntroduction Methodology of Methodology of EconometricsEconometrics(1) Statement of theory or

hypothesis:

Keynes stated: ”Consumption increases as income increases, but not as much as the increase in income”. It means that “The marginal propensity to consume (MPC) for a unit change in income is grater than zero but less than unit”


IntroductionIntroduction Methodology of Methodology of EconometricsEconometrics(2) Specification of the mathematical model of the theoryY = ß1+ ß2X ; 0 < ß2< 1Y= consumption expenditureX= incomeß1 and ß2 are parameters; ß1 isintercept, and ß2 is slope coefficients


IntroductionIntroduction Methodology of Methodology of EconometricsEconometrics(3) Specification of the econometric model of the theoryY = ß1+ ß2X + u ; 0 < ß2< 1;Y = consumption expenditure; X = income; ß1 and ß2 are parameters; ß1is intercept and ß2 is slope coefficients; u is disturbance term or error term. It is a random or stochastic variable


IntroductionIntroduction Methodology of Methodology of EconometricsEconometrics

(4) Obtaining Data (See Table 1.1, page 6)

Y= Personal consumption expenditure X= Gross Domestic Product all in Billion US Dollars


IntroductionIntroduction Methodology of Methodology of EconometricsEconometrics(4) Obtaining Data

Year X Y

198019811982198319841985198619871988198919901991

2447.12476.92503.72619.42746.12865.82969.13052.23162.43223.33260.43240.8

3776.33843.13760.33906.64148.54279.84404.54539.94718.64838.04877.54821.0



(5) Estimating the Econometric Model

Y^ = - 231.8 + 0.7194 X (1.3.3) MPC was about 0.72 and it means

that for the sample period when real income increases 1 USD, led (on average) real consumption expenditure increases of about 72 cents

Note: A hat symbol (^) above one variable will signify an estimator of the relevant population value



(6) Hypothesis TestingAre the estimates accord with the expectations of the theory that is beingtested? Is MPC < 1 statistically? If so,it may support Keynes’ theory.Confirmation or refutation of economic theories based onsample evidence is object of StatisticalInference (hypothesis testing)



(7) Forecasting or Prediction With given future value(s) of X, what

is the future value(s) of Y? GDP=$6000Bill in 1994, what is the

forecast consumption expenditure? Y^= - 231.8+0.7196(6000) = 4084.6 Income Multiplier M = 1/(1 – MPC)

(=3.57). decrease (increase) of $1 in investment will eventually lead to $3.57 decrease (increase) in income



(8) Using model for control or policy purposes

Y=4000= -231.8+0.7194 X X 5882 MPC = 0.72, an income of $5882 Bill will produce an expenditure of $4000 Bill. By fiscal and monetary policy, Government can manipulate the control variable X to get the desired level of target variable Y



Figure 1.4: Anatomy of economic modelling

• 1) Economic Theory• 2) Mathematical Model of Theory• 3) Econometric Model of Theory• 4) Data• 5) Estimation of Econometric Model• 6) Hypothesis Testing• 7) Forecasting or Prediction• 8) Using the Model for control or policy

purposes


Economic Theory

Mathematic Model Econometric Model Data Collection

Estimation

Hypothesis Testing

Forecasting

Applicationin control or

policy studies



Chapter 1: THE NATURE OF REGRESSION ANALYSIS


1-1. Historical origin of the term 1-1. Historical origin of the term “Regression”“Regression”

The term REGRESSION was introduced by Francis Galton

Tendency for tall parents to have tall children and for short parents to have short children, but the average height of children born from parents of a given height tended to move (or regress) toward the average height in the population as a whole (F. Galton, “Family Likeness in Stature”)


1-1. Historical origin of the term 1-1. Historical origin of the term “Regression”“Regression”

Galton’s Law was confirmed by Karl Pearson: The average height of sons of a group of tall fathers < their fathers’ height. And the average height of sons of a group of short fathers > their fathers’ height. Thus “regressing” tall and short sons alike toward the average height of all men. (K. Pearson and A. Lee, “On the law of Inheritance”)

By the words of Galton, this was “Regression to mediocrity”


1-2. Modern Interpretation of 1-2. Modern Interpretation of Regression AnalysisRegression Analysis

The modern way in interpretation of Regression: Regression Analysis is concerned with the study of the dependence of one variable (The Dependent Variable), on one or more other variable(s) (The Explanatory Variable), with a view to estimating and/or predicting the (population) mean or average value of the former in term of the known or fixed (in repeated sampling) values of the latter.

Examples: (pages 16-19)


Dependent Variable Y; Explanatory Variable XsDependent Variable Y; Explanatory Variable Xs

1. Y = Son’s Height; X = Father’s Height2. Y = Height of boys; X = Age of boys3. Y = Personal Consumption Expenditure X = Personal Disposable Income4. Y = Demand; X = Price5. Y = Rate of Change of Wages X = Unemployment Rate6. Y = Money/Income; X = Inflation Rate7. Y = % Change in Demand; X = % Change in the advertising budget8. Y = Crop yield; Xs = temperature, rainfall, sunshine, fertilizer


1-3. Statistical vs.1-3. Statistical vs.Deterministic Deterministic RelationshipsRelationships

In regression analysis we are concerned with STATISTICAL DEPENDENCE among variables (not Functional or Deterministic), we essentially deal with RANDOM or STOCHASTIC variables (with the probability distributions)


1-4. Regression vs. Causation:1-4. Regression vs. Causation:

Regression does not necessarily imply causation. A statistical relationship cannot logically imply causation. “A statistical relationship, however strong and however suggestive, can never establish causal connection: our ideas of causation must come from outside statistics, ultimately from some theory or other” (M.G. Kendal and A. Stuart, “The Advanced Theory of Statistics”)


1-5. Regression vs. 1-5. Regression vs. CorrelationCorrelation

Correlation Analysis: the primary objective is to measure the strength or degree of linear association between two variables (both are assumed to be random)

Regression Analysis: we try to estimate or predict the average value of one variable (dependent, and assumed to be stochastic) on the basis of the fixed values of other variables (independent, and non-stochastic)


1-6. Terminology and Notation1-6. Terminology and NotationDependent Variable

Explained Variable

Predictand

Regressand

Response

Endogenous

Explanatory Variable(s)

Independent

Variable(s)

Predictor(s)

Regressor(s)

Stimulus or control variable(s)

Exogenous(es)


1-7. The Nature and Sources1-7. The Nature and Sources of Data for Econometric of Data for Econometric Analysis Analysis

1) Types of Data : Time series data; Cross-sectional data; Pooled data2) The Sources of Data3) The Accuracy of Data


1-8. Summary and Conclusions1-8. Summary and Conclusions

1) The key idea behind regression analysis is the statistic dependence of one variable on one or more other variable(s)

2) The objective of regression analysis is to estimate and/or predict the mean or average value of the dependent variable on basis of known (or fixed) values of explanatory variable(s)



3) The success of regression depends on the available and appropriate data

4) The researcher should clearly state the sources of the data used in the analysis, their definitions, their methods of collection, any gaps or omissions and any revisions in the data



Chapter 2: TWO-VARIABLE REGRESSION ANALYSIS: Some basic Ideas


2-1. A Hypothetical Example2-1. A Hypothetical Example

Total population: 60 families Y=Weekly family consumption expenditureX=Weekly disposable family income60 families were divided into 10 groups of

approximately the same income level (80, 100, 120, 140, 160, 180, 200, 220, 240, 260)



Table 2-1 gives the conditional distribution of Y on the given values of X

Table 2-2 gives the conditional probabilities of Y: p(YX)

Conditional Mean

(or Expectation): E(YX=Xi )


XY

80 100 120 140 160 180 200 220 240 260

Weekly family consumption expenditure Y ($)

55 65 79 80 102 110 120 135 137 15060 70 84 93 107 115 136 137 145 15265 74 90 95 110 120 140 140 155 17570 80 94 103 116 130 144 152 165 17875 85 98 108 118 135 145 157 175 180-- 88 -- 113 125 140 -- 160 189 185-- -- -- 115 -- -- -- 162 -- 191

Total 325 462 445 707 678 750 685 1043 966 1211

Mean 65 77 89 101 113 125 137 149 161 173

Table 2-2: Weekly family income X ($), and consumption Y ($)



Figure 2-1 shows the population regression line (curve). It is the regression of Y on X

Population regression curve is the locus of the conditional means or expectations of the dependent variable for the fixed values of the explanatory variable X (Fig.2-2)


2-2. The concepts of population 2-2. The concepts of population regression function (PRF) regression function (PRF)

E(YX=Xi ) = f(Xi) is Population Regression Function (PRF) or Population Regression (PR)

In the case of linear function we have linear population regression function (or equation or model)

E(YX=Xi ) = f(Xi) = ß1 + ß2Xi


2-2. The concepts of population 2-2. The concepts of population regression function (PRF) regression function (PRF)

E(YX=Xi ) = f(Xi) = ß1 + ß2Xi

ß1 and ß2 are regression coefficients, ß1is intercept and ß2 is slope coefficient

Linearity in the VariablesLinearity in the Parameters


2-4. Stochastic Specification of PRF2-4. Stochastic Specification of PRF

Ui = Y - E(YX=Xi ) or Yi = E(YX=Xi ) + Ui

Ui = Stochastic disturbance or stochastic error term. It is nonsystematic component

Component E(YX=Xi ) is systematic or deterministic. It is the mean consumption expenditure of all the families with the same level of income

The assumption that the regression line passes through the conditional means of Y implies that E(UiXi ) = 0


2-5. The Significance of the Stochastic 2-5. The Significance of the Stochastic

Disturbance Term Disturbance TermUi = Stochastic Disturbance Term is a

surrogate for all variables that are omitted from the model but they collectively affect Y

Many reasons why not include such variables into the model as follows:


2-5. The Significance of the Stochastic 2-5. The Significance of the Stochastic

Disturbance Term Disturbance TermWhy not include as many as variable into the model (or the reasons for using ui)

+ Vagueness of theory+ Unavailability of Data+ Core Variables vs. Peripheral Variables+ Intrinsic randomness in human behavior+ Poor proxy variables+ Principle of parsimony+ Wrong functional form


2-6. The Sample Regression2-6. The Sample Regression Function (SRF) Function (SRF)

Table 2-4: A random sample from the

populationY X

------------------ 70 80

65 100 90 120

95 140110 160115 180120 200140 220155 240150 260

------------------

Table 2-5: Another random sample from the population

Y X-------------------

55 80 88 100 90 120 80 140118 160120 180145 200135 220145 240175 260

--------------------


SRF1

SRF2

Weekly Consumption Expenditure (Y)

Weekly Income (X)



Fig.2-3: SRF1 and SRF 2 Yî = ^1 + ^2Xi (2.6.1) Yî = estimator of E(YXi) ^1 = estimator of 1

^2 = estimator of 2 Estimate = A particular numerical value

obtained by the estimator in an application SRF in stochastic form: Yi= ^1 + ^2Xi + uî

or Yi= Yî + uî (2.6.3)



Primary objective in regression analysis is

to estimate the PRF Yi= 1 + 2Xi + ui on

the basis of the SRF Yi= ^1 + ^2Xi + ei

and how to construct SRF so that ^1 close

to 1 and ^2 close to 2 as much as

possible



Population Regression Function PRFLinearity in the parametersStochastic PRFStochastic Disturbance Term ui plays a

critical role in estimating the PRFSample of observations from

populationStochastic Sample Regression Function

SRF used to estimate the PRF



The key concept underlying regression analysis is the concept of the population regression function (PRF).

This book deals with linear PRFs: linear in the unknown parameters. They may or may not linear in the variables.



For empirical purposes, it is the stochastic PRF that matters. The stochastic disturbance term ui plays a critical role in estimating the PRF.

The PRF is an idealized concept, since in practice one rarely has access to the entire population of interest. Generally, one has a sample of observations from population and use the stochastic sample regression (SRF) to estimate the PRF.



Chapter 3: TWO-VARIABLE REGRESSION MODEL: The problem of Estimation


3-1. The method of ordinary 3-1. The method of ordinary least square (OLS)least square (OLS)

Least-square criterion: Minimizing U^2

i = (Yi – Y^i) 2

= (Yi- ^1 - ^2X)2 (3.1.2) Normal Equation and solving it for ^1 and ^2 = Least-square estimators [See (3.1.6)(3.1.7)] Numerical and statistical properties of OLS are as follows:


3-1. The method of ordinary least 3-1. The method of ordinary least square (OLS)square (OLS)

OLS estimators are expressed solely in terms of observable quantities. They are point estimators The sample regression line passes through sample means of X and Y The mean value of the estimated Y^ is equal to the mean value of the actual Y: E(Y) = E(Y^) The mean value of the residuals Uî is zero: E(uî )=0 uî are uncorrelated with the predicted Yî and with Xi : That are uîYî = 0; uîXi = 0


3-2. The assumptions underlying 3-2. The assumptions underlying the method of least squaresthe method of least squares

Ass 1: Linear regression model (in parameters) Ass 2: X values are fixed in repeated sampling Ass 3: Zero mean value of ui : E(uiXi)=0 Ass 4: Homoscedasticity or equal variance of ui : Var (uiXi) = 2 [VS. Heteroscedasticity] Ass 5: No autocorrelation between the disturbances: Cov(ui,ujXi,Xj ) = 0 with i # j [VS. Correlation, + or - ]


3-2. The assumptions underlying 3-2. The assumptions underlying the method of least squares the method of least squares

Ass 6: Zero covariance between ui and Xi Cov(ui, Xi) = E(ui, Xi) = 0

Ass 7: The number of observations n must be greater than the number of parameters to be estimated

Ass 8: Variability in X values. They must not all be the same Ass 9: The regression model is correctly specified Ass 10: There is no perfect multicollinearity

between Xs


3-3. Precision or standard errors of 3-3. Precision or standard errors of least-squares estimatesleast-squares estimates

In statistics the precision of an estimate is measured by its standard error (SE) var( ^2) = 2 / x2

i (3.3.1) se(^2) = Var(^2) (3.3.2) var( ^1) = 2 X2

i / n x2i (3.3.3)

se(^1) = Var(^1) (3.3.4)

^ 2 = u^2i / (n - 2) (3.3.5)

^ = ^ 2 is standard error of the estimate


3-3. Precision or standard errors of 3-3. Precision or standard errors of least-squares estimates least-squares estimates

Features of the variance:

+ var( ^2) is proportional to 2 and inversely proportional to x2

i

+ var( ^1) is proportional to 2 and X2i

but inversely proportional to x2i and the

sample size n.

+ cov ( ^1 , ^2) = - var( ^2) shows the independence between ^1 and ^2

X


3-4. Properties of least-squares 3-4. Properties of least-squares estimators: estimators: The Gauss-Markov TheoremThe Gauss-Markov Theorem

An OLS estimator is said to be BLUE if :+ It is linear, that is, a linear function of a random variable, such as the dependent variable Y in the regression model+ It is unbiased , that is, its average or expected value, E(^2), is equal to the true value 2 + It has minimum variance in the class of all such linear unbiased estimatorsAn unbiased estimator with the least variance is known as an efficient estimator


3-4. Properties of least-squares 3-4. Properties of least-squares estimators: estimators: The Gauss-Markov The Gauss-Markov TheoremTheorem

Gauss- Markov Theorem:Given the assumptions of the classical linear regression model, the least-squares estimators, in class of unbiased linear estimators, have minimum variance, that is, they are BLUE


3-5. The coefficient of determination 3-5. The coefficient of determination rr22:: A measure of “Goodness of fit”A measure of “Goodness of fit”

Yi = i + i or

Yi - = i - i + i or

yi = i + i (Note: = )

Squaring on both side and summing => yi

2 = 2 x2i + 2

i ; or

TSS = ESS + RSS

Y

Y

YY Y

Y

U

U

U

y U

2β

2β


3-5. The coefficient of determination r3-5. The coefficient of determination r22:: A measure of “Goodness of fit”A measure of “Goodness of fit”

TSS = yi2 = Total Sum of Squares

ESS = Y^ i2 = ^2

2 x2i =

Explained Sum of Squares RSS = u^2

I = Residual Sum of Squares ESS RSS 1 = -------- + -------- ; or TSS TSS

RSS RSS 1 = r2 + ------- ; or r2 = 1 - ------- TSS TSS


3-5. The coefficient of determination r3-5. The coefficient of determination r22:: A measure of “Goodness of fit”A measure of “Goodness of fit” r2 = ESS/TSS is coefficient of determination, it measures the proportion or percentage of the total variation in Y explained by the regression Model 0 r2 1; r = r2 is sample correlation coefficient Some properties of r


3-5. The coefficient of determination r3-5. The coefficient of determination r22:: A measure of “Goodness of fit”A measure of “Goodness of fit”

3-6. A numerical Example (pages 80-83)3-7. Illustrative Examples (pages 83-85)3-8. Coffee demand Function3-9. Monte Carlo Experiments (page 85)3-10. Summary and conclusions (pages 86-87)



Chapter 4: THE NORMALITY ASSUMPTION:Classical Normal Linear Regression Model (CNLRM)


4-2.The normality assumption4-2.The normality assumption

CNLR assumes that each u i is distributed normally u i N(0, 2) with:

Mean = E(u i) = 0 Ass 3 Variance = E(u2

i) = 2 Ass 4 Cov(u i , u j ) = E(u i , u j) = 0 (i#j) Ass 5 Note: For two normally distributed variables, the

zero covariance or correlation means independence of them, so u i and u j are not only uncorrelated but also independently distributed. Therefore u i NID(0, 2) is Normal and

Independently Distributed



Why the normality assumption?(1) With a few exceptions, the distribution of sum

of a large number of independent and identically distributed random variables tends to a normal distribution as the number of such variables increases indefinitely

(2) If the number of variables is not very large or they are not strictly independent, their sum may still be normally distributed



Why the normality assumption?(3) Under the normality assumption

for ui , the OLS estimators ^1 and ^2 are also normally distributed

(4) The normal distribution is a comparatively simple distribution involving only two parameters (mean and variance)


4-3. Properties of OLS 4-3. Properties of OLS estimators under the normality estimators under the normality assumptionassumption With the normality assumption the

OLS estimators ^1 , ^2 and ^2 have the following properties:

1. They are unbiased2. They have minimum variance.

Combined 1 and 2, they are efficient estimators

3. Consistency, that is, as the sample size increases indefinitely, the estimators converge to their true population values


4-3. Properties of OLS estimators 4-3. Properties of OLS estimators under the normality assumptionunder the normality assumption

4. ^1 is normally distributed

N(1, ^12)

And Z = (^1- 1)/ ^1 is N(0,1)

5. ^2 is normally distributed N(2 ,^22)

And Z = (^2- 2)/ ^2 is N(0,1)6. (n-2) ^2/ 2 is distributed as the

2(n-2)


4-3. Properties of OLS estimators 4-3. Properties of OLS estimators under the normality assumptionunder the normality assumption

7. ^1 and ^2 are distributed independently of ^2. They have minimum variance in the entire class of unbiased estimators, whether linear or not. They are best unbiased estimators (BUE)

8. Let ui is N(0, 2 ) then Yi is

N[E(Yi); Var(Yi)] = N[1+ 2X i ; 2]


Some last points of chapter 4Some last points of chapter 4

4-4. The method of Maximum likelihood (ML) ML is point estimation method with somestronger theoretical properties than OLS(Appendix 4.A on pages 110-114)The estimators of coefficients ’s by OLS and ML are identical. They are true estimators of the ’s (ML estimator of 2) = u^i

2/n (is biased estimator) (OLS estimator of 2) = u^i

2/n-2 (is unbiased estimator)

When sample size (n) gets larger the two estimators tend to be equal


Some last points of chapter 4Some last points of chapter 4

4-5. Probability distributions relatedto the Normal Distribution: The t, 2,and F distributionsSee section (4.5) on pages 107-108with 8 theorems and Appendix A, onpages 755-7764-6. Summary and ConclusionsSee 10 conclusions on pages 109-110



Chapter 5: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis Testing


Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION:Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing

5-1. Statistical Prerequisites See Appendix A with key concepts

such as probability, probability distributions, Type I Error, Type II Error,level of significance, power of a statistic test, and confidence interval


Chapter 5Chapter 5 TWO-VARIABLE REGRESSION: TWO-VARIABLE REGRESSION: Interval Estimation and Hypothesis TestingInterval Estimation and Hypothesis Testing

5-2. Interval estimation: Some basic Ideas How “close” is, say, ^2 to 2 ?

Pr (^2 - 2 ^2 + ) = 1 - (5.2.1)

Random interval ^2 - 2 ^2 +

if exits, it known as confidence interval

^2 - is lower confidence limit

^2 + is upper confidence limit



5-2. Interval estimation: Some basic Ideas

(1 - ) is confidence coefficient,

0 < < 1 is significance level

Equation (5.2.1) does not mean that the Pr of

2 lying between the given limits is (1 - ), but

the Pr of constructing an interval that contains

2 is (1 - )

(^2 - , ^2 + ) is random interval



5-2. Interval estimation: Some basic Ideas In repeated sampling, the intervals will

enclose, in (1 - )*100 of the cases, the true value of the parameters

For a specific sample, can not say that the probability is (1 - ) that a given fixed interval includes the true 2

If the sampling or probability distributions of the estimators are known, one can make confidence interval statement like (5.2.1)



5-3. Confidence Intervals for Regression Coefficients

Z= (^2 - 2)/se(^2) = (^2 - 2) x2

i / ~N(0,1)(5.3.1)

We did not know and have to use ^ instead, so:

t= (^2 - 2)/se(^2) = (^2 - 2) x2i /^ ~ t(n-2)

(5.3.2)=> Interval for 2

Pr [ -t /2 t t /2] = 1- (5.3.3)



5-3. Confidence Intervals for Regression Coefficients

Or confidence interval for 2 is

Pr [^2-t /2se(^2) 2 ^2+t /2se(^2)] = 1- (5.3.5)

Confidence Interval for 1

Pr [^1-t /2se(^1) 1 ^1+t /2se(^1)] = 1- (5.3.7)



5-4. Confidence Intervals for 2

Pr [(n-2)^2/ 2/2 2

(n-2)^2/ 21- /2] = 1-

(5.4.3)The interpretation of this interval is: If we

establish (1- ) confidence limits on 2 and if we maintain a priori that these limits will include true 2, we shall be right in the long run (1- ) percent of the time



5-5. Hypothesis Testing: General Comments The stated hypothesis is known as the null hypothesis: Ho

The Ho is tested against and alternative hypothesis: H1

5-6. Hypothesis Testing: The confidence interval approach

One-sided or one-tail Test H0: 2 * versus H1: 2 > *



Two-sided or two-tail Test H0: 2 = * versus H1: 2 # *

^2 - t /2se(^2) 2 ^2 + t /2se(^2) values

of 2 lying in this interval are plausible under Ho

with 100*(1- )% confidence. If 2 lies in this region we do not reject Ho (the

finding is statistically insignificant) If 2 falls outside this interval, we reject Ho (the

finding is statistically significant)



5-7. Hypothesis Testing: Hypothesis Testing: The test of significance approachThe test of significance approach

A test of significance is a procedure by which A test of significance is a procedure by which sample results are used to verify the truth or sample results are used to verify the truth or falsity of a null hypothesisfalsity of a null hypothesis

Testing the significance of regression Testing the significance of regression coefficient: The t-testcoefficient: The t-test

Pr [^2-t /2se(^2) 2 ^2+t /2se(^2)]= 1- (5.7.2)



5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approach

Table 5-1: Decision Rule for t-test of significanceTable 5-1: Decision Rule for t-test of significance

Type of Hypothesis

H0 H1 Reject H0 if

Two-tail 2 = 2* 2 # 2* |t| > t/2,df

Right-tail 2 2* 2 > 2* t > t,df

Left-tail 2 2* 2 < 2* t < - t,df



5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approachTesting the significance of Testing the significance of 2 2 : The : The 22 Test Test

Under the Normality assumption we have:Under the Normality assumption we have:

^2

22 = = (n-2) ------- ~ 22 (n-2) (5.4.1)

2 From (5.4.2) and (5.4.3) on page 520 =>



5-7. Hypothesis Testing: The test of Hypothesis Testing: The test of significance approachsignificance approach

Table 5-2: A summary of theTable 5-2: A summary of the 22 Test Test

H0 H1 Reject H0 if

2 = 20 2 > 2

0 Df.(^2)/ 20 > 22 ,df

2 = 20 2 < 2

0 Df.(^2)/ 20 < 22

((1-),df

2 = 20 2 # 2

0 Df.(^2)/ 20 > 22

/2,df

or < 2 2 ((1-/2), df



5-8. Hypothesis Testing: Hypothesis Testing: Some practical aspectsSome practical aspects1) The meaning of “Accepting” or 1) The meaning of “Accepting” or

“Rejecting” a Hypothesis“Rejecting” a Hypothesis2) The Null Hypothesis and the Rule of 2) The Null Hypothesis and the Rule of ThumbThumb3) Forming the Null and Alternative 3) Forming the Null and Alternative HypothesesHypotheses4) Choosing 4) Choosing , the Level of Significance, the Level of Significance



5-8. Hypothesis Testing: Hypothesis Testing: Some practical aspectsSome practical aspects5) The Exact Level of Significance: 5) The Exact Level of Significance: The p-Value [The p-Value [See page 132See page 132]6) Statistical Significance versus 6) Statistical Significance versus Practical Significance Practical Significance 7) The Choice between Confidence-7) The Choice between Confidence- Interval and Test-of-Significance Interval and Test-of-Significance Approaches to Hypothesis TestingApproaches to Hypothesis Testing [Warning: Read carefully pages 117-134 ]



5-9. Regression Analysis and Analysis of Variance TSS = ESS + RSSTSS = ESS + RSS F=[MSS F=[MSS of ESS]of ESS]/[MSS /[MSS of RSS] of RSS] = =

= = 22^22 xxii

22/ / ^2 2 (5.9.1)(5.9.1) If uIf uii are normally distributed; H are normally distributed; H00:: 2 2 = 0= 0

then F follows the F distribution with 1 then F follows the F distribution with 1 and n-2 degree of freedomand n-2 degree of freedom



5-9. Regression Analysis and Analysis of Variance

F provides a test statistic to test the F provides a test statistic to test the null hypothesis that true null hypothesis that true 22 is zero by is zero by compare this F ratio with the F-critical compare this F ratio with the F-critical obtained from F tables at the chosen obtained from F tables at the chosen level of significance, or obtain the p-level of significance, or obtain the p-value of the computed F statistic to value of the computed F statistic to make decisionmake decision



5-9. Regression Analysis and Analysis of Variance

Table 5-3. ANOVA for two-variable regression modelTable 5-3. ANOVA for two-variable regression model

Source of Variation

Sum of square ( SS) Degree of Freedom -(Df)

Mean sum of square ( MSS)

ESS (due to regression)

y^yîi2 2 = = 22^22

xxii22 1 22^22

xxii22

RSS (due to residuals)

uûîi22 n-2 uûîi

2 2 /(n-2)=/(n-2)=^22

TSS y y ii

22 n-1



5-10. Application of Regression Analysis: Problem of Prediction

By the data of Table 3-2, we obtained the sample regression (3.6.2) : Y^i = 24.4545 + 0.5091Xi , where Y^i is the estimator of true E(Yi)

There are two kinds of prediction as follows:



5-10. Application of Regression Analysis: Problem of Prediction Mean prediction: Prediction of the

conditional mean value of Y corresponding to a chosen X, say X0, that is the point on the population regression line itself (see pages 137-138 for details)

Individual prediction: Prediction of an individual Y value corresponding to X0

(see pages 138-139 for details)



5-11. Reporting the results of regression analysis

An illustration:

Y^I= 24.4545 + 0.5091Xi (5.1.1)

Se = (6.4138) (0.0357) r2= 0.9621t = (3.8128) (14.2405) df= 8P = (0.002517) (0.000000289) F1,2=2202.87



5-12. Evaluating the results of regression analysis:

Normality Test: The Chi-Square (2) Goodness of fit Test

2N-1-k = (Oi – Ei)2/Ei

(5.12.1)Oi is observed residuals (u^i) in interval iEi is expected residuals in interval iN is number of classes or groups; k is number ofparameters to be estimated. If p-value of

obtaining 2N-1-k is high (or 2

N-1-k is small) =>The Normality Hypothesis can not be rejected




Normality Test: The Chi-Square (2) Goodness of fit Test

H0: ui is normally distributed H1: ui is un-normally distributedCalculated-2

N-1-k = (Oi – Ei)2/Ei (5.12.1)Decision rule: Calculated-2

N-1-k > Critical-2N-1-k then H0 can

be rejected




The Jarque-Bera (JB) test of normalityThis test first computes the Skewness (S)and Kurtosis (K) and uses the followingstatistic:JB = n [S2/6 + (K-3)2/24] (5.12.2)

Mean= xbar = xi/n ; SD2 = (xi-xbar)2/(n-1)S=m3/m2

3/2 ; K=m4/m22 ; mk= (xi-xbar)k/n



5-12. (Continued) Under the null hypothesis H0 that the residuals are normally distributed Jarque and Bera show that in large sample (asymptotically) the JB statistic given in (5.12.12) follows the Chi-Square distribution with 2 df. If the p-value of the computed Chi-Square statistic in an application is sufficiently low, one can reject the hypothesis that the residuals are normally distributed. But if p-value is reasonable high, one does not reject the normality assumption.



5-13. Summary and Conclusions1. Estimation and Hypothesis testing

constitute the two main branches of classical statistics

2. Hypothesis testing answers this question: Is a given finding compatible with a stated hypothesis or not?

3. There are two mutually complementary approaches to answering the preceding question: Confidence interval and test of significance.



5-13. Summary and Conclusions

4. Confidence-interval approach has a specified probability of including within its limits the true value of the unknown parameter. If the null-hypothesized value lies in the confidence interval, H0 is not rejected, whereas if it lies outside this interval, H0 can be rejected



5-13. Summary and Conclusions5. Significance test procedure develops a

test statistic which follows a well-defined probability distribution (like normal, t, F, or Chi-square). Once a test statistic is computed, its p-value can be easily obtained.

The p-value The p-value of a test is the lowest significance level, at which we would reject H0. It gives exact probability of obtaining the estimated test statistic under H0. If p-value is small, one can reject H0, but if it is large one may not reject H0.



5-13. Summary and Conclusions 6. Type I error is the error of rejecting

a true hypothesis. Type II error is the error of accepting a false hypothesis. In practice, one should be careful in fixing the level of significance , the probability of committing a type I error (at arbitrary values such as 1%, 5%, 10%). It is better to quote the p-value of the test statistic.




7. This chapter introduced the normality test to find out whether ui follows the normal distribution. Since in small samples, the t, F,and Chi-square tests require the normality assumption, it is important that this assumption be checked formally



5-13. Summary and Conclusions (ended)

8. If the model is deemed practically adequate, it may be used for forecasting purposes. But should not go too far out of the sample range of the regressor values. Otherwise, forecasting errors can increase dramatically.



Chapter 6

EXTENSIONS OF THE TWO-VARIABLE LINEAR REGRESSION MODEL


Chapter 6Chapter 6 EXTENSIONS OF THE TWO-VARIABLE LINEAR EXTENSIONS OF THE TWO-VARIABLE LINEAR REGRESSION MODELSREGRESSION MODELS

6-1. Regression through the origin The SRF form of regression: Yi = ^2X i + u^ i (6.1.5) Comparison two types of regressions: * Regression through-origin model and * Regression with intercept



6-1. Regression through the origin Comparison two types of regressions:

^2 = XiYi/X2i (6.1.6) O

^2 = xiyi/x2i (3.1.6) I

var(^2) = 2/X2i (6.1.7) O

var(^2) = 2/x2i (3.3.1) I

^2 = u^i)2/(n-1) (6.1.8) O

^2 = u^i)2/(n-2) (3.3.5) I


Chapter 6Chapter 6 EXTENSIONS OF THE TWO-VARIABLE LINEAR EXTENSIONS OF THE TWO-VARIABLE LINEAR REGRESSION MODELSREGRESSION MODELS6-1. Regression through the origin r2 for regression through-origin model

Raw r2 = (XiYi)2 /X2

iY2i (6.1.9)

Note: Without very strong a priory expectation, well advise is sticking to the conventional, intercept-present model. If intercept equals to zero statistically, for practical purposes we have a regression through the origin. If in fact there is an intercept in the model but we insist on fitting a regression through the origin, we would be committing a specification error


Chapter 6Chapter 6 EXTENSIONS OF THE TWO-VARIABLE EXTENSIONS OF THE TWO-VARIABLE LINEAR REGRESSION MODELSLINEAR REGRESSION MODELS

6-1. Regression through the origin Illustrative Examples:1) Capital Asset Pricing Model - CAPM (page 156)2) Market Model (page 157)3) The Characteristic Line of Portfolio Theory

(page 159)



6-2. Scaling and units of measurement

Let Yi = ^1 + ^2Xi + u^ i (6.2.1) Define Y*i=w 1 Y i and X*i=w 2 X i then: ^2 = (w1/w2)^2

(6.2.15) ^1 = w1^1

(6.2.16) *^2 = w1

2^2

(6.2.17) Var(^1) = w2

1 Var(^1) (6.2.18) Var(^2) = (w1/w2)

2 Var(^2) (6.2.19)

r2xy = r2

x*y* (6.2.20)


Chapter 6Chapter 6 EXTENSIONS OF THE TWO-VARIABLE EXTENSIONS OF THE TWO-VARIABLE LINEAR REGRESSION MODELSLINEAR REGRESSION MODELS

6-2. Scaling and units of measurement From one scale of measurement, one can derive the results

based on another scale of measurement. If w1= w2 the

intercept and standard error are both multiplied by w1. If

w2=1 and scale of Y changed by w1, then all coefficients and

standard errors are all multiplied by w1. If w1=1 and scale of

X changed by w2, then only slope coefficient and its standard

error are multiplied by 1/w2. Transformation from (Y,X) to

(Y*,X*) scale does not affect the properties of OLSEstimators

A numerical example: (pages 161, 163-165)


6-3. Functional form of regression model6-3. Functional form of regression model

The log-linear model Semi-log model Reciprocal model


6-4. How to measure elasticity6-4. How to measure elasticity

The log-linear model Exponential regression model: Yi= 1Xi

e u i (6.4.1)

By taking log to the base e of both side: lnYi = ln1 +2lnXi + ui , by setting ln1 =

lnYi = +2lnXi + ui (6.4.3) (log-log, or double-log, or log-linear model) This can be estimated by OLS by letting Y*i = +2X*i + ui , where Y*i=lnYi, X*i=lnXi ; 2 measures the ELASTICITY of Y respect to X, that is,

percentage change in Y for a given (small) percentage change in X.


6-4. How to measure elasticity6-4. How to measure elasticity

The log-linear modelThe elasticity E of a variable Y with respect to variable X is defined as:E=dY/dX=(% change in Y)/(% change in X)

~ [(Y/Y) x 100] / [(X/X) x100]= = (Y/X)x (X/Y) = slope x (X/Y)

An illustrative example: The coffee

demand function (pages 167-168)


6-5. Semi-log model6-5. Semi-log model: : Log-lin and Lin-log ModelsLog-lin and Lin-log Models

How to measure the growth rate: The log-lin model Y t = Y0 (1+r) t

(6.5.1)

lnYt = lnY0 + t ln(1+r) (6.5.2)

lnYt = + 2t , called constant growth model (6.5.5) where 1 = lnY0 ; 2 = ln(1+r) lnYt = + 2t + ui (6.5.6) It is Semi-log model, or log-lin model. The slope

coefficient measures the constant proportional or relative change in Y for a given absolute change in the value of the regressor (t)

2 = (Relative change in regressand)/(Absolute change in regressor) (6.5.7)



Instantaneous Vs. compound rate of growth 2 is instantaneous rate of growth antilog(2) – 1 is compound rate of growth

The linear trend model Yt = + 2t + ut (6.5.9) If 2 > there is an upward trend in Y If 2 < there is an downward trend in Y Note: (i) Cannot compare the r2 values of

models (6.5.5) and (6.5.9) because the regressands in the two models are different, (ii) Such models may be appropriate only if a time series is stationary.



The lin-log model: Yi = 1 +2lnXi + ui (6.5.11) 2 = (Change in Y) / Change in lnX =

(Change in Y)/(Relative change in X) ~ (Y)/(X/X) (6.5.12)

or Y = 2 (X/X) (6.5.13) That is, the absolute change in Y equal

to 2 times the relative change in X.


6-6. Reciprocal Models6-6. Reciprocal Models:: Log-lin and Lin-log ModelsLog-lin and Lin-log Models

The reciprocal model: Yi = 1 + 2( 1/Xi ) + ui (6.5.14) As X increases definitely, the term

2( 1/Xi ) approaches to zero and Yi

approaches the limiting or asymptotic value 1 (See figure 6.5 in page 174)

An Illustrative example: The Phillips Curve for the United Kingdom 1950-1966


6-7. Summary of Functional Forms6-7. Summary of Functional Forms

Table 6.5 (page 178)

Model Equation Slope = dY/dX

Elasticity = (dY/dX).(X/Y)

Linear Y = X (X/Y) */

Log-linear (log-log)

lnY = lnX

(YX)

Log-lin lnY = X Y X */

Lin-log Y = lnX 2(1/X) Y) */

Reciprocal Y = X)

- 2(1/X2) - XY) */


6-7. Summary of Functional Forms6-7. Summary of Functional Forms

Note: */ indicates that the elasticity coefficient is variable, depending on the value taken by X or Y or both. when no X and Y values are specified, in practice, very often these elasticities are measured at the mean values E(X) and E(Y).

-----------------------------------------------6-8. A note on the stochastic error term6-9. Summary and conclusions (pages 179-180)



Chapter 7MULTIPLE REGRESSION ANALYSIS: The Problem of Estimation


7-1. The three-Variable Model: 7-1. The three-Variable Model: Notation and AssumptionsNotation and Assumptions

Yi = ß1+ ß2X2i + ß3X3i + u i (7.1.1) ß2 , ß3 are partial regression coefficients With the following assumptions:+ Zero mean value of U i:: E(u i|X2i,X3i) = 0. i (7.1.2)+ No serial correlation: Cov(ui,uj) = 0, i # j (7.1.3)+ Homoscedasticity: Var(u i) = 2 (7.1.4)+ Cov(ui,X2i) = Cov(ui,X3i) = 0 (7.1.5)+ No specification bias or model correct specified (7.1.6) + No exact collinearity between X variables (7.1.7)(no multicollinearity in the cases of more explanatory vars. If there is linear relationship exits, X vars. Are said to be linearly dependent)+ Model is linear in parameters


7-2. Interpretation of Multiple 7-2. Interpretation of Multiple RegressionRegression

E(Yi| X2i ,X3i) = ß1+ ß2X2i + ß3X3i (7.2.1)

(7.2.1) gives conditional mean or

expected value of Y conditional upon the given or fixed value of the X2 and X3


7-3. The meaning of partial 7-3. The meaning of partial regression coefficients regression coefficients

Yi= ß1+ ß2X2i + ß3X3 +….+ ßsXs+ ui ßk measures the change in the mean

value of Y per unit change in Xk, holding the rest explanatory variables

constant. It gives the “direct” effect of unit change in Xk on the E(Yi), net of Xj

(j # k) How to control the “true” effect of a

unit change in Xk on Y? (read pages 195-197)


7-4. OLS and ML estimation of the 7-4. OLS and ML estimation of the partial regression coefficientspartial regression coefficients

This section (pages 197-201) provides: 1. The OLS estimators in the case of three-

variable regression

Yi= ß1+ ß2X2i + ß3X3+ ui 2. Variances and standard errors of OLS

estimators3. 8 properties of OLS estimators (pp 199-201)4. Understanding on ML estimators


7-5. The multiple coefficient of 7-5. The multiple coefficient of determination Rdetermination R22 and the multiple and the multiple coefficient of correlation Rcoefficient of correlation R

This section provides:1. Definition of R2 in the context of multiple

regression like r2 in the case of two-variable regression

2. R = R2 is the coefficient of multiple regression, it measures the degree of association between Y and all the explanatory variables jointly

3. Variance of a partial regression coefficientVar(ß^k) = 2/ x2

k (1/(1-R2k)) (7.5.6)

Where ß^k is the partial regression coefficient of regressor Xk and R2

k is the R2 in the

regression of Xk on the rest regressors


7-6. Example 7.1: The 7-6. Example 7.1: The expectations-augmented Philips expectations-augmented Philips Curve for the US (1970-1982)Curve for the US (1970-1982)

This section provides an illustration for the ideas introduced in the chapter

Regression Model (7.6.1) Data set is in Table 7.1


7-7. Simple regression in the 7-7. Simple regression in the context of multiple regression: context of multiple regression: Introduction to specification biasIntroduction to specification bias

This section provides an understanding on “ Simple regression in the context of multiple regression”. It will cause the specification bias which will be discussed in Chapter 13


7-8. R7-8. R22 and the Adjusted-R and the Adjusted-R2 2

R2 is a non-decreasing function of the number of explanatory variables. An additional X variable will not decrease R2

R2= ESS/TSS = 1- RSS/TSS = 1-u^2I / y^2

i (7.8.1) This will make the wrong direction by adding more

irrelevant variables into the regression and give an idea for an adjusted-R2 (R bar) by taking account of degree of freedom

R2bar= 1- [ u^2

I /(n-k)] / [y^2i /(n-1) ] , or (7.8.2)

R2bar= 1- ^2

/ S2Y (S2

Y is sample variance of Y) K= number of parameters including intercept term– By substituting (7.8.1) into (7.8.2) we get R2

bar = 1- (1-R2) (n-1)/(n- k) (7.8.4)– For k > 1, R2

bar < R2 thus when number of X variables

increases R2bar increases less than R2 and R2

bar can be negative



R2 is a non-decreasing function of the number of explanatory variables. An additional X variable will not decrease R2

R2= ESS/TSS = 1- RSS/TSS = 1-u^2I / y^2

i (7.8.1) This will make the wrong direction by adding more

irrelevant variables into the regression and give an idea for an adjusted-R2 (R bar) by taking account of degree of freedom

R2bar= 1- [ u^2

I /(n-k)] / [y^2i /(n-1) ] , or (7.8.2)

R2bar= 1- ^2

/ S2Y (S2

Y is sample variance of Y) K= number of parameters including intercept term– By substituting (7.8.1) into (7.8.2) we get R2

bar = 1- (1-R2) (n-1)/(n- k) (7.8.4)– For k > 1, R2

bar < R2 thus when number of X variables

increases R2bar increases less than R2 and R2

bar can be negative



Comparing Two R2 Values:To compare, the size n and the dependent variable must be the same

Example 7-2: Coffee Demand Function Revisited (page 210)

The “game” of maximizing adjusted-R2: Choosing the model that gives the highest R2

bar may be dangerous, for in regression our objective is not for that but for obtaining the dependable estimates of the true population regression coefficients and draw statistical inferences about them

Should be more concerned about the logical or theoretical relevance of the explanatory variables to the dependent variable and their statistical significance


7-9. Partial Correlation 7-9. Partial Correlation CoefficientsCoefficients

This section provides:

1. Explanation of simple and partial correlation coefficients

2. Interpretation of simple and partial correlation coefficients

(pages 211-214)


7-10. Example 7.3: The Cobb-7-10. Example 7.3: The Cobb-Douglas Production functionDouglas Production functionMore on functional formMore on functional form

Yi = 1X22i X3

3ieUi (7.10.1)

By log-transform of this model: lnYi = ln1 + 2ln X2i + 3ln X3i + Ui

= 0 + 2ln X2i + 3ln X3i + Ui

(7.10.2)

Data set is in Table 7.3Report of results is in page 216


7-11 Polynomial Regression 7-11 Polynomial Regression ModelsModels

Yi = 0 + 1 Xi + 2 X2i +…+ k Xk

i + Ui

(7.11.3) Example 7.4: Estimating the Total Cost

Function Data set is in Table 7.4 Empirical results is in page 221

-------------------------------------------------------------- 7-12. Summary and Conclusions

(page 221)



Chapter 8MULTIPLE REGRESSION ANALYSIS: The Problem of Inference


Chapter 8Chapter 8MULTIPLE REGRESSION ANALYSIS: MULTIPLE REGRESSION ANALYSIS: The Problem of InferenceThe Problem of Inference

8-3. Hypothesis testing in multiple regression: Testing hypotheses about an individual partial regression coefficientTesting the overall significance of the estimated multiple regression model, that is, finding out if all the partial slope coefficients are simultaneously equal to zeroTesting that two or more coefficients are equal to one anotherTesting that the partial regression coefficients satisfy certain restrictionsTesting the stability of the estimated regression model over time or in different cross-sectional unitsTesting the functional form of regression models



8-4. Hypothesis testing about individual partial regression coefficientsWith the assumption that u i ~ N(0,2) we can use t-test to test a hypothesis about any individual partial regression coefficient.

H0: 2 = 0

H1: 2 0If the computed t value > critical t value at the chosen level of significance, we may reject the null hypothesis; otherwise, we may not reject it



8-5. Testing the overall significance of a multiple regression: The F-Test

For Yi = 1 + 2X2i + 3X3i + ........+ kXki + ui

To test the hypothesis H0: 2 =3 =....= k= 0 (all

slope coefficients are simultaneously zero) versus H1: Not at all slope coefficients are simultaneously zero, computeF=(ESS/df)/(RSS/df)=(ESS/(k-1))/(RSS/(n-k)) (8.5.7) (k = total number of parameters to be estimated including intercept) If F > F critical = F(k-1,n-k), reject H0 Otherwise you do not reject it



8-5. Testing the overall significance of a multiple regression Alternatively, if the p-value of F obtained from (8.5.7) is sufficiently low, one can reject H0 An important relationship between R2 and F: F=(ESS/(k-1))/(RSS/(n-k)) or

R2/(k-1) F = ---------------- (8.5.1) (1-R2) / (n-k) ( see prove on page 249)



8-5. Testing the overall significance of a multiple regression in terms of R2

For Yi = 1 + 2X2i + 3X3i + ........+ kXki + ui To test the hypothesis H0: 2 = 3 = .....= k = 0 (all slope coefficients are simultaneously zero) versus H1: Not at all slope coefficients are simultaneously zero, compute F = [R2/(k-1)] / [(1-R2) / (n-k)] (8.5.13) (k = total number of parameters to be estimated including intercept) If F > F critical = F (k-1,n-k), reject H0



8-5. Testing the overall significance of a multiple regressionAlternatively, if the p-value of F obtained from (8.5.13) is sufficiently low, one can reject H0

The “Incremental” or “Marginal” contribution of an explanatory variable: Let X is the new (additional) term in the right hand of a regression. Under the usual assumption of the normality of ui and the HO: = 0, it can be shown that the following F ratio will follow the F distribution with respectively degree of freedom



8-5. Testing the overall significance of a multiple regression [R2

new - R2old] / Df1

F com = ---------------------- (8.5.18)

[1 - R2new] / Df2

Where Df1 = number of new regressors Df2 = n – number of parameters in the new model

R2new is standing for coefficient of determination of the

new regression (by adding X);

R2old is standing for coefficient of determination of the old

regression (before adding X).


Chapter 8Chapter 8MULTIPLE REGRESSION ANALYSIS: MULTIPLE REGRESSION ANALYSIS: The Problem of InferenceThe Problem of Inference8-5. Testing the overall significance of a multiple regression

Decision Rule:

If F com > F Df1 , Df2 one can reject the Ho that = 0

and conclude that the addition of X to the model significantly increases ESS and hence the R2 value When to Add a New Variable? If |t| of coefficient of X > 1 (or F= t 2 of that variable exceeds 1) When to Add a Group of Variables? If adding a group of variables to the model will give F value greater than 1;


Chapter 8Chapter 8MULTIPLE REGRESSION ANALYSIS: MULTIPLE REGRESSION ANALYSIS: The Problem of InferenceThe Problem of Inference8-6. Testing the equality of two regression coefficients

Yi = 1 + 2X2i + 3X3i + 4X4i + ui (8.6.1)

Test the hypotheses:H0: 3 = 4 or 3 - 4 = 0 (8.6.2)

H1: 3 4 or 3 - 4 0Under the classical assumption it can be shown:

t = [(^3 - ^4) – (3 - 4)] / se(^3 - ^4)follows the t distribution with (n-4) df because (8.6.1) is a four-variable model or, more generally, with (n-k) df. where k is the total number of parameters estimated, including intercept term. se(^3 - ^4) = [var((^3) + var( ^4) – 2cov(^3, ^4)]

(8.6.4) (see appendix)



t = (^3 - ^4) / [var((^3) + var( ^4) – 2cov(^3, ^4)](8.6.5)

Steps for testing:

1. Estimate ^3 and ^4

2. Compute se(^3 - ^4) through (8.6.4)

3. Obtain t- ratio from (8.6.5) with H0: 3 = 4

4. If t-computed > t-critical at designated level of significance for given df, then reject H0. Otherwise do not reject it. Alternatively, if the p-value of t statistic from (8.6.5) is reasonable low, one can reject H0. Example 8.2: The cubic cost function revisited



8-7. Restricted least square: Testing linear equality restrictionsYi = 1X2

2i X33i e

ui (7.10.1) and (8.7.1)

Y = output

X2 = labor input

X3 = capital inputIn the log-form:

lnYi = 0 + 2lnX2i + 3lnX3i + ui (8.7.2)

with the constant return to scale: 2 + 3 = 1

(8.7.3)



8-7. Restricted least square: Testing linear equality restrictionsHow to test (8.7.3) The t Test approach (unrestricted): test of the hypothesis H0:2 + 3

= 1 can be conducted by t- test:

t = [(^2 + ^3) – (2 + 3)] / se(^2 - ^3) (8.7.4)

The F Test approach (restricted least square -RLS): Using, say, 2 = 1-3 and substitute it into (8.7.2) we get: ln(Yi /X2i) = 0 + 3 ln(X3i /X2i) + ui (8.7.8). Where (Yi /X2i) is output/labor ratio, and (X3i / X2i) is capital/labor ratio



8-7. Restricted least square: Testing linear equality restrictionsu^2

UR=RSSUR of unrestricted regression (8.7.2) and u^2

R = RSSR of restricted regression (8.7.7), m = number of linear restrictions, k = number of parameters in the unrestricted regression, n = number of observations. R2

UR and R2R are R2 values obtained from unrestricted and

restricted regressions respectively. ThenF=[(RSSR – RSSUR)/m]/[RSSUR/(n-k)] = = [(R2

UR – R2R) / m] / [1 – R2

UR / (n-k)] (8.7.10)follows F distribution with m, (n-k) df.

Decision rule: If F > F m, n-k , reject H0:2 + 3 = 1



8-7. Restricted least square: Testing linear equality restrictions

Note: R2UR R2

R (8.7.11)

and u^2UR u^2

R (8.7.12) Example 8.3: The Cobb-Douglas Production function for Taiwanese Agricultural Sector, 1958-1972. (pages 259-260). Data in Table 7.3 (page 216) General F Testing (page 260) Example 8.4: The demand for chicken in the US, 1960-1982. Data in exercise 7.23 (page 228)



8-8. Comparing two regressions: Testing for structural stability of regression models

Table 8.8: Personal savings and income data, UK, 1946-1963 (millions of pounds)Savings function: Reconstruction period:

Y t = 1+ 2X t + U1t (t = 1,2,...,n1) Post-Reconstruction period:

Y t = 1 + 2X t + U2t (t = 1,2,...,n2)Where Y is personal savings, X is personal income, the us are disturbance terms in the two equations and n1, n2 are the number of observations in the two period


Chapter 8Chapter 8MULTIPLE REGRESSION ANALYSIS: MULTIPLE REGRESSION ANALYSIS: The Problem of InferenceThe Problem of Inference8-8. Comparing two regressions: Testing for structural stability of regression models+ The structural change may mean that the two intercept are different, or the two slopes are different, or both are different, or any other suitable combination of the parameters. If there is no structural change we can combine all the n1, n2 and just estimate one savings function as:

Y t = 1 + 2X t + Ut (t = 1,2,...,n1, 1,....n2). (8.8.3)How do we find out whether there is a structural change in the savings-income relationship between the two period? A popular test is Chow-Test, it is simply the F Test discussed earlier

HO: i = i i Vs H1: i that i i




+ The assumptions underlying the Chow test u1t and u2t ~ N(0,s2), two error terms are normally distributed with the same variance u1t and u2t are independently distributedStep 1: Estimate (8.8.3), get RSS, say, S1 with df = (n1+n2 – k); k is number of parameters estimated )Step 2: Estimate (8.8.1) and (8.8.2) individually and get their RSS, say, S2 and S3 , with df = (n1 – k) and (n2-k) respectively. Call S4 = S2+S3; with df = (n1+n2 – 2k)Step 3: S5 = S1 – S4;




Step 4: Given the assumptions of the Chow Test, it can be show that

F = [S5 / k] / [S4 / (n1+n2 – 2k)] (8.8.4)follows the F distribution with Df = (k, n1+n2 – 2k)

Decision Rule: If F computed by (8.8.4) > F- critical at the chosen level of significance a => reject the hypothesis that the regression (8.8.1) and (8.8.2) are the same, or reject the hypothesis of structural stability; One can use p-value of the F obtained from (8.8.4) to reject H0 if p-value low reasonably.+ Apply for the data in Table 8.8



8-9. Testing the functional form of regression:

Choosing between linear and log-linear regression models: MWD Test (MacKinnon, White and Davidson)

H0: Linear Model Y is a linear function of regressors, the Xs;

H1: Log-linear Model Y is a linear function of logs of regressors, the lnXs;


Chapter 8Chapter 8MULTIPLE REGRESSION ANALYSIS:MULTIPLE REGRESSION ANALYSIS: The Problem of InferenceThe Problem of Inference8-9. Testing the functional form of regression:

Step 1: Estimate the linear model and obtain the estimated Y values. Call them Yf (i.e.,Y^). Take lnYf.Step 2: Estimate the log-linear model and obtain the estimated lnY values, call them lnf (i.e., ln^Y )Step 3: Obtain Z1 = (lnYf – lnf)

Step 4: Regress Y on Xs and Z1. Reject H0 if the coefficient of Z1 is statistically significant, by the usual t - testStep 5: Obtain Z2 = antilog of (lnf – Yf)

Step 6: Regress lnY on lnXs and Z2. Reject H1 if the coefficient of Z2 is statistically significant, by the usual t-test



Example 8.5: The demand for Roses (page 266-267). Data in exercise 7.20 (page 225)

8-10. Prediction with multiple regressionFollow the section 5-10 and the illustration in pages 267-268 by using data set in the Table 8.1 (page 241)

8-11. The troika of hypothesis tests: The likelihood ratio (LR), Wald (W) and Lagarange Multiplier (LM) Tests


may 2004 prof.vuthieu 1 basic econometrics course leader prof. dr.sc vuthieu

Documents

economic data slide

conjunction of economic

economic studies

sc vuthieu slide

economic mathematical

analysis of economic

theory y

unit slide