9-9-11

MGMG 522 : Session #9MGMG 522 : Session #9Binary RegressionBinary Regression

(Ch. 13)(Ch. 13)

9-9-22

Dummy Dependent VariableDummy Dependent Variable Up to now, our dependent variable is continuous.Up to now, our dependent variable is continuous. In some study, our dependent variable may take In some study, our dependent variable may take

on a few values.on a few values. We will deal with a case where a dependent We will deal with a case where a dependent

variable takes on the values of zero and one only in variable takes on the values of zero and one only in this session.this session.

Note:Note:– There are other types of regression that deal There are other types of regression that deal

with a dependent variable that takes on, say, 3-with a dependent variable that takes on, say, 3-4 values.4 values.

– The dependent variable needs not be a The dependent variable needs not be a quantitative variable, it could be a qualitative quantitative variable, it could be a qualitative variable as well.variable as well.

9-9-33

Linear Probability ModelLinear Probability Model

Example: DExample: Dii==00++11XX1i1i++22XX2i2i++εεii -- (1) -- (1)

DDii is a dummy variable. is a dummy variable. If we run OLS of (1), this is a “Linear If we run OLS of (1), this is a “Linear

Probability Model.”Probability Model.”

9-9-44

Problem of Linear Probability ModelProblem of Linear Probability Model1.1. The error term is not normally The error term is not normally

distributed.distributed. This violates the classical assumption #7.This violates the classical assumption #7. In fact, the error term is binomially In fact, the error term is binomially

distributed.distributed. Hence, hypothesis testing becomes Hence, hypothesis testing becomes

unreliable.unreliable.

2.2. The error term is heteroskedastic.The error term is heteroskedastic. Var(Var(εεii) = P) = Pii(1-P(1-Pii), where P), where Pii is the probability is the probability

that Dthat Dii = 1. = 1. PPii changes from one observation to another, changes from one observation to another,

therefore, Var(therefore, Var(εεii) is not constant.) is not constant. This violates the classical assumption #5.This violates the classical assumption #5.

9-9-55

3.3. RR22 is not a reliable measure of overall fit. is not a reliable measure of overall fit. RR22 reported from OLS will be lower than the reported from OLS will be lower than the

true Rtrue R22.. For an exceptionally good fit, RFor an exceptionally good fit, R22 reported reported

from OLS can be much lower than 1.from OLS can be much lower than 1.

4.4. is not bounded between zero and one.is not bounded between zero and one. Substituting values for XSubstituting values for X11 and X and X22 into the into the

regression equation, we could get > 1 or regression equation, we could get > 1 or

< 0.< 0.

iD̂

iD̂

iD̂

Problem of Linear Probability ModelProblem of Linear Probability Model

9-9-66

Remedies for Problems 1-2Remedies for Problems 1-21.1. The error term is not normal.The error term is not normal.

OLS estimator does not require that the error OLS estimator does not require that the error term be normally distributed.term be normally distributed.

OLS is still BLUE even though the classical OLS is still BLUE even though the classical assumption #7 is violated.assumption #7 is violated.

Hypothesis testing is still questionable, however.Hypothesis testing is still questionable, however.

2.2. The error term is heteroskedastic.The error term is heteroskedastic. We can use WLS: Divide (1) through byWe can use WLS: Divide (1) through by But we don’t know PBut we don’t know Pii, but we know that P, but we know that Pii is the is the

probability that Dprobability that Dii = 1. = 1. So, we will divide (1) through by So, we will divide (1) through by

DDii/Z/Zii==00++00/Z/Zii++11XX1i1i/Z/Zii++22XX2i2i/Z/Zii+u+uii : u : uii = = εεii/Z/Zii

can be obtained from substituting Xcan be obtained from substituting X11 and X and X22 into the regression equation.into the regression equation.

)1( ii PP

)ˆ1(ˆ iii DDZ

iD̂

9-9-77

Remedies for Problems 3-4Remedies for Problems 3-43.3. RR22 is lower than actual. is lower than actual.

Use RUse RPP22 = the percentage of observations = the percentage of observations

being predicted correctly.being predicted correctly. Set >= .5 to predict DSet >= .5 to predict Dii = 1 and < .5 to = 1 and < .5 to

predict Dpredict Dii = 0. OLS result does not report R = 0. OLS result does not report RPP22

automatically, you must calculate it by hand.automatically, you must calculate it by hand.

4.4. is not bounded between zero and one.is not bounded between zero and one. Follow this rule to avoid unboundedness Follow this rule to avoid unboundedness

problem.problem.

If > 1, then DIf > 1, then Dii = 1. = 1.

If < 0, then DIf < 0, then Dii = 0. = 0.

iD̂

iD̂

iD̂

iD̂

iD̂

9-9-88

Binomial Logit ModelBinomial Logit Model To deal with unboundedness problem, we To deal with unboundedness problem, we

need another type of regression that need another type of regression that mitigates the unboundedness problem, mitigates the unboundedness problem, called Binomial Logit model.called Binomial Logit model.

Binomial Logit model deals with Binomial Logit model deals with unboundedness problem by using a unboundedness problem by using a variant of the cumulative logistic function.variant of the cumulative logistic function.

We no longer model DWe no longer model Dii directly. directly. We will use ln[DWe will use ln[Dii/(1-D/(1-Dii)] instead of D)] instead of Dii.. Our model becomesOur model becomes

ln[Dln[Dii/(1-D/(1-Dii)]=)]=00++11XX1i1i++22XX2i2i++εεii --- (2) --- (2)

9-9-99

How does Logit model solve How does Logit model solve unbounded problem?unbounded problem?

ln[Dln[Dii/(1-D/(1-Dii)]=)]=00++11XX1i1i++22XX2i2i++εεii --- (2) --- (2) It can be shown that (2) can be written asIt can be shown that (2) can be written as

If the value in [..] = +∞, DIf the value in [..] = +∞, Di i = 1.= 1.

If the value in [..] = –∞, DIf the value in [..] = –∞, Di i = 0.= 0. Unboundedness problem is now solved.Unboundedness problem is now solved.

][ 221101

1iii XXi e

D See 13-4 See 13-4

on p. 601 on p. 601 for proof.for proof.

9-9-1010

Logit EstimationLogit Estimation Logit estimation of coefficients cannot be done by Logit estimation of coefficients cannot be done by

OLS due to non-linearity in coefficients.OLS due to non-linearity in coefficients. Use Maximum Likelihood Estimator (MLE) instead Use Maximum Likelihood Estimator (MLE) instead

of OLS.of OLS. MLE is consistent and asymptotically efficient MLE is consistent and asymptotically efficient

(unbiased and minimum variance for large (unbiased and minimum variance for large samples).samples).

It can be shown that for a linear equation that It can be shown that for a linear equation that meets all 6 classical assumptions plus normal meets all 6 classical assumptions plus normal error term assumption, OLS and MLE will produce error term assumption, OLS and MLE will produce identical coefficient estimates.identical coefficient estimates.

Logit estimation works well for large samples, Logit estimation works well for large samples, typically 500 observations or more.typically 500 observations or more.

9-9-1111

Logit: InterpretationsLogit: Interpretations 11 measures the impact of one unit measures the impact of one unit

increase in Xincrease in X11 on the ln[D on the ln[Dii/(1-D/(1-Dii)], holding )], holding other Xs constant.other Xs constant.

We still cannot use RWe still cannot use R22 to compare overall to compare overall goodness of fit because the variable goodness of fit because the variable ln[Dln[Dii/(1-D/(1-Dii)] is not the same as D)] is not the same as Dii in a in a linear probability model.linear probability model.

Even we use Quasi-REven we use Quasi-R22, the value of Quasi-, the value of Quasi-RR22 we calculated will be lower than its true we calculated will be lower than its true value.value.

Use RUse RPP22 instead. instead.

9-9-1212

Binomial Probit ModelBinomial Probit Model

Binomial Probit model deals with Binomial Probit model deals with unboundedness problem by using a variant unboundedness problem by using a variant of the cumulative normal distribution.of the cumulative normal distribution.

--- (3)--- (3)

Where, PWhere, Pii = probability that D = probability that Dii = 1 = 1

ZZii = = 00++11XX1i1i++22XX2i2i+…+…

s = a standardized normal variables = a standardized normal variable

dsePiZ

s

i

2

2

2

1

9-9-1313

(3) can be rewritten as Z(3) can be rewritten as Zii = F = F-1-1(P(Pii)) Where FWhere F-1-1 is the inverse of the normal is the inverse of the normal

cumulative distribution function.cumulative distribution function. We also need MLE to estimate We also need MLE to estimate

coefficients.coefficients.

9-9-1414

Similarities and Differences Similarities and Differences between Logit and Probitbetween Logit and Probit

SimilaritiesSimilarities– Graphs of Logit and Probit look very similar.Graphs of Logit and Probit look very similar.– Both need MLE to estimate coefficients.Both need MLE to estimate coefficients.– Both need large samples.Both need large samples.– RR22s produced by Logit and Probit are not an appropriate s produced by Logit and Probit are not an appropriate

measure of overall fit.measure of overall fit. DifferencesDifferences

– Probit takes more computer time to estimate coefficients Probit takes more computer time to estimate coefficients than Logit.than Logit.

– Probit is more appropriate for normally distributed Probit is more appropriate for normally distributed variables.variables.

– However, for an extremely large sample set, most However, for an extremely large sample set, most variables become normally distributed. The extra variables become normally distributed. The extra computer time required for running Probit regression is computer time required for running Probit regression is not worth the benefits of normal distribution assumption.not worth the benefits of normal distribution assumption.