Count Models

Sociology 229: Advanced Regression

Copyright © 2010 by Evan SchoferDo not copy or distribute without permission

Announcements

• Assignment #1 Due

• Assignment #2 handed out• Due in 1 week

• Agenda:• Basic count models• Intro to EHA (if time allows)

Count Variables

• Many dependent variables are counts: Non-negative integers

• # Crimes a person has committed in lifetime• # Children living in a household• # new companies founded in a year (in an industry)• # of social protests per month in a city

– Can you think of others?

Count Variables

• Count variables can be modeled with OLS regression… but:– 1. Linear models can yield negative predicted

values… whereas counts are never negative• Similar to the problem of the Linear Probability Model

– 2. Count variables are often highly skewed• Ex: # crimes committed this year… most people are

zero or very low; a few people are very high• Extreme skew violates the normality assumption of

OLS regression.

Count Models

• Two most common count models:• Poisson Regression Model• Negative Binomial Regression Model

• Both based on the Poisson distribution:• = expected count (and variance)

– Called lambda () in some texts; I rely on Freese & Long 2006

• y = observed count

!y

eyP

y

Poisson Regression

• Strategy: Model log of as a function of Xs• Quite similar to modeling log odds in logit• Again, the log form avoids negative values

K

jjijX

1

ln

• Which can be written as:

K

jjijX

e 1

Poisson Regression: Example• Hours per week spent on web

0.0

5.1

.15

.2D

en

sity

0 10 20 30 40 50www hours per week

Poisson Regression: Web Use• Output = similar to logistic regression. poisson wwwhr male age educ lowincome babies

Poisson regression Number of obs = 1552 LR chi2(5) = 525.66 Prob > chi2 = 0.0000Log likelihood = -8598.488 Pseudo R2 = 0.0297

------------------------------------------------------------------------------ wwwhr | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- male | .3595968 .0210578 17.08 0.000 .3183242 .4008694 age | -.0097401 .0007891 -12.34 0.000 -.0112867 -.0081934 educ | .0205217 .004046 5.07 0.000 .0125917 .0284516 lowincome | -.1168778 .0236503 -4.94 0.000 -.1632316 -.0705241 babies | -.1436266 .0224814 -6.39 0.000 -.1876892 -.0995639 _cons | 1.806489 .0641575 28.16 0.000 1.680743 1.932236------------------------------------------------------------------------------

Men spend more time on the web than women

Number of young children in household reduces web use

Poisson Regression: Stata Output

• Stata output yields familiar statistics:– Standard errors, z/t- values, and p-values for

coefficient hypothesis tests– Pseudo R-square for model fit

• Not a great measure… but gives a crude explained variance

– MLE log likelihood– Likelihood ratio test: Chi-square and p-value

• Comparing to null model (constant only)• Tests can also be conducted on nested models with

stata command “lrtest”.

Interpreting Coefficients

• In Poisson Regression, Y is typically conceptualized as a rate…

• Positive coefficients indicate higher rate; negative = lower rate

• Like logit, Poisson models are non-linear• Coefficients don’t have a simple linear interpretation

• Like logit, model has a log form; exponentiation aids interpretation

• Exponentiated coefficients are multiplicative• Analogous to odds ratios… but called “incidence rate

ratios”.

Interpreting Coefficients

• Exponentiated coefficients: indicate effect of unit change of X on rate

• In STATA: “incidence rate ratios”: “poison … , irr”• eb= 2.0 indicates that the rate doubles for each unit

change in X• eb= .5 indicates that the rate drops by half for each unit

change in X

• Recall: Exponentiated coefs are multiplicative• If eb= 5.0, a 2-point change in X isn’t 10; it is 5 * 5 = 25

– Also: you must invert to see opposite effects• If eb= 5.0, a 1-point decrease in X isn’t -5, it is 1/5 = .2

Interpreting Coefficients

• Again, exponentiated coefficients (rate ratios) can be converted to % change

• Formula: (eb - 1) * 100%• Ex: Coefficent = -.693• (e-.693 - 1) * 100% = 50% decrease in rate.

Interpreting Coefficients• Exponentiated coefficients yield multiplier:. poisson wwwhr male age educ lowincome babies

Poisson regression Number of obs = 1552 LR chi2(5) = 525.66 Prob > chi2 = 0.0000Log likelihood = -8598.488 Pseudo R2 = 0.0297

------------------------------------------------------------------------------ wwwhr | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- male | .3595968 .0210578 17.08 0.000 .3183242 .4008694 age | -.0097401 .0007891 -12.34 0.000 -.0112867 -.0081934 educ | .0205217 .004046 5.07 0.000 .0125917 .0284516 lowincome | -.1168778 .0236503 -4.94 0.000 -.1632316 -.0705241 babies | -.1436266 .0224814 -6.39 0.000 -.1876892 -.0995639 _cons | 1.806489 .0641575 28.16 0.000 1.680743 1.932236------------------------------------------------------------------------------

Exponentiation of .359 = 1.43; Rate is 1.43 times higher for men

(1.43-1) * 100 = 43% more

Exp(-.14) = .87. Each baby reduces rate by factor of .87

(.87-1) * 100 = 13% less

Probabilities of Count Outcomes

• Stata extension “prcount” can compute probabilities for each possible count outcome

• For all cases, of for particular groups• It plugs values (m), Xs, & bs into formula:

!

|m

XeXmP

mX

Rate: 5.7446 [ 5.6238, 5.8655] Pr(y=0|x): 0.0032 [ 0.0028, 0.0036] Pr(y=1|x): 0.0184 [ 0.0165, 0.0202] Pr(y=2|x): 0.0528 [ 0.0486, 0.0570] Pr(y=3|x): 0.1011 [ 0.0953, 0.1069] Pr(y=4|x): 0.1452 [ 0.1399, 0.1505] Pr(y=5|x): 0.1668 [ 0.1642, 0.1694] Pr(y=6|x): 0.1597 [ 0.1589, 0.1606] Pr(y=7|x): 0.1311 [ 0.1276, 0.1345] Pr(y=8|x): 0.0941 [ 0.0897, 0.0986] Pr(y=9|x): 0.0601 [ 0.0560, 0.0642]

male age educ lowincome babiesx= .4503866 40.992912 14.345361 .7371134 .20296392

Predicted Counts

• Stata “predict varname, n” computes predicted value for each case

. predict predwww if e(sample), n

. list wwwhr predwww if e(sample)

+------------------+ | wwwhr predwww | |------------------| 1. | 1 5.659943 | 2. | 3 7.090338 | 3. | 2 5.281404 | 12. | 5 6.09473 | 13. | 4 6.968055 | 15. | 3 5.815624 | 16. | 0 5.539187 | 19. | 0 7.207257 | 20. | 8 8.03906 | 21. | 5 4.400002 | 23. | 1 6.77004 | 24. | 1 4.806245 | 25. | 8 5.710855 | 27. | 12 3.687142 | 33. | 40 4.997193 |

Some of the predictions are close to the observed values…

Many of the predictions are quite bad…

Recall that the model fit was VERY poor!

Predicted Counts

• Stata command adjust (Stata 9/10) and margins (Stata 11) can summarize predicted counts

• You can compute average predictions for each case in your data… or for sub-groups of the data.

– The trick is to figure out what values to use for OTHER variables when you compute probabilities

• Hold other variables at the mean of all cases?• Hold other variables at the mean for each subgroup of

the variable of interest?• Set other variables at values corresponding to an

interesting hypothetical case?

Predicted Counts: adjust/margins• Example: comparing women and men. margins , at(male=(0 1)) atmeans

Adjusted predictions Number of obs = 1552

Expression : Predicted number of events, predict()

1._at : male = 0 age = 40.99291 (mean) educ = 14.34536 (mean) lowincome = .1945876 (mean) babies = .2029639 (mean)

2._at : male = 1 age = 40.99291 (mean) educ = 14.34536 (mean) lowincome = .1945876 (mean) babies = .2029639 (mean)------------------------------------------------------------------------------ | Delta-method | Margin Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- _at | 1 | 4.872327 .208613 23.36 0.000 4.463453 5.281201 2 | 6.999391 .3246504 21.56 0.000 6.363087 7.635694------------------------------------------------------------------------------

This prediction refers to men, with other variables held at the mean of all cases

Issue: Exposure

• Poisson outcome variables are typically conceptualized as rates

• Web hours per week• Number of crimes committed in past year

• Issue: Cases may vary in exposure to “risk” of a given outcome

• To properly model rates, we must account for the fact that some cases have greater exposure than others

• Ex: # crimes committed in lifetime– Older people have greater opportunity to have higher counts

• Alternately, exposure may vary due to research design– Ex: Some cases followed for longer time than others…

Issue: Exposure

• Poisson (and other count models) can address varying exposure:

K

jijij tX

ii et 1)ln(

• Where ti = exposure time for case i

• It is easy to incorporate into stata, too:• Ex: poisson NumCrimes SES income, exposure(age)• Note: Also works with other “count” models.

Poisson Model Assumptions

• Poisson regression makes a big assumption: That variance of = (“equidisperson”)

• In other words, the mean and variance are the same• This assumption is often not met in real data• Dispersion is often greater than : overdispersion

– Consequence of overdispersion: Standard errors will be underestimated

• Potential for overconfidence in results; rejecting H0 when you shouldn’t!

• Note: overdispersion doesn’t necessarily affect predicted counts (compared to alternative models).

Poisson Model Assumptions

• Overdispersion is most often caused by highly skewed dependent variables – Often due to variables with high numbers of zeros

• Ex: Number of traffic tickets per year• Most people have zero, some can have 50!• Mean of variable is low, but SD is high

– Other examples of skewed outcomes• # of scholarly publications• # cigarettes smoked per day• # riots per year (for sample of cities in US).

Negative Binomial Regression

• Strategy: Modify the Poisson model to address overdispersion

• Add an “error” term to the basic model:

• Additional model assumptions:• Expected value of exponentiated error = 1 (e = 1)• Exponentiated error is Gamma distributed• We hope that these assumptions are more plausible

than the equidispersion assumption!

K

jijijX

e 1

Negative Binomial Regression

• Full negative biniomial model:

y

y

yXyP

11

1

1

11

!|

• Note that the model incorporates a new parameter:

• Alpha represents the extent of overdispersion• If = 0 the model reduces to simple poisson regression

Negative Binomial Regression

• Question: Is alpha () = 0?• If so, we can use Poisson regression• If not, overdispersion is present; Poisson is inadequate

• Strategy: conduct a statistical test of the hypothesis: H0: = 0; H1: > 0

• Stata provides this information when you run a negative binomial model:

• Likelihood ratio test (G2) for alpha• P-value < .05 indicates that overdispersion is present;

negative binomial is preferred• If P>.05, just use Poisson regression

– So you don’t have to make assumptions about gamma dist….

Negative Binomial Regression

• Interpreting coefficients: Identical to poisson regression

• Predicted probabilities: Can be done. You must use big Neg Binomial formula

• Plugging in observed Xs, estimates of a, Bs…

y

y

yXyP

ˆ

ˆ

ˆ!|ˆ

11

1

1

11

• Probably best to get STATA to do this one…• Long & Freese created command: prvalue

Negative Binomial Example: Web Use• Note: Bs are similar but SEs change a lot!Negative binomial regression Number of obs = 1552 LR chi2(5) = 57.80 Prob > chi2 = 0.0000Log likelihood = -4368.6846 Pseudo R2 = 0.0066

------------------------------------------------------------------------------ wwwhr | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- male | .3617049 .0634391 5.70 0.000 .2373666 .4860433 age | -.0109788 .0024167 -4.54 0.000 -.0157155 -.006242 educ | .0171875 .0120853 1.42 0.155 -.0064992 .0408742 lowincome | -.0916297 .0724074 -1.27 0.206 -.2335457 .0502862 babies | -.1238295 .0624742 -1.98 0.047 -.2462767 -.0013824 _cons | 1.881168 .1966654 9.57 0.000 1.495711 2.266625-------------+---------------------------------------------------------------- /lnalpha | .2979718 .0408267 .217953 .3779907-------------+---------------------------------------------------------------- alpha | 1.347124 .0549986 1.243529 1.459349------------------------------------------------------------------------------Likelihood-ratio test of alpha=0: chibar2(01) = 8459.61 Prob>=chibar2 = 0.000

Note: Standard Error for education increased from .004 to .012! Effect is no longer statistically significant.

Negative Binomial Example: Web Use• Note: Info on overdispersion is providedNegative binomial regression Number of obs = 1552 LR chi2(5) = 57.80 Prob > chi2 = 0.0000Log likelihood = -4368.6846 Pseudo R2 = 0.0066

------------------------------------------------------------------------------ wwwhr | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- male | .3617049 .0634391 5.70 0.000 .2373666 .4860433 age | -.0109788 .0024167 -4.54 0.000 -.0157155 -.006242 educ | .0171875 .0120853 1.42 0.155 -.0064992 .0408742 lowincome | -.0916297 .0724074 -1.27 0.206 -.2335457 .0502862 babies | -.1238295 .0624742 -1.98 0.047 -.2462767 -.0013824 _cons | 1.881168 .1966654 9.57 0.000 1.495711 2.266625-------------+---------------------------------------------------------------- /lnalpha | .2979718 .0408267 .217953 .3779907-------------+---------------------------------------------------------------- alpha | 1.347124 .0549986 1.243529 1.459349------------------------------------------------------------------------------Likelihood-ratio test of alpha=0: chibar2(01) = 8459.61 Prob>=chibar2 = 0.000

Alpha is clearly > 0! Overdispersion is evident; LR test p<.05

You should not use Poisson Regression in this case

General Remarks

• Poisson & Negative binomial models suffer all the same basic issues as “normal” regression

• Model specification / omitted variable bias• Multicollinearity• Outliers/influential cases

– Also, it uses Maximum Likelihood• N > 500 = fine; N < 100 can be worrisome

– Results aren’t necessarily wrong if N<100; – But it is a possibility; and hard to know when problems crop up

• Plus ~10 cases per independent variable.

General Remarks

• It is often useful to try both Poisson and Negative Binomial models

• The latter allows you to test for overdispersion• Use LRtest on alpha () to guide model choice

– If you don’t suspect dispersion and alpha appears to be zero, use Poission Regression

• It makes fewer assumptions– Such as gamma-distributed error.

Example: Labor MilitancyIsaac & Christiansen 2002

Note: Results are presented as % change

Zero-Inflated Poisson & NB Reg

• If outcome variable has many zero values it tends to be highly skewed

• Under those circumstances, NBREG works better than ordinary Poisson due to overdispersion

– But, sometimes you have LOTS of zeros. Even nbreg isn’t sufficient

• Model under-predicts zeros, doesn’t fit well

– Examples:• # violent crimes committed by a person in a year• # of wars a country fights per year• # of foreign subsidiaries of firms.

Zero-Inflated Poisson & NB Reg

• Logic of zero-inflated models: Assume two types of groups in your sample

• Type A: Always zero – no probability of non-zero value• Type ~A: Non-zero chance of positive count value

– Probability is variable, but not zero

– 1. Use logit to model group membership – 2. Use poisson or nbreg to model counts for

those in group ~A– 3. Compute probabilities based on those results.

Zero-Inflated Poisson & NB Reg• Example: Web usage at work

• More skewed than overall web usage. Why?

0.1

.2.3

De

nsity

0 20 40 60hours per week using work computer www

Many people don’t have computers at work!

So, web usage is zero for many

Zero-Inflated Poisson & NB Reg

• Zero-inflated models in Stata• “zip” = Poisson, zinb = negative binomial

• Commands accept two separate variable lists– Variables that affect counts

• For those with non-zero counts• Modeled with Poisson or NB regression

– Variables that predict membership in “zero” group• Modeled with logit

– Ex: zinb webatwork male age educ lowincome babies, inflate(male age educ lowincome babies)

ZINB Example: Web Hrs at Work• “Inflate” output = logit for group membershipZero-inflated negative binomial regression Number of obs = 1135 Nonzero obs = 562 Zero obs = 573

Inflation model = logit LR chi2(5) = 13.25Log likelihood = -2239.23 Prob > chi2 = 0.0212------------------------------------------------------------------------------ | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------webatwork | male | .2348353 .1298324 1.81 0.070 -.0196315 .4893021 age | -.0152071 .0053766 -2.83 0.005 -.0257451 -.0046692 educ | .0126503 .0265321 0.48 0.634 -.0393517 .0646523 lowincome | -.4183108 .2164324 -1.93 0.053 -.8425105 .0058889 babies | .0588977 .1385245 0.43 0.671 -.2126053 .3304008 _cons | 1.703158 .4538886 3.75 0.000 .8135524 2.592763-------------+----------------------------------------------------------------inflate | male | .2630493 .340892 0.77 0.440 -.4050866 .9311853 age | -.0197401 .0195075 -1.01 0.312 -.057974 .0184939 educ | -.3601863 .071167 -5.06 0.000 -.4996711 -.2207015 lowincome | .844378 .4013074 2.10 0.035 .0578299 1.630926 babies | .4504404 .2502363 1.80 0.072 -.0400138 .9408947 _cons | 4.137417 1.172503 3.53 0.000 1.839354 6.43548

Education reduces odds of zero value

But doesn’t have an effect on count for those that are non-zero

Model predicting zero group

Zero-Inflated Poisson & NB Reg• Remarks

– ZINB produces estimate of alpha• Helps choose between zip & zinb

– Long and Freese (2006) have helpful tool to compare fit of count models: countfit

• See textbook

– Zero-inflated models seem very useful• Count variables often have many zeros• It is often reasonable to assume a “always zero” group

– But, they are fairly new• Not many examples in the literature• Haven’t been widely scrutinized.

Zero-truncated Poisson & NB reg• Truncation – the absence of information

about cases in some range of a variable• Example: Suppose we study income based on data

from tax returns… – Cases with income below a certain value are not required to

submit a tax return… so data is missing

• Example: Data on # crimes committed, taken from legal records

– Individuals with zero crimes are not evident in data

• Example: An on-line survey of web use– Individuals with zero web use are not in data

• Poisson & NB have been adapted to address truncated data:– Zero-truncated Poisson & Zero-trunciated NB reg.

Example: Zero-truncated NB Reg• Web use (zeros removed)Zero-truncated negative binomial regression Number of obs = 1304 LR chi2(5) = 34.87Dispersion = mean Prob > chi2 = 0.0000Log likelihood = -3653.162 Pseudo R2 = 0.0047

------------------------------------------------------------------------------ wwwhr | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- male | .3744582 .0874595 4.28 0.000 .2030407 .5458758 age | -.0114399 .0033817 -3.38 0.001 -.0180679 -.0048119 educ | .0081191 .016731 0.49 0.627 -.024673 .0409112 lowincome | .1899431 .1111248 1.71 0.087 -.0278574 .4077437 babies | -.1375942 .0860954 -1.60 0.110 -.306338 .0311496 _cons | 1.533013 .2907837 5.27 0.000 .9630872 2.102938-------------+---------------------------------------------------------------- /lnalpha | 1.099164 .1385789 .8275543 1.370774-------------+---------------------------------------------------------------- alpha | 3.001656 .4159661 2.287717 3.938396------------------------------------------------------------------------------Likelihood-ratio test of alpha=0: chibar2(01) = 6857.67 Prob>=chibar2 = 0.000

Coefficient interpretation works just like ordinary poisson or NB regression.

Empirical Example 2

• Example: Haynie, Dana L. 2001. “Delinquent Peers Revisited: Does Network Structure Matter?” American Journal of Sociology, 106, 4:1013-1057.