aqm 2008 lecture random

7/23/2019 Aqm 2008 Lecture Random

http://slidepdf.com/reader/full/aqm-2008-lecture-random 1/121

Fixed and Random Effects

Jos Elkink

April, 2008

http://find/

http://goback/



1 Introduction

2 Motivation

3 Fixed effects

4 Random effects

5

Random coefficients

6 Further information

http://find/



Outline

1 Introduction

2 Motivation

3 Fixed effects

4 Random effects

5 Random coefficients


http://find/



Four topics

Missing data March 27

Fixed & random effects April 3Time-series models April 10

Causation and inference April 17

http://find/



Outline

1 Introduction

2 Motivation

3 Fixed effects

4 Random effects



http://find/



Motivations

Clustered sampling

http://find/



Sampling strategies

Probability sampling:

http://find/



Sampling strategies


Simple random sampling

http://find/

http://goback/




The sampling here is a purely random

selection from the sampling frame, selected

without replacement.

http://find/






without replacement.Each subject from a population has the exact

same chance of being selected in the sample.

http://find/

http://goback/






without replacement.Each subject from a population has the exact

same chance of being selected in the sample.

The sample probability for each subject is the

same.

http://find/



Sampling strategies


Simple random samplingSystematic random sampling

http://find/

http://goback/



Sampling strategies



Stratified sampling

http://find/

http://goback/



Sampling strategies



Stratified sampling

Cluster sampling

http://find/

http://goback/



Cluster sampling

To reduce costs, clusters are (randomly)sampled first, before lower levels are

clustered.

http://find/

http://goback/



Cluster sampling


clustered.

E.g. selecting schools before selecting

students, so that fewer schools need to be

visited.

http://find/

http://goback/



Cluster sampling


clustered.

E.g. selecting schools before selecting

students, so that fewer schools need to be

visited.Individual observations from a clustered

sample are not independent .

http://find/

http://goback/



Motivations

Clustered sampling

Inherent structure

http://find/



Examples

schools teachers

classes pupils

firms employees

countries political parties

doctors patients

subjects measurementsinterviewers respondents

judges suspects

http://find/

http://goback/



Motivations

Clustered sampling

Inherent structure

Panel data

http://find/

http://goback/



Motivations

Clustered sampling

Inherent structure

Panel data

Time-Series Cross-Section

http://find/

http://goback/



Multilevel characteristics

Observations are not completely

independent

http://find/





independent

Variance can be divided inbetween-group and within-group

variances

http://find/





independent

Variance can be divided inbetween-group and within-group

variances

Variables can be measured at eithermicro- or marco-level, or both

http://find/



Example

−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

http://find/

http://goback/



Overall mean

−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

http://find/

http://goback/



Group means

−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

http://goforward/

http://find/



Between variation

−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

s ope = − .

http://find/

http://goback/



Group means

−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

http://find/

http://goback/



Within variation

−4 −2 0 2 4 6 8

− 1 .

5

− 1 . 0

− 0 .

5

0 .

0

0

. 5

1 .

0

x

y .

d e v

http://find/

http://goback/



Outline1

Introduction2 Motivation

3 Fixed effects

4 Random effects



http://find/



Pooled model

When we simply run a regression using all

micro-level data, ignoring the multilevel

structure, we call this a pooled model.

http://find/

http://goback/



Pooled model

If we have some observations at the

macro-level, we are artificially increasing the

number of observations.

P l d d l

http://find/



Pooled model



number of observations. Thus we will beoverconfident in our results.

P l d d l

http://find/



Pooled model



number of observations. Thus we will beoverconfident in our results.

E.g. characteristics of judges in explainingthe severity of court rulings.

P l d d l

http://find/



Pooled model

−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

s ope = .

Fi d ff d l

http://find/



Fixed effects model

With a fixed effects model we explain thewithin-group variation, removing the

between-group variation by:

Fi d ff d l

http://find/



Fixed effects model



Adding dummy variables for each group

Fi d ff t d l

http://find/



Fixed effects model




Subtracting the group means from all

variables

Fi d ff t d l

http://find/

http://goback/



Fixed effects model




Subtracting the group means from all

variables

The two are equivalent.

http://find/



Fi d ff ts d l



Fixed effects model

In essence, we thus have different interceptsfor each group.

y i = β 0 + X i β + µ j [i ] + εi ,

whereby i denotes the individual unit, j the

group, and j [i ] the group of i .

Fixed effects model

http://find/



Fixed effects model

In essence, we thus have different interceptsfor each group.

y i = β 0 + X i β + µ j [i ] + εi ,

whereby i denotes the individual unit, j the

group, and j [i ] the group of i .

If the fixed effects model is the true model,

pooled estimates are biased and inconsistent.

Pooled model

http://find/



Pooled model

−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

s ope = .

Fixed effects model (1)

http://find/




−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

s ope = .


http://find/




−6 −4 −2 0 2 4

− 1 .

5

− 1 .

0

− 0 .

5

0 .

0

0 .

5

1 .

0

x.dev

y .

d e v

s ope = .

http://find/



Between effects model




Another way of dealing with clustered data islooking at the between model:

¯ y j = β 0 + X j β + ε j

Typical mistake: conclusions aboutindividuals from aggregate data - ecological

fallacy.


http://find/




−4 −2 0 2 4 6 8

1

2

3

4

5

6

x

y

s ope = − .

http://find/



Fixed effects in Stata




xtreg grade aptitude age, i(school) fe


http://find/




xtreg grade aptitude age, i(school) fe

Or, “manually”:

xi: reg grade aptitude age i.school

http://find/



Group-level variables



Group level variables

Note that fixed effects models cannot deal

with group-level variables.


http://find/



Group level variables

Note that fixed effects models cannot deal

with group-level variables.

The effect would be perfect multicollinearity .

http://find/



p

In most cases with group-level variables,

however, a random effects or randomintercept model is more appropriate.

Outline

http://find/



1 Introduction

2 Motivation

3

Fixed effects4 Random effects



http://find/



Random effects



For the random effects model we still have:

y i = β 0 + X i β + µ j [i ] + εi .

However, this time we assume µ j ∼ N (0, σ2µ).

By assuming that µ j comes from a normal

distribution, we have fewer parameters to

estimate (only one σ2µ instead of J µ’s).

Variance components

http://find/



p

In the population, the variance of the

dependent variable can be split in

within-group and between-group variance:

σ

2

Y = σ

2

between + σ

2

within

Intraclass correlation

http://find/



Aside: the proportion of the variance that is

accounted for by the group level is the

intraclass correlation.

ρintra =

σ2between

σ2between + σ2within

Variance estimators

http://find/



σ2within = s 2within

Variance estimators

http://find/



σ2within = s 2within

σ2between = s 2between −

s 2withinn ,

where

n = n −s 2n j

N n

Fixed vs random

http://find/



When to use random effects?

Fixed vs random

http://find/




A group effect is random if we can think

of the levels we observe in that group tobe samples from a larger population.

Fixed vs random

http://find/






When making out-of-sample inferences.

Fixed vs random

http://find/







When there are group-level variables.

Fixed vs random

http://find/







When there are group-level variables.When the sizes of groups are small.

Fixed vs random

http://find/




Alternatively, one can primarily look at n j

and N :N small fixed effects

N not small, n j small random effects

n j larger not as importantBut this is only a preliminary quick judgment!

Fixed vs random

http://find/




Gelman & Hill (2007): “Our advice (...) is toalways use multilevel modeling (’random

effects’).”

http://find/



Random effects in R



library(arm)lmer(grade ~ aptitude + age + (1|school))

Random effects in Stata

http://find/



xtreg grade aptitude age, i(school) re

xtreg grade aptitude age, i(school) re mle

School example

http://find/



Note that we are talking about 3 schools -

this is too few groups to seriously consider arandom effects model!

School example

http://find/



Linear mixed-effects model fit by REML

Random effects:

Groups Name Variance Std.Dev.

school (Intercept) 1.737 1.318

Residual 0.293 0.542number of obs: 60, groups: school, 3

Fixed effects:

Estimate Std. Error t value(Intercept) 3.0259 1.1360 2.66

aptitude 0.9216 0.0723 12.75

age 0.2020 0.0675 2.99

http://find/



School example (fixed)



−2 −1 0 1 2

2

3

4

5

6

7

8

9

aptitude

g r a d e

School example (random)

http://find/



−2 −1 0 1 2

2

3

4

5

6

7

8

9

aptitude

g r a d e

School example

http://goforward/

http://find/

http://goback/



Random-effects ML regression Number of obs = 60Group variable (i): school Number of groups = 3

Random effects u_i ~ Gaussian Obs per group: min = 20avg = 20.0

max = 20

LR chi2(2) = 83.55Log likelihood = -53.896431 Prob > chi2 = 0.0000

------------------------------------------------------------------------------grade | Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+----------------------------------------------------------------aptitude | .9210743 .0710091 12.97 0.000 .781899 1.06025

age | .2022943 .0662681 3.05 0.002 .0724112 .3321775_cons | 3.021863 1.034865 2.92 0.003 .993565 5.050161

-------------+----------------------------------------------------------------/sigma_u | 1.073727 .4438302 .4775795 2.414027/sigma_e | .5320764 .0498341 .4428439 .639289

rho | .8028508 .1342531 .4616776 .9640621------------------------------------------------------------------------------Likelihood-ratio test of sigma_u=0: chibar2(01)= 83.34 Prob>=chibar2 = 0.000

R-squared

http://find/



In linear regression, a popular statistics is R 2

,which is the squared multiple correlation

coefficient

http://goforward/

http://find/

http://goback/



R-squared



Remember, variance of a multilevel model

has different components:

σ2Y = σ2

between + σ2within

R-squared: individuallevel

http://find/



level

Estimated two models, one with and one

without explanatory variables (A and B ,

respectively).

Then,

R 2within = σ2

µ,A + σ2

ε,A

σ2µ,B + σ2

ε,B

http://find/



Predicted random effects



With a fixed effects model, we have the

coefficients on the group dummies which we

can interpret as group-level predictors.


http://find/



With a fixed effects model, we have the

coefficients on the group dummies which we

can interpret as group-level predictors.In a random effects model, we do not have

these predictions, as we only estimated σ2µ

and β 0.


http://find/



The predicted group levels can be estimatedusing:

β 0, j = λ j ¯ y j + (1 − λ j )β 0

λ j =σ2µ

σ2µ + σ2

ε/n j ,

whereby ¯ y j is the mean on y of group j .


I R h i d fi d ff i h

http://find/



In R, you get the estimated fixed effects with:

model.random <- lmer(y ~ x1 + x2 + (1|

fixef(model.random)

and the predicted random effects with:

ranef(model.random)

http://find/



Random coefficients



In the random effects model, we assume thatgroup intercepts vary according to a normal

distribution.

Random coefficients

http://find/




distribution.

But what about the coefficients?

Random coefficients

http://find/




distribution.

But what about the coefficients?

I.e. what about group slopes that vary

following a normal distribution?

Random coefficients

http://find/



y i = β 0 + X i β + X i γ j [i ] + µ j [i ] + εi

µ j ∼ N (0, σ2µ)

γ j ∼ N (0, σ2γ )

Random coefficients

http://find/



y i = β 0 + X i β + X i γ j [i ] + µ j [i ] + εi

µ j ∼ N (0, σ2µ)

γ j ∼ N (0, σ2γ )

Note that a model with random coefficients,

but a constant intercept across groups rarely

makes sense, especially because of the often

arbitrary location if x = 0.

Random effects in R

http://find/



library(arm)

lmer(grade ~ aptitude + age + (aptitude|school))

School example

Linear mixed-effects model fit by REML

http://find/



Linear mixed effects model fit by REML

Random effects:

Groups Name Variance Std.Dev. Corr

school (Intercept) 1.74e+00 1.32e+00

aptitude 1.47e-10 1.21e-05 0.000

Residual 2.93e-01 5.42e-01

number of obs: 60, groups: school, 3

Fixed effects:

Estimate Std. Error t value(Intercept) 3.0259 1.1359 2.66

aptitude 0.9216 0.0723 12.75

age 0.2020 0.0675 2.99

School example (random)

http://find/



−2 −1 0 1 2

2

3

4

5

6

7

8

9

aptitude

g r a d e

Example

http://find/



−5 0 5 10

− 5

0

5

1 0

x

y

Pooled model

http://find/



−5 0 5 10

− 5

0

5

1 0

x

y

s ope = .

Fixed effects model

http://find/



−5 0 5 10

− 5

0

5

1 0

x

y

s ope = .

Random effects model

http://find/



−5 0 5 10

− 5

0

5

1 0

x

y

s ope = .

Random coefficientsmodel

http://find/



− 5

0

5

1 0

y

mean s ope = .

http://find/



Outline1 Introduction



2 Motivation

3 Fixed effects

4 Random effects

5

Random coefficients6 Further information

Important other topics

http://find/



Time-dependence within groups (nextweek)


http://find/




Predictors on the random coefficients


http://find/





Bayesian estimation


http://find/





Bayesian estimationMore complex models dealing with panel

data structures


http://find/





Bayesian estimationMore complex models dealing with panel

data structures

Extensions towards limited dependent

variables

Further information

A clear, relatively introductory textbook on

http://find/



, y y

multilevel modeling is Snijders & Bosker

(1999), Multilevel analysis. An introduction

to basic and advanced multilevel modeling .

Further information

A clear, relatively introductory textbook on

http://find/



, y y

multilevel modeling is Snijders & Bosker

(1999), Multilevel analysis. An introduction

to basic and advanced multilevel modeling .An excellent, modern book on multilevel

modeling, using primarily R and Bugs, is

Gelman & Hill (2007), Data analysis using regression and multilevel/hierarchical models .

Further information

http://find/



Their websites are also interesting:

Snijders: http://stat.gamma.rug.nl/snijders/

Gelman: http://www.stat.columbia.edu/ gelman/

Further information

When using Stata, the

http://find/



g

Longitudinal/panel-data reference manual is

of very high quality. The relevant chapters

for this lecture are in fact freely available assample chapters (xtreg and xtmixed) at

http://www.stata.com/bookstore/xt.html.

http://find/



Further information

Two standard textbooks on panel data are



Baltagi (2005), Econometric analysis of

panel data (primarily for small N , large T )

and Hsiao (2003), Analysis of panel data

(primarily for large N , small T ). Both are

very technical in nature. Perhaps an easier

introduction is Wooldridge (2002),

Econometric analysis of cross-section and

panel-data.

http://find/

aqm 2008 lecture random

Documents