aqm 2008 lecture random

7/23/2019 Aqm 2008 Lecture Random

http://slidepdf.com/reader/full/aqm-2008-lecture-random 1/121

Fixed and Random Effects

Jos Elkink

April, 2008

1 Introduction

2 Motivation

3 Fixed effects

4 Random effects

Random coefficients

6 Further information

Outline

1 Introduction

2 Motivation

3 Fixed effects

4 Random effects

5 Random coefficients

Four topics

Missing data March 27

Fixed & random effects April 3Time-series models April 10

Causation and inference April 17

Outline

1 Introduction

2 Motivation

3 Fixed effects

4 Random effects

Motivations

Clustered sampling

Sampling strategies

Probability sampling:

Sampling strategies

Simple random sampling

The sampling here is a purely random

selection from the sampling frame, selected

without replacement.

without replacement.Each subject from a population has the exact

same chance of being selected in the sample.

without replacement.Each subject from a population has the exact

same chance of being selected in the sample.

The sample probability for each subject is the

Sampling strategies

Simple random samplingSystematic random sampling

Sampling strategies

Stratified sampling

Sampling strategies

Stratified sampling

Cluster sampling

To reduce costs, clusters are (randomly)sampled first, before lower levels are

clustered.

Cluster sampling

clustered.

E.g. selecting schools before selecting

students, so that fewer schools need to be

visited.

Cluster sampling

clustered.

E.g. selecting schools before selecting

students, so that fewer schools need to be

visited.Individual observations from a clustered

sample are not independent .

Motivations

Clustered sampling

Inherent structure

Examples

schools teachers

classes pupils

firms employees

countries political parties

doctors patients

subjects measurementsinterviewers respondents

judges suspects

Motivations

Clustered sampling

Inherent structure

Panel data

Motivations

Clustered sampling

Inherent structure

Panel data

Time-Series Cross-Section

Multilevel characteristics

Observations are not completely

independent

Variance can be divided inbetween-group and within-group

variances

independent

Variance can be divided inbetween-group and within-group

variances

Variables can be measured at eithermicro- or marco-level, or both

Example

−4 −2 0 2 4 6 8

Overall mean

−4 −2 0 2 4 6 8

Group means

−4 −2 0 2 4 6 8

Between variation

−4 −2 0 2 4 6 8

s ope = − .

Group means

−4 −2 0 2 4 6 8

Within variation

−4 −2 0 2 4 6 8

− 1 .

− 1 . 0

− 0 .

Outline1

Introduction2 Motivation

3 Fixed effects

4 Random effects

Pooled model

When we simply run a regression using all

micro-level data, ignoring the multilevel

structure, we call this a pooled model.

Pooled model

If we have some observations at the

macro-level, we are artificially increasing the

number of observations.

P l d d l

Pooled model

number of observations. Thus we will beoverconfident in our results.

P l d d l

Pooled model

number of observations. Thus we will beoverconfident in our results.

E.g. characteristics of judges in explainingthe severity of court rulings.

P l d d l

Pooled model

−4 −2 0 2 4 6 8

s ope = .

Fi d ff d l

Fixed effects model

With a fixed effects model we explain thewithin-group variation, removing the

between-group variation by:

Fi d ff d l

Fixed effects model

Adding dummy variables for each group

Fi d ff t d l

Fixed effects model

Subtracting the group means from all

variables

Fi d ff t d l

Fixed effects model

Subtracting the group means from all

variables

The two are equivalent.

Fi d ff ts d l

Fixed effects model

In essence, we thus have different interceptsfor each group.

y i = β 0 + X i β + µ j [i ] + εi ,

whereby i denotes the individual unit, j the

group, and j [i ] the group of i .

Fixed effects model

In essence, we thus have different interceptsfor each group.

y i = β 0 + X i β + µ j [i ] + εi ,

whereby i denotes the individual unit, j the

group, and j [i ] the group of i .

If the fixed effects model is the true model,

pooled estimates are biased and inconsistent.

Pooled model

−4 −2 0 2 4 6 8

s ope = .

Fixed effects model (1)

−4 −2 0 2 4 6 8

s ope = .

−6 −4 −2 0 2 4

− 1 .

− 0 .

s ope = .

Between effects model

Another way of dealing with clustered data islooking at the between model:

¯ y j = β 0 + X j β + ε j

Typical mistake: conclusions aboutindividuals from aggregate data - ecological

fallacy.

−4 −2 0 2 4 6 8

s ope = − .

Fixed effects in Stata

xtreg grade aptitude age, i(school) fe

Or, “manually”:

xi: reg grade aptitude age i.school

Group-level variables

Group level variables

Note that fixed effects models cannot deal

with group-level variables.

Group level variables

Note that fixed effects models cannot deal

with group-level variables.

The effect would be perfect multicollinearity .

In most cases with group-level variables,

however, a random effects or randomintercept model is more appropriate.

Outline

1 Introduction

2 Motivation

Fixed effects4 Random effects

Random effects

For the random effects model we still have:

y i = β 0 + X i β + µ j [i ] + εi .

However, this time we assume µ j ∼ N (0, σ2µ).

By assuming that µ j comes from a normal

distribution, we have fewer parameters to

estimate (only one σ2µ instead of J µ’s).

Variance components

In the population, the variance of the

dependent variable can be split in

within-group and between-group variance:

Y = σ

between + σ

within

Intraclass correlation

Aside: the proportion of the variance that is

accounted for by the group level is the

intraclass correlation.

ρintra =

σ2between

σ2between + σ2within

Variance estimators

σ2within = s 2within

Variance estimators

σ2within = s 2within

σ2between = s 2between −

s 2withinn ,

n = n −s 2n j

Fixed vs random

When to use random effects?

Fixed vs random

A group effect is random if we can think

of the levels we observe in that group tobe samples from a larger population.

Fixed vs random

When making out-of-sample inferences.

Fixed vs random

When there are group-level variables.

Fixed vs random

When there are group-level variables.When the sizes of groups are small.

Fixed vs random

Alternatively, one can primarily look at n j

and N :N small fixed effects

N not small, n j small random effects

n j larger not as importantBut this is only a preliminary quick judgment!

Fixed vs random

Gelman & Hill (2007): “Our advice (...) is toalways use multilevel modeling (’random

effects’).”

Random effects in R

library(arm)lmer(grade ~ aptitude + age + (1|school))

Random effects in Stata

xtreg grade aptitude age, i(school) re

xtreg grade aptitude age, i(school) re mle

School example

Note that we are talking about 3 schools -

this is too few groups to seriously consider arandom effects model!

School example

Linear mixed-effects model fit by REML

Random effects:

Groups Name Variance Std.Dev.

school (Intercept) 1.737 1.318

Residual 0.293 0.542number of obs: 60, groups: school, 3

Fixed effects:

Estimate Std. Error t value(Intercept) 3.0259 1.1360 2.66

aptitude 0.9216 0.0723 12.75

age 0.2020 0.0675 2.99

School example (fixed)

−2 −1 0 1 2

aptitude

g r a d e

School example (random)

−2 −1 0 1 2

aptitude

g r a d e

School example

Random-effects ML regression Number of obs = 60Group variable (i): school Number of groups = 3

Random effects u_i ~ Gaussian Obs per group: min = 20avg = 20.0

max = 20

LR chi2(2) = 83.55Log likelihood = -53.896431 Prob > chi2 = 0.0000

------------------------------------------------------------------------------grade | Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+----------------------------------------------------------------aptitude | .9210743 .0710091 12.97 0.000 .781899 1.06025

age | .2022943 .0662681 3.05 0.002 .0724112 .3321775_cons | 3.021863 1.034865 2.92 0.003 .993565 5.050161

-------------+----------------------------------------------------------------/sigma_u | 1.073727 .4438302 .4775795 2.414027/sigma_e | .5320764 .0498341 .4428439 .639289

rho | .8028508 .1342531 .4616776 .9640621------------------------------------------------------------------------------Likelihood-ratio test of sigma_u=0: chibar2(01)= 83.34 Prob>=chibar2 = 0.000

R-squared

In linear regression, a popular statistics is R 2

,which is the squared multiple correlation

coefficient

R-squared

Remember, variance of a multilevel model

has different components:

σ2Y = σ2

between + σ2within

R-squared: individuallevel

Estimated two models, one with and one

without explanatory variables (A and B ,

respectively).

R 2within = σ2

µ,A + σ2

σ2µ,B + σ2

Predicted random effects

With a fixed effects model, we have the

coefficients on the group dummies which we

can interpret as group-level predictors.

With a fixed effects model, we have the

coefficients on the group dummies which we

can interpret as group-level predictors.In a random effects model, we do not have

these predictions, as we only estimated σ2µ

and β 0.

The predicted group levels can be estimatedusing:

β 0, j = λ j ¯ y j + (1 − λ j )β 0

λ j =σ2µ

σ2µ + σ2

ε/n j ,

whereby ¯ y j is the mean on y of group j .

I R h i d fi d ff i h

In R, you get the estimated fixed effects with:

model.random <- lmer(y ~ x1 + x2 + (1|

fixef(model.random)

and the predicted random effects with:

ranef(model.random)

Random coefficients

In the random effects model, we assume thatgroup intercepts vary according to a normal

distribution.

Random coefficients

distribution.

But what about the coefficients?

Random coefficients

distribution.

But what about the coefficients?

I.e. what about group slopes that vary

following a normal distribution?

Random coefficients

y i = β 0 + X i β + X i γ j [i ] + µ j [i ] + εi

µ j ∼ N (0, σ2µ)

γ j ∼ N (0, σ2γ )

Random coefficients

y i = β 0 + X i β + X i γ j [i ] + µ j [i ] + εi

µ j ∼ N (0, σ2µ)

γ j ∼ N (0, σ2γ )

Note that a model with random coefficients,

but a constant intercept across groups rarely

makes sense, especially because of the often

arbitrary location if x = 0.

Random effects in R

library(arm)

lmer(grade ~ aptitude + age + (aptitude|school))

School example

Linear mixed-effects model fit by REML

Linear mixed effects model fit by REML

Random effects:

Groups Name Variance Std.Dev. Corr

school (Intercept) 1.74e+00 1.32e+00

aptitude 1.47e-10 1.21e-05 0.000

Residual 2.93e-01 5.42e-01

number of obs: 60, groups: school, 3

Fixed effects:

Estimate Std. Error t value(Intercept) 3.0259 1.1359 2.66

aptitude 0.9216 0.0723 12.75

age 0.2020 0.0675 2.99

School example (random)

−2 −1 0 1 2

aptitude

g r a d e

Example

−5 0 5 10

Pooled model

−5 0 5 10

s ope = .

Fixed effects model

−5 0 5 10

s ope = .

Random effects model

−5 0 5 10

s ope = .

Random coefficientsmodel

mean s ope = .

Outline1 Introduction

2 Motivation

3 Fixed effects

4 Random effects

Random coefficients6 Further information

Important other topics

Time-dependence within groups (nextweek)

Predictors on the random coefficients

Bayesian estimation

Bayesian estimationMore complex models dealing with panel

data structures

Bayesian estimationMore complex models dealing with panel

data structures

Extensions towards limited dependent

variables

Further information

A clear, relatively introductory textbook on

multilevel modeling is Snijders & Bosker

(1999), Multilevel analysis. An introduction

to basic and advanced multilevel modeling .

Further information

A clear, relatively introductory textbook on

multilevel modeling is Snijders & Bosker

(1999), Multilevel analysis. An introduction

to basic and advanced multilevel modeling .An excellent, modern book on multilevel

modeling, using primarily R and Bugs, is

Gelman & Hill (2007), Data analysis using regression and multilevel/hierarchical models .

Further information

Their websites are also interesting:

Snijders: http://stat.gamma.rug.nl/snijders/

Gelman: http://www.stat.columbia.edu/ gelman/

Further information

When using Stata, the

Longitudinal/panel-data reference manual is

of very high quality. The relevant chapters

for this lecture are in fact freely available assample chapters (xtreg and xtmixed) at

http://www.stata.com/bookstore/xt.html.

Further information

Two standard textbooks on panel data are

Baltagi (2005), Econometric analysis of

panel data (primarily for small N , large T )

and Hsiao (2003), Analysis of panel data

(primarily for large N , small T ). Both are

very technical in nature. Perhaps an easier

introduction is Wooldridge (2002),

Econometric analysis of cross-section and

panel-data.

aqm 2008 lecture random

Documents

aqm 220 classic.ppt

18.175: lecture 15 .1in random walks -...

probability and random variable lecture(5)

lecture 7 random variables - university of...

probability and random variable lecture (lec10)

probability and random variable lecture 1

18.440: lecture 17 .1in continuous random...

random process lecture 7. wide sense stationary random...

stat 22000 lecture slides random variables

probability and random variable lecture(4)

aqm mehanizmi prezentacija

probability and random variable lecture(lec2)

lecture 8 - random number generation

aqm guideline - textile quality control guideline.pdf ·...

aqm germany - iran

lecture 3: random testing

advanced signal processing lecture 1: random...

lecture 4 linear random coefficients models

lecture 5: random variables - shaobo han · lecture 5:...

ws for aqm