groups change too: analyzing repeated measures on individuals embedded within dynamic groups
DESCRIPTION
Groups Change Too: Analyzing Repeated Measures on Individuals Embedded Within Dynamic Groups. Daniel J. Bauer. Outline of Talk. Goal: - PowerPoint PPT PresentationTRANSCRIPT
Groups Change Too: Analyzing Repeated Measures on Individuals Embedded Within
Dynamic Groups
Daniel J. Bauer
Goal: To offer a more realistic model for repeated measures
data when individuals are clustered within groups that undergo structural or functional change over time
Roadmap: Causes and consequences of clustered data Multilevel modeling Analyzing change over time Stable versus dynamic groups Two applications of dynamic group models
Outline of Talk
Clustering is a Natural Feature of DataHumans exist within a social ecology including both natural and constructed groups (e.g., family and school)
Observations on individuals from the same group tend to be correlated Peer groups subject to selection effects (homophily) and
socialization effects (group norms) Schools include students drawn from similar
sociodemographic backgrounds, and students are exposed to common teachers and curricula
Family members have common genes, environmental exposures, and social influences
Yet most statistical models assume independence of observations (more specifically, independent residuals)
Clustering Usually Implies Correlation
Consequences of Ignoring Dependence
What happens if we erroneously analyze the data as if they were independent? Standard errors, test statistics, degrees of freedom, p-
values, and confidence intervals are all incorrect Tests tend to be too liberal, inflating Type I errors
Most importantly, we neglect important processes in the data How similar are individuals within groups? How strong are group effects on individuals? What predictors account for within- versus between-group
differences?
Appropriately Analyzing Clustered Data
There are several possible ways to analyze cluster-correlated data Fixed-effects approaches Generalized estimating equations Multilevel models with random effects
Multilevel models offer unique insights into both individual and group-level processes
A Classic Example
Science achievement scores for student from different schools:
A Simple Multilevel Model
A basic two-level model for clustered data:
0ij j ijy v r
2~ 0,j vv N
2~ 0,ij rr N
2
2 2ICC v
v r
Overall average (fixed effect)
Group-level influences (random effect)
Individual-level influences that are independent of group (random effect)
Variance component associated with each random effect
Correlation between individuals’ scores
The Variance Components
Here we see how each component of variability maps onto our plot of the data
b0vj
rij
Extending the Model
Normally, our next step would be to incorporate predictors at the individual and group level to explain each source of variability in the data
Suppose, however, we didn’t just measure our outcome once, but multiply over time, with the goal of capturing individual trajectories of change
Modeling Individual Trajectories
0 1 2 3
Person i=1 in Group j=1
Time
y
Modeling Individual Trajectories
Person 1, Group 1
Person 2, Group 1
Person 4, Group 2
Time
y
0 1 2
Person 3, Group 2
Mean
3
0 1 0 1tij tij ij ij ti j tiy Time u u Time v r
MeanTrajector
y
Individual Difference
s
Time-Specific Residual
Group Effect
Taking a Closer Look
This is a typical three-level model for capturing individual change when individuals are clustered within groups
Note that the group effect, vj, is constant over time Is this consistent with theory?
0 1 0 1tij tij ij ij ti j tiy Time u u Time v r
MeanTrajector
y
Individual Difference
s
Time-Specific Residual
Group Effect
Chronosystem
Dynamic Groups
Just an individuals change, so too does the social ecology
Dynamic Groups
We refer to dynamic groups as those that undergo structural and/or functional change over time yet maintain their core integrity as units
Examples: Rockbridge and Hickman High Schools both experience
turnover in students, teachers, administrators, and curricula, yet continue to be characterized by distinctive school cultures
The Jones Family undergoes structural changes as a consequence of divorce, remarriage, and child birth, and undergoes functional changes as a consequence of parental addiction and unemployment, yet the Jones Family remains distinct from the Smith Family
Rewriting the Model
With dynamic groups, we would expect group effects to be correlated over time, but not necessarily constant
A more realistic model might thus be
The group effect is now time-varying Key is then to discern its temporal structure
0 1 0 1tij tij ij ij ti tj tiy Time u u Time v r
MeanTrajector
y
Individual Difference
s
Time-Specific Residual
Group Effect
17
Temporal Structure of Group Effects
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
a
a a
a a a
a a a a
?
Temporal Structure of Group Effects
Traditional “stable groups” model
1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
2
1 1 1 1
1 1 1 1
1 1 1 1
1 1 1 1
v
Correlated 1.0 over time
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
Temporal Structure of Group Effects
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
Toeplitz model
1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
2
1
1
1
1
v
a b c
a a b
b a a
c b a
Banded Covariance
Matrix
Temporal Structure of Group Effects
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
Stabilization model
1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
2
1
1
1
1
v
a b b
a a b
b a a
b b a
Stabilizing Banded
Covariance Matrix
Temporal Structure of Group Effects
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
Compound symmetric model
1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
2
1
1
1
1
v
a a a
a a a
a a a
a a a
Equal covariances
Temporal Structure of Group Effects
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
AR(1) model1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
2 3
22
2
3 2
1
1
1
1
v
Exponential Decay
ARMA(1,1) model
1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
2
2
2
1
1
1
1
v
Temporal Structure of Group Effects
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
Rapid Decay
Temporal Structure of Group Effects
Unstructured model
1
2
3
4
j
j
j
j
v
v
v
v
1 2 3 4j j j jv v v v
21
22
23
24
v
v
v
v
a b c
a d e
b d f
c e f
Say we have 4 time points How should we structure the covariance matrix of
the group effects over time?
No Structure Imposed
Why It Matters
Specifying a poor temporal structure for the group effects risks Incorrect tests of regression coefficients for predictors Biased estimates of variance components at each level of
the model Occluding important findings regarding the nature and
stability of group effects over time
Goal is thus to identify a theoretically plausible structure that fits the data well Some structures are nested and can be compared using LRT Others are non-nested and can be compared by BIC, AIC,
etc.
26
Example: Attitudes About Science Data drawn from the Longitudinal Study of American
Youth (LSAY) Cohort 1 (N=2091): 1987, 1988, 1989 Cohort 2 (N=1407): 1990, 1991, 1992
51 schools, 3498 students, 7756 observations from grades 10-12
Outcome is an IRT developmental scale score of science ability:
Goals for analysis
Evaluate the relationship between religious attitudes towards science and science achievement “Science undermines morality” “We need less science, more faith” “The theory of evolution is true” (R) “The bible is God’s word”
Separate within-school and between-school effects, while controlling for SES
Obtain accurate tests of these effects by appropriately accounting for school effects
Determine the temporal structure of school effects
2 3
4
0
5 6 7
1
0 1
tij tij ij ij
ij ij j j
ij ij tij tj ti
ti
j
jgrade
studentSES studentatt schoolSES schoola
science grade
tt
cohort cohort
u u v rgrade
Fitted Model
4 5
1
6 7
0
0 1
2 3
ij i
tij tij i t
j j j
ij ij tij tj ti
iji
j
j j
studentSES studentatt schoolSES schoola
s
t
cience grade cohort c
t
u u
ohor
grade
gt
r
rad
v
e
4 5
1
6 7
0 2 3
0 1
ij i
tij ij ij tij
ij ij tij tj
j
j
j
j
j
ti
ti
studentSES studentatt schoolSES schoola
s
t
grade cohort cohort grade
u u grade v
ci ce
r
t
en
Captures average trajectory for each cohort
Captures within- and between- school effects of SES and attitudes
0 1 2 3
4 5 6 7
0 1
tij ij ij tij
ij
ij ij tij t
ti
j tij
j
j
ij j
grade cohort cohort grade
studentSES studentatt schoolSES schoolatt
u u grade v r
science
Captures individual differences in change over time, time-varying school effects, and residuals from the individual trajectories
Structure Parameters
AIC BIC
Intercept 1 52055.0 52064.6
Intercept + Slope
3 51981.6 51995.1
Selecting a Temporal Structure
Structure Parameters
AIC BIC
Intercept 1 52055.0 52064.6
Intercept + Slope
3 51981.6 51995.1
Toeplitz 6 51903.1 51922.5
Stabilizing Lag 4
5 51901.8 51919.2
Stabilizing Lag 3
4 51920.9 51936.4
Stabilizing Lag 2
3 51919.0 51932.5
CS 2 51929.2 51940.7
AR(1) 2 51909.1 51920.7
ARMA(1,1) 3 51911.1 51924.6
Structure Parameters
AIC BIC
Intercept 1 52055.0 52064.6
Intercept + Slope
3 51981.6 51995.1
Toeplitz 6 51903.1 51922.5
Stabilizing Lag 4
5 51901.8 51919.2
Stabilizing Lag 3
4 51920.9 51936.4
Stabilizing Lag 2
3 51919.0 51932.5
CS 2 51929.2 51940.7
AR(1) 2 51909.1 51920.7
ARMA(1,1) 3 51911.1 51924.6
Traditional models
Dynamic group models
Fixed Effects
Stable Group Model Dynamic Group Model
Estimate 95% CI Estimate 95% CI
Intercept 60.51* (59.61,61.42)
60.54* (59.55,61.54)
Grade 2.58* (2.43,2.73) 2.49* (2.17,2.81)
Cohort 1.44* (.74,2.14) 1.32* (.37,2.27)
Grade*Cohort
-.78* (-1.04,-.53) -.61* (-1.11,-.11)
Student Attitudes
-2.67* (-3.37,-1.98)
-2.68* (-3.37,-1.98)
School Attitudes
-8.20* (-16.01,-.38)
-8.83* (-16.47,-.29)
Student SES .13* (.10,.15) .13* (.10,.15)
School SES .12 (-.09,.32) .12 (-.09,.34)* p<.05
Dynamic group model shows diminishing correlation of school effect over time
** Superior model fit
School Effects Over Time
1.00
1.00 1.00
1.00 1.00 1.00
1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00
Traditional model with random intercept for school assumes constant school effect over time
87 88 89 90 91 92
8
7
8
8
8
9
9
0
9
1
9
2
1.00
.90 1.00
.82 .90 1.00
.76 .82 .90 1.00
.62 .76 .82 .90 1.00
.62 .62 .76 .82 .90 1.00
87 88 89 90 91 92
8
7
8
8
8
9
9
0
9
1
9
2
Summary
Though the regression coefficient estimates are similar, the dynamic groups model fits the data better and likely provides more accurate tests of these coefficients Suggests both within and between-school effects of
fundamentalist religious attitudes on science achievement
The dynamic groups model also provides insights into the stability and change of school effects over time School effects highly stable from one year to the next But the correlation decays to .62 over a period of 4 years,
indicating some drift in nature of school effects over time
Example: Family Effects on Psychopathology
Data drawn from the Michigan Longitudinal Study (PI: Zucker) 280 families, 588 children, 2468 repeated measures Repeated measures included from age 11-17 and span 12
calendar years
Outcomes are IRT scores of self-reported externalizing and depression
Primary goal is to examine temporal stability of family effects on psychopathology
Ancillary goals are to estimate trajectories of externalizing and depression for boys and girls, and to evaluate added risk due to parental impairment (alcoholism, depression, ASP)
Fitted Model
0 12
2
23 4 5
6 7 8
21 20
tij
ij tij ij tij ij
j j j
ij i
tij tij
ij tij j tij tj tij
y age age
male age male age male
pAlc pDep pASP
u age u agu e v r
0 1
6 7
22
23 4
8
20 1 2
5
tij
ij
j j j
ij ij tij i
tij i
j tij tj ti
tij tij
j tij
j
ij
y age age
male age male age mal
pAlc pDep pASP
u u age u ag
e
e v r
Captures differences in average trajectories of girls and boys
6 7
20 1 2
23 4
20 1 2
8
5
tij tij
ij tij ij tij ij
ij ij tij ij tij tj tij
tij
j j j
age age
male age male age male
u u age u age v
y
pAlc pDep pASP
r
Captures effects of parental impairment
Captures individual differences in change over time, time-varying family effects, and residuals from the individual trajectories
20 1 2
23 4 5
6 7
2
8
0 1 2
tij tij
i
ij ij tij ij tij tj t
j tij i
t
j
ij
i
tij ij
j
j
j j
age age
male age male age mal
y
u u age u age v
e
pAlc pDep pASP
r
Selecting a Temporal Structure
For both outcomes, the AR(1) dynamic group model fits best
Externalizing Depression
Structure Parameters
AIC BIC AIC BIC
Intercept 1 4769.5 4798.6 6251.9 6280.9
Intercept + Slope
3 4768.9 4805.2 6252.7 6289.0
CS 2 4770.0 4802.7 6252.3 6284.9
AR(1) 2 4754.5 4787.2 6245.9 6278.6
Externalizing Depression
Structure Parameters
AIC BIC AIC BIC
Intercept 1 4769.5 4798.6 6251.9 6280.9
Intercept + Slope
3 4768.9 4805.2 6252.7 6289.0
CS 2 4770.0 4802.7 6252.3 6284.9
AR(1) 2 4754.5 4787.2 6245.9 6278.6
Fixed Effects
Externalizing Internalizing
Estimate 95% CI Estimate 95% CI
Intercept-.133 (-.272,.006) -1.307*
(-1.495,-1.120)
Age .055* (.027,.084) .070* (.031,.110)Age2
-.034*(-.047,-.021
).010 (-.000,.020)
Male .217* (.096,.338) -.273* (-.410,-.135)Age × Male
-.070*(-.104,-.036
)-.078* (-.124,-.031)
Age2 × Male .028* (.012,.043)Parent Alc .415* (.291,.540) .207* (.024,.390)Parent Dep .098 (-.027,.223) .179 (-.004,.363)Parent ASP .207* (.049,.364) .257* (.028,.486)
* p<.05
Family Effects Over Time
Summary
Gender differences consistent with other literature
Parental history of alcoholism elevates risk of both depression and externalizing, and this is compounded by history of ASP History of parental depression does not have a significant
effect
Family effects are highly fluid A family that is troubled in one year is likely to continue to
function poorly in the next year or two, but may right itself over the longer term
Conversely, a family functioning well at one point in time is not immune from later difficulties
Family effects less stable for externalizing than depression
Conclusions
Standard multilevel models fail to account for the fact that groups undergo change over time
The effect of the group on its members is unlikely to be constant
We propose the use of dynamic group models to obtain new insights on the temporal structure of group effects At a time lag of five years, school effects on science
achievement were correlated .62 In contrast, family effects were correlated .53 for
depression and only .25 for externalizing behavior This difference in stability is perhaps not surprising.
Schools are large institutions with a great deal of inertia, whereas families are small groups that are potentially more vulnerable to stochastic events
Acknowledgements and Disclaimer
The project described was supported by supported by National Institutes of Health grants R01 DA 025198 (PI: Antonio Morgan-Lopez), R37 AA 07065 (PI: Robert Zucker), and R01 DA 015398 (PIs: Andrea Hussong and Patrick Curran). The content is solely the responsibility of the author and does not represent the official views of the National Institute on Drug Abuse, National Institute on Alcohol Abuse and Alcoholism, or the National Institutes of Health. Nisha Gottfredson
Danielle Dean
Robert Zucker
Antonio Morgan-Lopez
Andrea Hussong