issues with mixed models. model doesn’t converge… or
TRANSCRIPT
![Page 1: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/1.jpg)
Issues withMixed Models
![Page 2: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/2.jpg)
Model doesn’t converge…
OR
![Page 3: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/3.jpg)
Convergence
![Page 4: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/4.jpg)
Likelihood Landscape
![Page 5: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/5.jpg)
Likelihood Landscape
![Page 6: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/6.jpg)
Maximum Likelihood Estimation
Mea
n
![Page 7: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/7.jpg)
Maximum Likelihood Estimation
Mea
n
![Page 8: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/8.jpg)
Maximum Likelihood Estimation
Mea
n
![Page 9: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/9.jpg)
Maximum Likelihood Estimation
Mea
n
![Page 10: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/10.jpg)
Maximum Likelihood Estimation
Likelihood = the probability of seeing the data we actually collected given a particular model
Maximum Likelihood Estimates = those values that make the observed data most likely to have happened
![Page 11: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/11.jpg)
Sources of Convergence Problems
• You estimate more parameters than data (or, in general, too many parameters
• Severe collinearity (e.g., two predictors are exactly correlated)
• Missing cells in your design
• Predictors of vastly different metrics
![Page 12: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/12.jpg)
Failure to converge
GENDERATTITUDE male
femalepolite 16
0informal 1632
… and then trying to test the ATTITUDE*GENDER interaction
![Page 13: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/13.jpg)
How can this happen?
“Death by Design”
(coined byRoger Mundry)
![Page 14: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/14.jpg)
designanalysis
![Page 15: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/15.jpg)
Solutions to Convergence Problems
• Drop a random slope(not preferred, should be reported)
• Drop subjects/items for which there is not enough data (not preferred, should be reported)
• Rescale variables so that they lie range between 0 and 1; or make them on similar metrics overall
• Center continuous predictors
• Nonlinear transformations of skewed predictors
![Page 16: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/16.jpg)
Solutions to Convergence Problems
• Change order of variable names in model formula
• Have a balanced and complete design
![Page 17: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/17.jpg)
p-values
![Page 18: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/18.jpg)
The p-value conundrum
What are the degrees of freedom?
How to get p-values out ofmixed models is not entirely straightforward…
DouglasBates
![Page 19: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/19.jpg)
“There are a number of ways to compute p-values from LMEMs, none of which is
uncontroversially the best.”
Barr et al. (2013)
![Page 20: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/20.jpg)
Ways to get p-values
• t-test/F-test with normal approximation
• Likelihood Ratio Test
• Boostrapping
• Permutation
• Markov Chain Monte Carlo (MCMC)
![Page 21: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/21.jpg)
Getting p-vals with normal approximation
xmdlcoefs=data.frame(summary(xmdl)@coefs)coefs$p = 2*(1-pnorm(abs(coefs$t.value)))coefs
![Page 22: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/22.jpg)
Function for getting p-vals with normal approximation
create.sig.table = function(x){
coefs=data.frame(summary(x)@coefs)coefs$p = 2*(1-pnorm(abs(coefs$t.value)))coefs$sig = character(nrow(coefs))coefs[which(coefs$p < 0.05),]$sig = "*"coefs[which(coefs$p < 0.01),]$sig = "**"coefs[which(coefs$p < 0.001),]$sig = "***"return(coefs)
}
![Page 23: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/23.jpg)
Likelihood Ratio Test
First model needs to be nested in second
![Page 24: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/24.jpg)
Likelihood Ratio
The likelihood ratio expresses how many times more likely the data are under one model than the other
Mea
n
Mea
n
![Page 25: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/25.jpg)
Likelihood Ratio Test
![Page 26: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/26.jpg)
Likelihood Ratio Test
![Page 27: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/27.jpg)
Important whendoing likelihood ratio tests
lmer(…,REML=FALSE)
http://anythingbutrbitrary.blogspot.com/2012/06/random-regression-coefficients-using.html
![Page 28: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/28.jpg)
Final issue:Random slopes
![Page 29: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/29.jpg)
DANGEROUS!!!
Random intercept onlymodels are known to bevery anti-conservativein many circumstances
(cf. Barr et al., 2013,Schielzeth & Forstmeier, 2008)
![Page 30: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/30.jpg)
Schielzeth & Forstmeier (2008)
Random intercept only
![Page 31: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/31.jpg)
Type I error simulation
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.052LRT slope ML 0.035LRT intercept REML 0.052LRT slope REML 0.035
z-test intercept ML 0.053z-test slope ML 0.039z-test intercept REML 0.054z-test slope REML 0.042
![Page 32: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/32.jpg)
Add to this explicit subject slopesfor A/B
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.24LRT slope ML 0.15LRT intercept REML 0.24LRT slope REML 0.069
z-test intercept ML 0.24z-test slope ML 0.079z-test intercept REML 0.25z-test slope REML 0.091
![Page 33: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/33.jpg)
Add to this explicit subject slopesfor A/B
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.24LRT slope ML 0.15LRT intercept REML 0.24LRT slope REML 0.069
z-test intercept ML 0.24z-test slope ML 0.079z-test intercept REML 0.25z-test slope REML 0.091
![Page 34: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/34.jpg)
Add to this explicit subject slopesfor A/B + take item slopes
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.18LRT slope ML 0.085LRT intercept REML 0.18LRT slope REML 0.052
z-test intercept ML 0.21z-test slope ML 0.064z-test intercept REML 0.23z-test slope REML 0.08
![Page 35: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/35.jpg)
LRT intercept ML 0.18LRT slope ML 0.085LRT intercept REML 0.18LRT slope REML 0.052
z-test intercept ML 0.21z-test slope ML 0.064z-test intercept REML 0.23z-test slope REML 0.08
Add to this explicit subject slopesfor A/B + take item slopes
10 subjects
10 data points each
5 of those in condition A,5 in B
![Page 36: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/36.jpg)
“Keep it maximal”
![Page 37: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/37.jpg)
“Keep it maximal”
random effectsjustified by the design
vs.
random effectsjustified by the data
Barr et al. (2013)
![Page 38: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/38.jpg)
“Keep it maximal”
“for whatever fixed effects are of critical interest, the
corresponding random effects should be in that analysis”
Barr et al. (2013)
![Page 39: Issues with Mixed Models. Model doesn’t converge… OR](https://reader030.vdocuments.site/reader030/viewer/2022032605/56649e855503460f94b873df/html5/thumbnails/39.jpg)
That’s it(for now)