penalized maximum likelihood logistic regression joseph coveney cobridge co., ltd
TRANSCRIPT
![Page 1: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/1.jpg)
Penalized Maximum Likelihood Logistic Regression
Joseph Coveney
Cobridge Co., Ltd.
![Page 2: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/2.jpg)
Topics
• Separation in Logistic Regression
• Approaches to Separation
• Firth’s Bias-reduced GLMs
• firthlogit: syntax and examples
• Caveats and to-do’s
![Page 3: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/3.jpg)
Separation in Logistic Regression
![Page 4: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/4.jpg)
![Page 5: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/5.jpg)
Dataset adapted from D. W. Hosmer and S. Lemeshow, Applied Logistic Regression Second Edition. (New York: John Wiley & Sons, 2000), pp. 138–39.
Complete Separation
![Page 6: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/6.jpg)
Quasi-complete Separation
Dataset adapted from D. W. Hosmer and S. Lemeshow, Applied Logistic Regression Second Edition. (New York: John Wiley & Sons, 2000), pp. 138–39.
![Page 7: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/7.jpg)
![Page 8: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/8.jpg)
Approaches to Separation
• Remove predictors– Pool groups– Remove interaction terms
• Gather more data
• Use alternatives
![Page 9: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/9.jpg)
Exact Logistic Regression
![Page 10: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/10.jpg)
But . . .
Dataset from D. M. Potter. 2005. A permutation test for inference in logistic regression with small- and moderate-sized data sets. Statistics in Medicine 24:693–708.
![Page 11: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/11.jpg)
![Page 12: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/12.jpg)
[19] D. Firth. 1993. Bias reduction in maximum likelihood estimates. Biometrika 80:27–38.
![Page 13: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/13.jpg)
firthlogit
![Page 14: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/14.jpg)
![Page 15: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/15.jpg)
![Page 16: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/16.jpg)
But . . . redux
![Page 17: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/17.jpg)
But . . . redux, continued
![Page 18: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/18.jpg)
0
.2
.4
.6
.8
1A
ctua
l Pro
babi
lity
(LO
WE
SS
Sm
ooth
ed)
0 .2 .4 .6 .8 1Predicted Probability
bandwidth = .8
![Page 19: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/19.jpg)
![Page 20: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/20.jpg)
Profile Likelihood Ratio CIs
![Page 21: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/21.jpg)
![Page 22: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/22.jpg)
Caveats
• Profile Penalized Likelihood CIs
• Small-sample Behavior
![Page 23: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/23.jpg)
G. Heinze and M. Ploner, A SAS macro, S-PLUS library and R package to perform logistic regression without convergence problems. Technical Report 2/2004. Medical University of Vienna. p. 36.
![Page 24: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/24.jpg)
![Page 25: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/25.jpg)
![Page 26: Penalized Maximum Likelihood Logistic Regression Joseph Coveney Cobridge Co., Ltd](https://reader035.vdocuments.site/reader035/viewer/2022062313/56649cd85503460f949a040b/html5/thumbnails/26.jpg)
To-do’s
• Profile Penalized Likelihood CIs
• Modify ml d0