murpy's machine learning 9. generalize linear model
TRANSCRIPT
![Page 1: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/1.jpg)
Machine Learning
Logistic RegressionGeneralized Linear Model
![Page 2: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/2.jpg)
8.1 Introduction, overview8.2 Model specification8.3 Model fitting
8.3.1 MLE8.3.2 Steepest descent8.3.3 Newton's method8.3.6 l2 regularization8.3.7 Multi-class logistic regression
8.4 Bayesian logistic regression8.4.1 Laplace approximation8.4.2 Derivation of the BIC(Bayesian Information Criterion)8.4.3 Gaussian approximation for logistic regression8.4.4 Approximating the posterior predictive
8.5 Online learning and stochastic optimization8.5.3 The LMS algorithm8.5.4 The perceptron algorithm8.5.5 A Bayesian view
8.6 Generative vs discriminative classifiers8.6.1 Pros and cons of each approach
![Page 3: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/3.jpg)
![Page 4: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/4.jpg)
![Page 5: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/5.jpg)
다른 최적화 기법 ( 경사 강하 , 뉴턴 ,…) 을 사용해서 최적화한다
![Page 6: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/6.jpg)
예측치 - 실제치
![Page 7: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/7.jpg)
![Page 8: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/8.jpg)
에타에 대해서 미분
![Page 9: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/9.jpg)
![Page 10: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/10.jpg)
![Page 11: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/11.jpg)
Recall Ridge regresion
![Page 12: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/12.jpg)
![Page 13: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/13.jpg)
![Page 14: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/14.jpg)
![Page 15: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/15.jpg)
![Page 16: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/16.jpg)
정규분포라서 그냥 평균이고 평균은 MAP추정치 였으므로 l2 reg 와 같아짐
![Page 17: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/17.jpg)
![Page 18: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/18.jpg)
![Page 19: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/19.jpg)
베이지안 linear regression
![Page 20: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/20.jpg)
![Page 21: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/21.jpg)
9.1 Introduction9.2 The exponential family
9.2.1 Definition9.2.2.1 Bernoulli9.2.2.2 Multinoulli9.2.2.3 Univariate Gaussian
9.2.3 Log partition function9.2.3.1 Example: the Bernoulli distribution
9.3 Generalized linear models (GLMs) 9.3.1 Basics 9.3.2 ML and MAP estimation 9.3.3 Bayesian inference
![Page 22: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/22.jpg)
![Page 23: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/23.jpg)
Normalize 하는 term
어떤 분포를 지수형태의 같은 모양으로 표현할 수 있으면 지수족이라고 한다 .
족
![Page 24: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/24.jpg)
![Page 25: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/25.jpg)
![Page 26: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/26.jpg)
![Page 27: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/27.jpg)
![Page 28: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/28.jpg)
![Page 29: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/29.jpg)
Likelihood 의 충분통계량
![Page 30: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/30.jpg)
전체 앞면수 / 전체 시도수
![Page 31: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/31.jpg)
![Page 32: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/32.jpg)
![Page 33: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/33.jpg)
![Page 34: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/34.jpg)
![Page 35: Murpy's Machine Learning 9. Generalize Linear Model](https://reader030.vdocuments.site/reader030/viewer/2022013011/5588f6d5d8b42a9e4e8b45a0/html5/thumbnails/35.jpg)
logistic regression 의 경우 μ = 1/(1+exp(-w'x)) 이므로 S 는 섹션 8.3.1의 결과와 같아진다 .
Logistic R 의 gradi-ent
부호가 바뀐건 위의 결과는 NLL 에 대해서 한거라