University of Southern California Department Computer Science
Bayesian Logistic Regression Model (Final Report)Graduate Student Teawon HanProfessor Schweighofer, Nicolas
9/23/2011
• Introduction
Bayesian Logistic Regression Model (Final Report)
1. The purpose of the project - Experiment ?
2. Summary of Bayesian Logistic Regression (BLR) - How do I apply BLR to the BART or ART
3. What is next?
• The purpose of the project
Bayesian Logistic Regression Model (Final Report)
1. Predict accurate status of rehabilitation - Reduce rehabilitation time ( No un-necessary
training )- Rise efficiency in rehabilitation process
2. Data Collection method - use 3 days data in my program (regression)
for test
• The purpose of the project
Bayesian Logistic Regression Model (Final Report)
3. Experiment Environment
`
Success!
• The purpose of the project
Bayesian Logistic Regression Model (Final Report)
4. Given Data type (collected data 150)Error ==0 && Hit Hand ==1
Data (1 day)
Data (2 day)
Pattern analysi
s
Prior Weight value
New Weight value
Pattern analysi
s
NewNew
Weight value
Successcondition
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
1. What is regression? Why do we use regression?2. Example ( Linear Regression )
Regression can help
to represent complete model
by partially
observed data.
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
3. How do I apply BLR to the project - First, we have two classes for classification. ( Success and Fail ) - Expression a. p(C1 | Ф ) = y (Ф) = Ϭ (WT Ф) success
b. p(C2 | Ф ) = 1 - p(C1 | Ф ) fail
where Ф is feature vector ( data ) and w is weight vector.
Error ==0 && Hit Hand ==1
Successcondition
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
3. How do I apply BLR to the project (continue) - Second, to represent Logistic Regression, I
used Ϭ(·). where Ϭ(α) = 1 / 1 + exp (-α) a. range is limited (0 ~ 1) b. TO MAKE EASY, I used simplest formula (next page)
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
3. How do I apply BLR to the project (continue) b. TO MAKE EASY, I used simplest formula
which includes the least number of parameters
(features) Formula : W0 + W1Ф1 +W2Ф2
this should be updated more accurately by
Nuero-Scientific knowledge.
4. The goal in here is ‘Finding accurate W vector’ to predict posterior result. (next page)
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
4. The goal in here is ‘Finding accurate W vector’ to predict posterior result.
- Process of calculation W vector (W can be represented by Gaussian) a. Wmap (mean) SN (covariance)
: Wmap can be calculated by Newton-Raphson rule.
b. Newton-Raphson rule
: Iterative Optimization Scheme to make minimize
the error of weight vector. [link]
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
4. The goal in here is ‘Finding accurate W vector’ to predict posterior result.
- Process of calculation posterior W vector c. Equation of Newton’s method (Wmap )
( Pattern Recognition and machine learning book
p208 )
d. Covariance of W
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
4. The goal in here is ‘Finding accurate W vector’ to predict posterior result.
- Process of calculation W vector e. Finally, we can get distribution of posterior
W
5. To get the posterior probability given data with posterior W
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
5. To get the posterior probability given data with posterior W (derivation)
- you can find “Pattern recognize and machine learning
book” - I also attached from Srihari’s lecture note.
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
6. How do I apply BLR to the project
a. Initial weight vector = [0.001,0.001,0.001] b. Initial covariance vector = [1,0,0 ; 0,1,0; 0,0,1]
Data (1 day)
Data (2 day)
Pattern analysi
s
Prior Weight value
New Weight value
Pattern analysi
s
NewNew
Weight value
new
NewNew
• Summary of Bayesian Logistic Regression (BLR)
Bayesian Logistic Regression Model (Final Report)
7. Resultspredic
t predict
