bayesian nonparametric poisson factorization for ...lcarin/yunchen10.31.2014.pdf · 31/10/2014 ·...
TRANSCRIPT
Bayesian Nonparametric Poisson Factorization forRecommendation Systems
Prem Gopalan, Masashi Sugiyama, Rajesh Ranganath, David M. BleiAISTATS 2014
Presented by Yunchen Pu
October 31st, 2014
1 / 17
Outline
Recommendation Systems
Bayesian Nonparametric Poisson Factorization
Variational Inference algorithm
Experiment results
2 / 17
Recommendation Systems
Goal: Predict ’ratings’ or ’preferences’ of items for users
A general method: Collaborative filtering
3 / 17
Possion Factorization
Each user is represented with a K -dimensional latent vector ofpositive weights: θT
u = {θu1, ..., θuK}T
4 / 17
Possion Factorization
Each item is also represented with a K -dimensional latent vector ofpositive weights: βT
i = {βi1, ..., βiK}T
K Components
5 / 17
Possion Factorization
Ratings come from a distribution involving the inner product: θTu βi
6 / 17
Possion Factorization
The prior for the weights is Gamma:
θuk ∼ Gamma(a, b) (1)βik ∼ Gamma(c, d) (2)
Each observation is modeled by a Poisson distribtuion
yui ∼ Poisson(θTu βi) (3)
7 / 17
Bayesian Nonparametric Possion Factorization
How to determine the dimensionality K of the latent vectors?
Three properties of Bayesian nonparametric:Allow K →∞E(θT
u βi) <∞The item weights βi should be shared among all users.
Stick-breaking construction for the user weights
8 / 17
Bayesian Nonparametric Possion Factorization
The generative process for the Bayesian nonparametric Poissonfactorization model with N users and M items can be formed as:1. For each users u = 1, ...,N :
(a) Draw su ∼ Gamma(α, c)
(b) Draw vuk ∼ Beta(1, α), k = 1, ...,∞
(c) Set θuk = su︸︷︷︸User scaling factor
· vuk
k−1∏i=1
(1− vui)︸ ︷︷ ︸Stick proportions
, k = 1, ...,∞
2. Draw βik ∼ Gamma(a, b), k = 1, ...,∞, i = 1, ...,M.
3. Draw yui ∼ Poisson(∑∞
k=1 θukβik), u = 1, ...,N, i = 1, ...,M.Guarantee a sparse observation matrix:
p(yui = 0) ≥ exp(−aαbc ) (4)
9 / 17
Variational Inference
Variational algorithm to minimizing KL divergence between jointlyposterior and mean-field family:
q(z , β, s, v) =N∏
u=1q(su)
∞∏k=1
N∏u=1
q(vuk)∞∏
k=1
M∏i=1
q(βik)n∏
u=1
M∏i=1
q(zui)
(5)
Auxiliary variable to obtain a conditionally conjugate model:
yui =∞∑
k=1zui ,k , zui ,k ∼ Poisson(θukβik) (6)
To handle the infinite variational factors, the variational families aretruncated on T
q(vuk) = p(vuk), q(βik) = p(βik), for k ≥ T + 1
10 / 17
Variational Inference
Resort a degenerate delta distribution
q(vuk) = δτuk (vuk) (7)q(vuk) ln p(vuk , zuik |su, βik , α) = I(vuk = τuk) ln p(τuk , zui |su, βik , α)
(8)
Variational family:
q(su) = Gamma(su|γu,0, γu,1) (9)
q(vuk) =
{δτuk (vuk), for k ≤ Tp(vuk), for k ≥ T + 1
(10)
q(βik) =
{Gamma(βik |λik,0, λik, 1), for k ≤ Tp(βik), for k ≥ T + 1
(11)
q(zui) = Multinomial(zui |yui , φui) (12)
11 / 17
Experiment Results
Three databases:Movielens1M: 1 million ratings (0 to 5 stars), 6,040 users, 3,980movies.Movielens10M: 10 million ratings (0 to 10 stars), 71,567 users, 10,681movies.Netflix: 100 million ratings (0 to 5 stars), 480,000 users, 17,770 movies.
Metrics:Predictive log-likelihood (on a test set).Mean precision (retrieve the top 100 items): Fraction of recommendeditems that are relevant to the user.Mean recall (retrive the top 100 items): Fraction of relevant items thatare recommended.
12 / 17
Experiment Results
13 / 17
Experiment Results
14 / 17
Supplementary MaterialThe objective function can be formed as:
Eq
[ ∞∑k=1
N∑u=1
ln p(vuk |α) +∞∑
k=1
N∑u=1
M∑i=1
ln p(zuik |θuk , βik)
]
= Eq
[ ∞∑k=1
N∑u=1
(α− 1) ln(1− τuk) +∞∑
k=1
N∑u=1
M∑i=1
(zuik ln τuk
+k−1∑j=1
zuik ln(1− τuj)− βiksuτuk
k−1∏j=1
(1− τuj))
+ const
(13)
For each τuk , the objective function can be formed as:
Eq
(α− 1) ln(1− τuk) +M∑
i=1zuik ln τuk +
M∑i=1
∞∑j=k+1
zuij ln(1− τuk)
−su
M∑i=1
∞∑l=k+1
τul(l−1∏
j=1,j 6=k(1− τuj))βil
(14)
15 / 17
Supplementary Material
∂L∂τuk
=α− 11− τuk
− E [∑M
i=1 zuik ]
τuk+
E [∑M
i=1∑∞
j=k+1 zuij]1− τuk
− Auk
=α− 11− τuk
−∑M
i=1 yuiφui ,kτuk
+
∑Mi=1 yui(1−
∑kj=1 φui ,j)
1− τuk− Auk
= 0
(15)
⇔ Aukτ2uk +
α− 1+M∑
i=1yuiφui ,k +
M∑i=1
yui(1−k∑
j=1φui ,j)
τuk
−M∑
i=1yuiφui ,k = 0
(16)
16 / 17
Supplementary Material
Auk = E [su]∞∑
l=k+1τul
l−1∏j=1,j 6=k
(1− τuj)
× ( M∑i=1
E [βil ]
)
− E [su]
k−1∏j=1
(1− τuj)
( M∑i=1
E [βik ]
)+ E [su]M
ab
∞∏j=1,j 6=k
(1− τuj)
(17)
17 / 17