© ü b factorization machine - github pages · 2019-01-19 · • rendle s. factorization...
Post on 30-May-2020
12 Views
Preview:
TRANSCRIPT
Factorization Machine
2016/12
Outline• FM
• vs
• vs
• vs
• FM FFM FM
FM
•
• sigmoid
•
•
• w_ = 0.5 y =
•
• CTR etc
•
gbdt DNN deep-width model
•
• 0-18 19-25 25-30 30+
• onehot
• eg: =20: [0, 1, 0, 0] [1, 1, 0, 0]
•
•
•
•
• gini
• + = += etc
•
• + +
•
•
• SVM, d=2
•
•
•
• xi xj != 0
• xi xj
• CTR xi xj
•
•
•
• k << n
• W
• 0
•
• Rendle S. Factorization Machines[C]. international conference on data mining, 2010. • Rendle S. Factorization Machines with libFM[J]. ACM Transactions on Intelligent Systems and Technology, 2012, 3(3).
• ( )
• 0
• SGD
• ALS
• MCMC
•
• libFM
• Spark FM DMSPA
• difacto DMLC PS FM
• tensorflow FM https://github.com/kopopt/fast_tffm
Rendle S. Factorization Machines with libFM[J]. ACM Transactions on Intelligent Systems and Technology, 2012, 3(3).
• movielens
FM vs
review CF•
• FM
CF FM• (userID, itemID) label score
• f(userID, itemID) = ID onehot ++ ID one hot
123 45 3
[0,0,…,1,0,…,0] [0,0,…,1,0,…,0]
[0,0,…,1,0,…,1,0,…0]
3
3
CF FM
FM
• FM
• poi
FM vs
FM vs •
•
• FM
•
x1=1
x2=1
100
40 60
40
x1=1 x1=0
x2=1x2=0
x 2 = 1
! FMFM
FM vs
• Hintonbottleneck dark knowledge
•
dark knowledge
embedding•
•
• FM embedding
• word2vecFM
• FM embedding
deep & wide •
•
Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016: 7-10.
deep & wide
Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016: 7-10.
FFM• (Field)
• FM FFM
•
x_i w_i
x_iw_i1
w_i2
w_if
x_jw_j(w_i · w_j)
x_j
w_j1
w_j2
w_jf
w_i,fj w_j,fi
(w_i,fj · w_j,fi)
FFM•
M. Jahrer, A. To ̈scher, J.-Y. Lee, J. Deng, H. Zhang, and J. Spoelstra, “Ensemble of collaborative filtering and feature engineered model for click through rate prediction,” in KDD Cup 2012 Workshop, ACM, 2012.
FFM• O(k·n0^2), n0 0
• k_ffm << k_fm, n0 k_ffm 10
•
• libFFM ffm Spark DMSPA
Juan Y, Zhuang Y, Chin W, et al. Field-aware Factorization Machines for CTR Prediction[C]. conference on recommender systems, 2016: 43-50.
KDD CUP 2012• KDD Cup 2012 CTR
• 234M
• AUC
M. Jahrer, A. To ̈scher, J.-Y. Lee, J. Deng, H. Zhang, and J. Spoelstra, “Ensemble of collaborative filtering and feature engineered model for click through rate prediction,” in KDD Cup 2012 Workshop, ACM, 2012.
• CTR Criteo Avazu Kaggle
Juan Y, Zhuang Y, Chin W, et al. Field-aware Factorization Machines for CTR Prediction[C]. conference on recommender systems, 2016: 43-50.
FM• d-way FM d
• High order FM d 2,3...,d-1
• Rendle S. Factorization Machines[C]. international conference on data mining, 2010. • Blondel M, Fujino A, Ueda N, et al. Higher-Order Factorization Machines[C]. neural information processing systems, 2016: 3351-3359.
FM• FM
• Blondel M, Fujino A, Ueda N, et al. Higher-Order Factorization Machines[C]. neural information processing systems, 2016: 3351-3359.
ANOVA kernel
ANOVA kernel 递推关系
©2016-2017 XX
top related