deep learning for recommender systems · deep learning for recommender systems oliver gindele...
TRANSCRIPT
![Page 1: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/1.jpg)
Deep Learning for Recommender SystemsOliver [email protected]@datatonic.com Big Data Conference Vilnius28.11.2018
join at Slido.com with #bigdata2018
![Page 2: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/2.jpg)
Who is Oliver?+ Head of Machine Learning+ PhD in computational physics
Who is datatonic?We are a strong team of data scientists, machine learning experts, software engineers and mathematicians.
Our mission is to provide tailor-made systems to help your organization get smart actionable insights from large data volumes.
![Page 3: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/3.jpg)
Recommender Systems
![Page 4: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/4.jpg)
Recommender Systems
![Page 5: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/5.jpg)
Recommender Systems
![Page 6: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/6.jpg)
Collaborative Filtering - Introduction
![Page 7: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/7.jpg)
Collaborative Filtering - IntroductionObjective:
+ Netflix Prize (2009)+ Solve via SVD (ALS or SGD)+ Regression problem
![Page 8: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/8.jpg)
Finding Love with Numbers
![Page 9: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/9.jpg)
Online Dating Dataset - LibimSeTi
![Page 10: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/10.jpg)
Online Dating Dataset - LibimSeTi
+ http://www.libimseti.cz/+ 2005+ 17,359,346 ratings+ 135,359 users+ Ratings: 1-10
+ Female (%): 69 + Male (%): 31+ Mean(rating): 5.9+ Std(rating): 3.1
userId profileId rating gender
![Page 11: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/11.jpg)
Tensorflow: High Level APIsDataset:
Estimator:
![Page 12: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/12.jpg)
Tensorflow: High Level APIsDataset:
Estimator: Easy distributed training (Cloud ML Engine,Kubeflow)
Boosted TreesGMMKMeansAR Models
![Page 13: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/13.jpg)
Tensorflow: High Level APIsFeature Columns:
![Page 14: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/14.jpg)
Dataset API (tf.data)
![Page 15: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/15.jpg)
MF Model
![Page 16: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/16.jpg)
Estimator API (tf.estimator)
![Page 17: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/17.jpg)
Going Deeper - Beyond MFNeural Collaborative Filtering (He et al.)
![Page 18: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/18.jpg)
Going Deeper - Beyond MF
User Metadata Item Metadata
userid age gender
1225 ‘30-40’ ‘F’
itemid genre length
44044 ‘Drama’ 129
Add user and item (profile) characteristics
![Page 19: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/19.jpg)
Results - LibimSeTi
MF MLP MF + MLP Research [1]
RMSE 2.137 2.112 2.071 2.077
MAE 1.552 1.541 1.432 1.410
[1] Trust-Based Recommendation: an Empirical Analysis, O’Doherty, Jouili, Van Roy (2008)
Training details:+ 40 epochs+ MLP: 4 layers (256 units pyramid)+ Adam optimiser+ Results calculated on held out test set (5 rating per user)+ No tuning of hyperparameters
![Page 20: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/20.jpg)
How can we do better?
![Page 21: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/21.jpg)
Better/More Data
![Page 22: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/22.jpg)
Single Customer View
Personal Details Demographics Rule-Based Segmentation
Purchase History Online Interactions Support Interactions
Propensity Scores Dynamic Segmentation Churn Prediction
Recommendations Lifetime Value Sentiment Scores
Analytics &Forecasting
PersonalisedAdvertising
Better/More Data
Customer Interactions
Typical information available to Retailers &
CPG companies
Pro-Active Machine Learning
Model Output
$DynamicPricing
Personalised Web Experience
![Page 23: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/23.jpg)
Better Loss Functions
![Page 24: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/24.jpg)
Better Loss functions+ Implicit feedback (Hu, Koren, Volinsky 2008):
+ Logistic Matrix Factorisation (Johnson, Spotify, 2014):
+ Use ranking loss (or pairwise loss functions)
![Page 25: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/25.jpg)
Improved Metrics
![Page 26: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/26.jpg)
Precision@k
Of the top k recommendations, how many were relevant (interacted with)?
Precision@5 = 3/5
![Page 27: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/27.jpg)
(Normalized) Discounted Cumulative Gain
DCG given by formula
1
8
3
4
5
7
2
9
6
1 / log2(1+1)
1 / log2(4+1)
2.246DCG =
1 / log2(3+1)
1 / log2(8+1)
Best =DCG 2.5620.877=
![Page 28: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/28.jpg)
Go even deeper and embed all the things!
![Page 29: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/29.jpg)
Deep Recommender Systems - AdvancesWide & Deep model (Cheng et al., 2016)
Continuous Features Categorical Features
![Page 30: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/30.jpg)
Results - Real Client Data
SVD (MF) MF - Implicit Feedback model
Wide & Deep
P@5 0.33 0.51 0.79
NDCG@5 0.18 0.30 0.37
Training details:+ Feature columns include user demographics to complement the lack of interactions of cold users+ 100 epochs+ Adam optimiser+ Results calculated on held out test set (up to 5 ratings per user)+ Tuning of the dimension size of the embedding vector of the deep part
![Page 31: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/31.jpg)
Deep Recommender Systems - AdvancesDeep Neural Networks for YouTube Recommendations (Covington, Adams, Sargi, 2016)
![Page 32: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/32.jpg)
Deep Recommender Systems - AdvancesLatent Cross: Making Use of Context in Recurrent Recommender Systems (Alex Beutel, Paul Covington, Sagar Jain et al. , 2018)
![Page 33: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/33.jpg)
Takeaways
![Page 34: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/34.jpg)
Takeaways+ Tensorflow can do more than vision or translation+ High level APIs make model building and training painless+ Custom algorithms and specific loss functions are easily implemented+ Deep Recommender systems work well on real data+ Embeddings and hidden layers allow for many ways to improve a recommender
system
![Page 35: Deep Learning for Recommender Systems · Deep Learning for Recommender Systems Oliver Gindele @tinyoli oliver.gindele@datatonic.com Big Data Conference Vilnius 28.11.2018 join at](https://reader035.vdocuments.site/reader035/viewer/2022062317/5f0cf6867e708231d437ff27/html5/thumbnails/35.jpg)
Thank you.Blog.datatonic.com
facebook.com/datatonic
linkedin.com/company/datatonic
twitter.com/teamdatatonic