recsys 2015: making meaningful restaurant recommendations at opentable
TRANSCRIPT
![Page 1: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/1.jpg)
Making Meaningful Restaurant Recommendations @OpenTable
Sudeep Das, PhD Data Scientist OpenTable @datamusing
![Page 2: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/2.jpg)
CONFIDENTIAL 2
![Page 3: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/3.jpg)
• Over 32,000 restaurants worldwide
• more than 885 million diners seated since 1998, representing more than $30 billion spent at partner restaurants
• Over 17 million diners seated every month
• OpenTable has seated over 254 million diners via a mobile device. Almost 50% of our reservations are made via a mobile device
• OpenTable currently has presence in US, Canada, Mexico, UK, Germany and Japan
• OpenTable has nearly 600 partners including Bing, Facebook, Google, TripAdvisor, Urbanspoon, Yahoo and Zagat.
3
OpenTable the world’s leading provider of online restaurant reservations
![Page 4: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/4.jpg)
At OpenTable we aim to power the best dining
experiences!
![Page 5: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/5.jpg)
Ingredients of a magical experience
Understanding the diner Understanding the restaurant
Building up a profile of you as a diner from explicit and implicit signals - information you have provided, reviews you have written, places you have dined at etc.
What type of restaurant is it? What dishes are they known for? Is it good for a date night/ family friendly/ has amazing views etc. What’s trending?
Connecting the dots
![Page 6: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/6.jpg)
we have a wealth of data
32 million reviews
diner requests and
notes
menus
external
ratings, searches and transactions
images
![Page 7: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/7.jpg)
Making meaningful recommendations
![Page 8: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/8.jpg)
diner-restaurant Interactions
restaurant metadata
The basic ingredients
user metadata
ratings|searches|reviews …
cuisine|price range|hours|topics …
user profile
![Page 9: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/9.jpg)
There are various approaches to making meaningful recommendations
Nearest neighbor approaches in user-user or item-item space
Collaborative Filtering based on explicit/implicit interactions
Content-based approach leveraging restaurant metadata
Factorization machines that include interactions, metadata, as well as context.
![Page 10: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/10.jpg)
10
Recommendations: Restaurant Similarity
![Page 11: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/11.jpg)
![Page 12: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/12.jpg)
![Page 13: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/13.jpg)
Matrix Factorization: Implicit preferences
Restaurant_1 Restaurant_2 … Restaurant_M
Diner_1 50 ? … 100
Diner_2 ? 1 … ?
… … … …
Diner_N 3 30 … 1
Implicit Preferences (Hu, Koren, Volinsky 2008)
Confidence Matrix
Binary Preference
Matrix
![Page 14: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/14.jpg)
14
Ensemble parameter is a function of the user support
Purely Similarity
Purely Model based
Weighted mean inverse rank
a = ↵ 1r1
+ (1� ↵) 1r2
![Page 15: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/15.jpg)
15
![Page 16: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/16.jpg)
Mining the wealth of
textual data for cold start
and beyond …
![Page 17: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/17.jpg)
Content Based Approach
• Comes in very handy for cold start where users have very few interactions Very useful for cold start where users have very few interactions.
Given a few interactions we can find similar restaurants.
Bayesian information retrieval approach.
Content based
approach
![Page 18: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/18.jpg)
18
Our reviews are rich and verified, and come in all shapes and sizes
Superb!
This really is a hidden gem and I'm not sure I want to share but I will. :) The owner, Claude, has been here for 47 years and is all about quality, taste, and not overcharging for what he loves. My husband and I don't often get into the city at night, but when we do this is THE place. The Grand Marnier Souffle' is the best I've had in my life - and I have a few years on the life meter. The custard is not over the top and the texture of the entire dessert is superb. This is the only family style French restaurant I'm aware of in SF. It also doesn't charge you an arm and a leg for their excellent quality and that also goes for the wine list. Soup, salad, choice of main (try the lamb shank) and choice of dessert - for around $42 w/o drinks.
Many restaurants have thousands of reviews.
![Page 19: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/19.jpg)
Word2Vec: Word Embeddings
[1] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. In Proceedings of Workshop at ICLR, 2013. [2] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed Representations of Words and Phrases and their Compositionality. In Proceedings of NIPS, 2013. [3] Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic Regularities in Continuous Space Word Representations. In Proceedings of NAACL HLT, 2013.
“We've [been here for afternoon tea multiple times, and each time] we find it very pleasant”
[ 0.00513298, 0.10313627, 0.0773475 , ..., -0.07634512, 0.00877244, 0.04441034]Vec[tea]=
‘teas', ‘empress', ‘scones', ‘iced’, 'fortnum', ‘salon', ‘teapot', ‘teapots', ‘savories', ‘afternoon', ‘earlgrey' ….
model.most_similar(‘tea’ ):
![Page 20: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/20.jpg)
20
bouillabaisse muscles diavalo linguini clams mussels diavlo pescatore risotto linguine pescatora seafood rissoto
diabolo mussles ciopino swordfish mussel fettuccine gumbo brodetto ciopinno capellini cockles langostines cannelloni
rockfish bisques diavolo cockle stew shrimp prawns fettucine cardinale bouillabaise pasta jambalaya chippino
Early explorations with Word2vec: Find synonyms for “cioppino”
![Page 21: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/21.jpg)
21
Early explorations with word2vec: pairings
Halibut: Chardonnay Lamb: ?
![Page 22: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/22.jpg)
22
Early explorations with word2vec: pairings
Halibut: Chardonnay Lamb: Zinfandel
![Page 23: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/23.jpg)
23
Early explorations with word2vec: pairings
Halibut: Chardonnay Lamb: Zinfandel
![Page 24: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/24.jpg)
24
Sushi of Gari, Gari Columbus, NYC
Masaki Sushi Chicago
Sansei Seafood Restaurant & Sushi Bar, Maui
A restaurant like your favorite one but in a different city. Find the “synonyms” of the restaurant in question, then filter by location!
Akiko’s, SF
San Francisco Maui Chicago New York
'
Downtown upscale sushi experience with sushi bar
![Page 25: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/25.jpg)
25
Harris’ Steakhouse in
Downtown area
~v(Harris’) + ~v(jazz)Broadway Jazz Club Steakhouse with live jazz
~v(Harr
is’)+ ~v(p
atio)
~v(Harris’) + ~v(scenic) Celestial Steakhouse Steakhouse with a view
Patio at Las Sendas Steakhouse with amazing patio
Translating restaurants via concepts
![Page 26: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/26.jpg)
Going beyond the metadata
with Topic Modeling
![Page 27: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/27.jpg)
27
We expect diner reviews to be broadly composed of a handful of broad themes
Food & Drinks Ambiance Service Value for
Money Special
occasions
This motivated diving into the reviews with topic modeling
![Page 28: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/28.jpg)
28
We applied non-negative matrix factorization to learn topics …
• stopword removal • vectorization • TFIDF • NNMF
![Page 29: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/29.jpg)
29
Topics fell nicely into categories
DrinksFood Ambiance
![Page 30: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/30.jpg)
30
Topics fell nicely into categories
ServiceValue Occasions
![Page 31: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/31.jpg)
Our topics reveal the unique aspects of each restaurant without having to read the reviews …
Each review for a given restaurant has certain topic distribution
Combining them, we identify the top topics for that restaurant.
0
0.5
1
Topic 01 Topic 02 Topic 03 Topic 04 Topic 05
0
0.5
1
Topic 01 Topic 02 Topic 03 Topic 04 Topic 05
0
0.5
1
Topic 01 Topic 02 Topic 03 Topic 04 Topic 05
review 1
review 2
review N
.
.
.
0
0.5
1
Topic 01 Topic 02 Topic 03 Topic 04 Topic 05Rest
aura
nt
![Page 32: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/32.jpg)
Looking at the topics and the top reviews associated with it , we know Espetus Churrascaria is not just about meat and steak, but has good salad as well! The service is top notch, its kid friendly, and people go for special occasions, …
![Page 33: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/33.jpg)
Content Based Approach
• Comes in very handy for cold start where users have very few interactions Very useful for cold start where users have very few interactions.
Given a few interactions we can find similar restaurants.
Bayesian information retrieval approach.
Content based
approach
+ Topic Weights
![Page 34: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/34.jpg)
Adding value beyond just making the
recommendation
![Page 35: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/35.jpg)
35
We leveraged food and drink related topics to expand our corpus of dishes and drinks
Most dishes are usually 1-grams (“tiramisu”) 2-grams (“pork cutlets”) or 3-grams (“lemon ricotta pancake”)
For each restaurant, we perform an N-gram analysis of the reviews within the scope of food topics and surface candidate dish tags
We were able to generate several thousands of dish tags using this methodology!
![Page 36: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/36.jpg)
EDINBURGH
MANCHESTER
YORK
SHIRE
KENT
LON
DO
N
![Page 37: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/37.jpg)
37
Sentiments - we use ratings as labels for positive and negative sentiments
Ingredients of a stellar experience
![Page 38: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/38.jpg)
38
Sentiments - we use ratings as labels for positive and negative sentiments
Ingredients of a terrible experience
![Page 39: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/39.jpg)
39
The model knows that “to die for”, “crispy”, “moist” are actually indicative of positive sentiment when it comes to food!
• The lobster and avocado eggs Benedict are to die for. • We finished out meal with the their blackberry bread pudding which was so moist and
tasty. • The pork and chive dumplings were perfectly crispy and full of flavor. • I had the Leg of Lamb Tagine and it was "melt in-your-mouth" wonderful. • … we did our best with the scrumptious apple tart and creme brulee. • My husband's lamb porterhouse was a novelty and extremely tender. • We resisted ordering the bacon beignets but gave in and tried them and were glad we
did---Yumm! …
![Page 40: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/40.jpg)
40
![Page 41: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/41.jpg)
41
We also learn restaurant specific attributes from review text
We learn features using one vs. all Logistic Regression with L1 regularization via a mech turk curated labeled set.
For outdoor seating features include obvious ones such as ‘outdoor’, ‘patio’, as well as ‘raining’, ‘sunny’, ‘smoke’, etc. …
![Page 42: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/42.jpg)
42
Dish+Attribute tags and topics can be used to enhance user profiles
![Page 43: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/43.jpg)
• Rendle (2010) www.libfm.org
Including everything + context: Factorization Machines
WORK IN
PROGRESS
![Page 44: Recsys 2015: Making Meaningful Restaurant Recommendations at OpenTable](https://reader031.vdocuments.site/reader031/viewer/2022030310/58f9a986760da3da068b6f88/html5/thumbnails/44.jpg)
CONFIDENTIAL
keep in touch
@datamusing