building a recommendation engine - a balancing act
TRANSCRIPT
Building a Recommendation Engine
A Balancing ActElad Rosenheim, Software ArchitectJune 2015
Who am I
• Architect at Dynamic Yield
• Ex-Team Lead - “Predictors”
Current Team Lead - Data Team
Who are We?
Dynamic Yield is a SaaS-based solution
for real-time personalization.
Agenda
• “Previously on ML…”
• A New (Medium) Challenge
• A New (Big) Challenge
• Tooling Up
Previously on ML
• What’s the most effective layout?
- a ton of questions
• From the Stone Age to A/B Tests
- Good: data beats opinion
- Bad: A/B Testing takes time
• Time is Money
• We don’t have time:
- Trending stories, products on sale…
From A/B Testing to Personalization
• “Multi-Arm Bandits” (Google et al)
- Converge towards the leader
- Doesn’t know nor care
about specific users
• ML: We can do better!
- Each of us is a beautiful and unique feature vector
- Find the best variation for each user,
and beat the “on average” leader! (with science OMG)
2.4% 1.7% 0.4%
ML for Online Learning
• Collaborative Filtering?
- Cold Start, Batch process
- Not suitable in this case
• It’s also not a classification problem
• We need online learning over a stream: Contextual Bandits
- Have the model ready at all times!
- Ensure all variations are explored!
Nice, but…
• Limited to a small set of variations (~5-50)
• Model-based, rather than Content-based
- Hard to “cut to the chase”
What to do with a feed of 300 Top Videos?
Shrinking the problem with Heuristics!
• We think we know something about the world,
and want to formalize our intuitions.
• Apply ranking and filter to the top few dozen,
using a formula like Reddit, StumbleUpon, Hacker News…
- Votes, views, time decay, social signals, etc.
• Now, problem is “small” again (yay). Just add ML and shake.
That’s also nice, but…
• What to do with a product catalog of 30k items?
- Classifiers would have a hard (and long) time
• Back to the basics:
- Collaborative Filtering to the rescue?
- Heuristics in a big way
“Collaborative Filtering, What is it Good For?”
• The Classic Case: Netflix
• Explicit vs. Implicit Feedback
- Yehuda Koren Strikes Again
• Heuristics might enter here,
through the back door
• I can haz history data?
This would help warm the model
…but it would still apply only to returning users
So, no more (just) black box?
• I want a (rather) quick bootstrap
• I must target new users
• Let’s think about the contexts in which we recommend
- Homepage
- Product Page
- Cart
The No-Context Context
• What to show in the homepage?
• Let’s apply a ranking formula, this time for e-commerce
- Long-time data, recent trends,
views, carts, promotions,
search, social, …?
• I have to add component <X> to the mix!
- No you don’t. You don’t know all forces at play.
- Better test it.
The Specific Product Context
• Substitute vs. Complementary products
- Not the same for everyone always
• “Similar Items”
- What is similar?
• “Bought Together”
- Hehe, that’s easy right?
Similar Items to <x>
• Keyword Similarity
- Score rare keywords higher!
- We’re in the realm of search engines
- TF/IDF, BM25, Practical Scoring
• Balancing Similarity with Popularity
- Somewhat popular “close” items, very popular “far” items
- At a high-level: a•similarity + b•popularity
- The secret sauce: achieving a stable formula that works.
Bought Together with <x>
• Items commonly in the same cart. Sort and serve!
• Wrong, because I always buy milk.
• We must mitigate the effect of the globally popular
• Again a TF/IDF-type problem
• A very good read:
“People who like this also like…”
Tying it all together
• Which strategy to use, and when?
- CF when you have enough data (on the whole and for the user)
- You can also use heuristics to determine the best strategy,
but don’t go crazy with those
- You can also A/B test, tune, repeat
• ML for strategy selection and item selection?
Can we make the problem “small” enough again?
You’d need to sign the NDA, sorry.
Let’s talk a bit about
Tools
ElasticSearch, so hot right now
• In the center:
- Receive catalog updates, collect all signals
- Pre-calculate and bake rankings
- Intermediate data vs. final index for recommendations
• Should ElasticSearch be the database?
- Yes and no. It’s cool to have an all-in-one document store,
but we can always recreate from “the” big-data store (HBase)
Nice, but… I want to recommend globally!
• For latency, high-availability, security and sanity.
• ES doesn’t do XDCR, and we don’t need the whole thing anyway.
• Stand-alone ES machines in each geo
- Take snapshots in the center & restore in each geo,
via S3 with the AWS Cloud Plugin.
- Fine-grained access control via IAM. Never use “root”-level keys.
• Redis (yay) for pushing notifications and metadata
Center Site
Page Views,Carts,
Search, etc.
Catalog Updates
Worker Process
US East
Recommender
APJ
Recommender
Europe
Recommender
US West
Recommender
Parting words
• ‘ts all about balance
• Formulae won’t make you any less a Jon Snow
• So start small & test
