evaluating the robustness of learning from implicit feedback filip radlinski thorsten joachims...
TRANSCRIPT
Evaluating the Robustness of Learning from Implicit FeedbackFilip Radlinski
Thorsten Joachims
Presentation by Dinesh Bhirud
Introduction
• The paper evaluates the robustness of learning to rank documents based on Implicit feedback.
• What is implicit feedback?– Relevance feedback obtained from search engine
log files– Easier to collect large amount of such training data
as against explicitly collecting relevance feedback.
Osmot
• Osmot – Search engine developed at Cornell University based on Implicit Feedback
• Name Osmot comes from the word “osmosis” – learning from the users by osmosis
• Query Chains – Sequence of reformulated queries.– Osmot learns ranked retrieval function by
observing query chains and monitoring user clicks.
High Level Block Diagram
Data generation
User behavior
simulation (based on original
ranking
fucntion)
Preference generation
SVM Learning
User behavior
simulatoin (based on learn
ed ranki
ng function)
Data Generation
• Set of W words are chosen, word frequencies obeying a Ziph’s law
• T topics are picked by picking N words/topic uniformly from W.
• Each document d is generated as– Pick kd binomially from [0,T]– Repeat kd times
• Pick topic t• Pick L/kd words from topic t.
Relevance
• 3 kinds of relevance– Relevance with respect to topic
• Can be measured/known because document collection and topics are synthetic
• Used for evaluating the ranking function.
– Relevance with respect to query• Actual relevance score of a document with respect to a query• Used to rank documents
– Observed relevance• Relevance of a document as judged by the user seeing only the
abstract.• Used to simulate user behavior.
User behavior parameters
• Noise – Accuracy of user’s relevance estimate– Affects observed relevance. (obsRel)– obsRel is drawn from an incomplete Beta distribution
where α gives noise level and β is selected so that mode is at rel(d,q)
• Threshold – User selectivity over results (rT)• Patience – Number of results user looks at before
giving up (rP)• Reformulation – How likely is the user to
reformulate query(Preform)
User Behavior ModelWhile question T is unanswered
1.1 Generate query q (Let d1,d2..,dn be results for q)1.2 Start with document 1 ie i = 11.3 while patience (Rp) > 0
1.3.1 if obsRel(di,q) > rT1.3.1.1 if obsRel(di+1, q) > obsRel(di,q) + c then continue
looking further in the list 1.3.1.2 else
di is a good document, click on it.If rel(di,T) is 1, user is DONEDecrease patience Rp.
1.3.2 elseDecrease patience RpRp = Rp - (rT – obsRel(di,q))
1.3. 3 Set i = i + 11.4 With probability (1 – Preform) , user gives up.
User Preference Model
• Based on the clickthrough log files, users’ preferences for documents given query q can be found.
• Clickthrough logs generated by simulating users.
• From preference, features values are calculated.
Feedback Strategies Single Query Strategy
• Click >q Skip Above– For query q, if document di is clicked, di is
preferred over all dj, j < i.
• Click 1st >q No-Click 2nd
– For query q, if document 1 is clicked, it is preferred over the 2nd document in the list.
Feedback Strategies 2-Query Strategy 1
• This strategy uses 2 queries in a query chain, but document rankings only for the later query.
• Given queries q' and q in a query chain• Click >q' Skip Above– For query q', if document di is clicked in query q, di
is preferred over all dj, j < i • Click 1st > q' No-Click 2nd
– For query q', if document 1 is clicked, it is preferred over the 2nd document in the list for q
Feedback Strategies 2-Query Strategy 2
• This strategy uses 2 queries in a query chain, and document rankings for both used.
• Given queries q' and q in a query chain• Click >q' Skip Earlier Query– For query q', if document di is clicked in query q, di is
preferred over seen documents in query previous query.
• Click > q' Top two earlier Query– If no document clicked for query q', then di preferred
over top two in previous query.
Example Q1 Q2
D1 D4
D2 D5
D3 D6
Preferences• D2 >q1 D1• D4 >q2 D5• D4 >q1 D5• D4 >q1 D1• D4 >q1 D3
Features
• Document di would be mapped to feature vector with respect to query q.
• 2 types of features defined– Rank Features – Term/Document Features
q) , (diØ
q) , (diØ q) , Ø(di
term
rank
q) , Ø(di
q) , (diØrank
q) , (diØterm
Rank Features
• Rank features allow representation of ranking given by the existing static retrieval function.
• Used a simple TFIDF weighted cosine similarity metric (rel0)
• 28 rank features used for ranks 1,2,..,10,15,20,…100.
• Set to 1 if clicked document is at or above specified rank.
Term Features
• Allows representation of fine grained relationship between query terms and documents.
• If for query q, document d is clicked, then for each word ,
• Forms a sparse feature vector, as only very few words are included in query.
qw 1 w), (dØterm
Learning
• Retrieval Function rel(di, q) defined as
where is the weight vector. • Intuitively, weight vector assigns weight to each feature
identified.• Task of learning a ranking function is reduced to the task
of learning an optimal weight vector.
q) , Ø(di q) rel(di, w
w
How does affect ranking?
• Points are ordered by their projections onto
• For the ordering will be 1,2,3,4.
• For the ordering will be 2,3,1,4.
• Weight vector needs to be learnt that will minimize number of discordant rankings.
w
w
1w
2w
w
Learning Problem
Learning problem can be formalized as follows• Find weight vector such that maximum of following inequalities fulfilled. such that then• Without using slack variables, this is NP-hard
problem.
w
1),( rdd ji ),(),( 11 qdrqdr ji q) , Ø(d q) , Ø(d ji ww
SVM Learning
• Equivalent optimization problem would be
Minimize
Subject to rearranging which we get constraint
and and
ij
Cww ij2
1
ij - 1 q) , Ø(dw q) , Ø(d:),,( ji
wjiq
ij - 1 q)) , Ø(d - q) , Ø(d(:),,( ji wjiq
0: ijij 01.0:]28,1[ iwi
Re-ranking using the learnt model
• SVM-Light package is used.• Model provides values for all support
vectors.• User behavior is again simulated, this time using
the learnt ranking function.• How does reranking work?– First, a ranked list of documents is obtained using the
original ranking function.– This list is re-ordered, using the weights of each feature
obtained from the learnt model.
y
Experiments
• Experiments done to study the behavior of the search engine by varying parameters like– Noise in users’ relevance judgement– Ambiguity of words in topics and queries– Threshold value which user considers good
document– Users’ trust in ranking– Users’ probability of reformulation of query.
Results - Noise
0 1 2 3 4 5 670
75
80
85
90
95
100
Ranking Function performance at various noise levels
Noise Low Noise Medium Noise High Maximum Noise
Learning Iterations
Expe
cted
Rel
evan
ce
Noise – My experiment
• Did implementation for extracting preferences and encoding them in features.
0.0000 1.0000 2.0000 3.0000 4.0000 5.0000 6.000070
72
74
76
78
80
82
84
86
Ranking function performance at various noise levels (My implementation)
Noise LowNoise MediumNoise High
Learning Iterations
Expe
cted
Rel
evan
ce
Topic and Word Ambiguity
0 1 2 3 4 5 670
75
80
85
90
95
100
Ranking function performance at different levels of word ambiguity
No ambiguous wordsWords somewhat ambiguouosWords more ambiguous
Learning Iterations
Expe
cted
Rel
evan
ce
Probability of user reformulating query
0 1 2 3 4 5 670
75
80
85
90
95
100
25% give up probability50% Give up probabilty75% Give up probability100% Give up probability
Learning Iterations
Expe
cted
Rel
evan
ce
Thank You