Download - Techniques For Deep Query Understanding
Techniques forDeep Query
Understanding“Beware of the man who knows the answer before he
understands the question”
Guided By:
Dr. Dhaval Patel,Assistant Professor,
Department Of CSE,IIT Roorkee.
Presented By:
Abhay Prakash, En. No. – 10211002, CSI, V Year, IIT Roorkee.
(Source: Google)
Purpose:
To understand what exactly the user is searching for – his precise intent
To correct mistakes and guide user to formulate a precise intended query
Introduction: Query Understanding
Query Refinement
Why only this phrase in Bold?(Source: Google)Query Suggestion
Emerging Variety of Queries
Natural Language Queries instead of Keyword Represented Queries
“who is the best classical singer in India” instead of “best classical singer India”
Use of NL Queries increasing (Makoto et.al in [1])
Local Search Queries
“Where can I eat cheesecake right now?”
Context Dependent Queries (Interactive Question Answering
(Source: Bing – location set as US)
Background: How results are generated
High Level Architecture of Search Mechanism (Source: Self Made)
INDEX(Knowledge Base)
Document Understanding
(What and how to Index)
Query Understanding
(which index parameters to be used)
User Query
Results Ranking
Entities: Hotel ABC, cheesecakeLocation: Civil LinesQuality: 0.8Time: 8:15 PM
“where can I eat cheesecake right now?”
Data(Text documents,
User Reviews, Blogs, Tweets, Linkedin …)
Review: Hotel ABC, Civil Lines:I ate cheesecake, which was really awesome. (4/5 star)
[Time: 8:15 PM]
Intent: Hotel SearchSearch for: cheesecakeLocation: Civil linesTime: 8:20 PM
Mechanism in Basic Search (Source: Self Made)
Background: QU & Adv. In Search (Weotta in [3])
1. Basic Search
Direct text match based retrieval of documents
Restrict search space using facet values provided by user
Current day example: Online shopping sites
Example of Facets (Source: Flipkart.com)
Background: QU & Adv. In Search (Weotta in [3])
2. Advanced Search
Ranking of result documents based on:
TF-IDF to identify more relevant documents
Website authority and popularity
Keyword weighting
Not Considered:
Context, NLP for semantic understanding
Location of query, time of query
Example: Google as was in its early stage
Background: QU & Adv. In Search (Weotta in [3])
3. Deep Search
What difference does it bring?
Requirements:
Semantic Understanding of Query
Knowledge of Context, previous tasks
User Understanding and Personalization
Architecture: Query Understanding Module
Query
Query Suggestion
Query Correction
Query Expansion
Query Classification
Semantic Tagging
3. Query Intent Detection2. Query Refinement
QUERY UNDERSTASNDING MODULE
ANSWER GENERATION
MODULE
1. Query Suggestion
Result
Components of Query Understanding Module (Source: Self Made)
Architecture: Query Understanding Module
Query
Query Suggestion
Query Correction
Query Expansion
Query Classification
Semantic Tagging
Example of purpose of each Component (Source: Self Made)
michal jrdan
michael jordan
i) michael jordan berkleyii) michael jordan NBA
i) michael jordan berkleyii) michael l. jordan berkley
i) michael jordan berkley: academicii) michael l. Jordan Berkley: academic
i) [michael jordan: PersonName][berkley: Location]: academic
ii) [michael l. jordan: PersonName][berkley: Location]: academic
Reformulates the ill-formed (mistaken) search queries
ex. Macine learning Machine Learning
Refinements:
Spelling error, Two words Merged together, One word separated
Phrase segmentation (machine + learning machine learning)
Acronym Expansion (CSE Computer Science & Engineering)
Refinement may be mutually dependent
“lectures on machne learn”
learn is a correct term, but should have been learning
Hence, different terms need to be addressed simultaneously
Query Correction
Problem Modeled by Jiafeng et.al in [10] as
Original Query () Corrected Query ()
Get y(complete sequence) which has maximum probability of occurrence, given the sequence x.
Simple Technique
Assume terms independent take with max
Prime Disadvantage:
Reality deviates a lot from assumption
Ex. “Lectures on machine learning”
Independent Corrections
Query Correction
What is CRF?
Probabilistic graphical model, models conditional distributionof unobserved state sequences
Trained on given observation sequence
Trained for getting
Why use CRF? Conditioned on?
Sequence of words matters (learning machine?)
conditioned on other s as well, along with
Corrections are mutually dependent (e.g. machine learning)
Disadvantage:
Will require very large amount of data, candidates’ domain open
Query Correction Using Conventional CRF
Conventional CRF
Restricting space of y for the given x
conditioned on operation also
, such that required to get from
is operation like deletion, insertion of characters, etc.
Learning and Prediction
Dataset of
Features
, where the prob. calculated using corpus
Whether --{0|1}
Basic CRF-QR Model
Query Correction Basic CRF-QR Model (Jiafeng et. al in [10])
What is new?
Handles scenario with more than one refinements
Machine learm learn learning
Sequence of (sequence of operation)
i.e. multiple operations on each word
Intermediate results:
Extended CRF-QR
Query Correction Extended CRF-QR Model (Jiafeng et. al in [10])
Query Suggestion
Purpose:
Suggest similar queries
Query auto-completion
Requirements
Context consideration [7]
Identifying Interleaved Tasks [9]
Personalized suggestion [2]
Suggestions on “iit r..”
Query Suggestion Mechanism (Source: [7])
Query – mapped Concept
Concept Suffix tree from log
Suggestion time: Transition on tree with each query’s concept
Suggest top queries of that state
Query Suggestion
Context aware Query Suggestion (Huanhuan et.al in [7])
Concept Suffix Tree
Concept Discovery
Queries clustered using set of clicked URLs
Feature vector
Each identified cluster is taken as a Concept
Concept Suffix Tree
Vertex: state after transition through a sequence of concepts (of queries)
Transition in a session
C2C3C1: transition Beginning C1 C3 C2
Click-Through Bipartite
Query Suggestion
Context aware Query Suggestion (Huanhuan et.al in [7])
Why task identification Important?
Considering Off-Task query in context adversely affect quality of recommendation
30% sessions contained multiple tasks (Zhen et.al in [8])
5% sessions have Interleaved tasks (Zhen et.al in [8])
Identify similar previous queries as On-Task
consider only On-Task queries as context
Query Suggestion
Task aware Query Suggestion (Allan et.al in [9])
Effect of On-Task and Off-Taskqueries
Query Suggestion
Task aware Query Suggestion (Allan et.al in [9])
Measures to evaluate similarity between two queries
Lexical Score: captures similarity at word level directly. Average of:
Jaccard Coefficient between trigrams from the two queries: how many common trigrams?
(1 - Levenshtein Edit Distance), which shows closeness at word level
Semantic Score: maximum of the following two
: cosine similarity of vector of tf-idf score of Wikipedia documents w.r.t the two queries.
: similar to above on Wiktionary entries
Final Similarity() = . Lexical Score + (1-) . Semantic Score
If Similarity(, Reference_q) greater than threshold is On-Task Query
Query Suggestion
Personalization in Query Suggestion (Milad et.al in [2])
On character hit of ‘i’
“Instagram” more popular for female below 25
“Imdb” more popular for male in 25-44.
Candidate queries generated by prior general method
Personalization by re-ranking candidate queries
Features for feedback earlier global rank Original position
Original score
Short History Features 3-Gram similarity with just previous query
Avg. 3-gram similarity with all previous queries in the session
Query Suggestion
Personalization in Query Suggestion (Source: [2])
Long History Features
No. of times candidate query issued in past
Avg. 3-gram similarity with all previous queries in the past
Demographic Features
Candidate query frequency over queries by same age group
Candidate query likelihood -- same age group
Candidate query frequency -- same gender group
Candidate query likelihood -- same gender group
Candidate query frequency -- same region group
Candidate query likelihood -- same region group
Query Expansion
Sending more words (should generate similar result) to tackle term-miss
Ex. “Tutorial lecture on ABC” “Video Lecture on ABC”
Expansion Tasks:
Adding synonyms of words
Morphological words by stemming
Naïve Approach
Exhaustive lookup in thesaurus
Time taking
Still miss terms of similar intent (terms even semantically far)
Query Expansion
Path Constrained Random Walk (Jianfeng et.al in [11])
Exploiting search logs for identifying terms having similar end result
Search log data of <Query, Document> clicks
Graph Representation
Node Q: seed queryNodes Q’: queries in search logNodes D: documentsNodes W: words that occur in queries and documents
Word nodes are the candidate expansion terms
Edges have scoring function
Represents probability of transition from start node to end node
Search Log as Graph
Query Expansion
Path Constrained Random Walk (Jiafeng et.al in [11])
Probability of using w as an expansion word?
Product of probabilities in Paths starting at node Q and ending at w
Top probable words picked, obtained from random walk
Search Log as Graph
Query Classification
Classifying given query in a predefined Intent Class
Ex. michael Jordan berkley: academic
Precise intent by sequence of nodes from root to leaf
More challenging than document classification
Short length
Keyword representation, makes more ambiguous
Ex. query “brazil germany”
Older basic techniques
Considering single query statistical techniques like 2-gram/3-gram inference
Example Taxonomy (Source: [6])
Query Classification
Context aware Query Classification (Huanhuan et.al in [6])
Resolving ambiguity using context
Previous Queries sports, then “Michael Jordan” sports (Basketball Player)
Previous Queries academic, then “Michael Jordan” academic (ML professor)
Use of CRF (because training and prediction on sequence)
Local Features
Query Terms: Each supports a target category
Pseudo Feedback:
with concept , submitted to an external web directory
How many of top M results have concept?
Implicit Feedback:
Instead of Top M results – only the clicked documents taken
Query Classification
Context aware Query Classification (Huanhuan et.al in [6])
Contextual Features
Direct association between adjacent labels
Number of occurrences of adjacent labels
Higher weight higher probability of transit from to
Taxonomy-based association between adjacent labels
Given pair of adjacent labels at level n
n-1 features of taxonomy-based association between considered
e.g. Computer/Software related to Computer/Hardware, matching at (n-1)th level Computer
Semantic Tagging
Identifies the semantic concepts of a word or phrase [michael jordan: PersonName] [berkley: Location]: academic
Useful only if phrases in documents also tagged
Shallow Parsing Methods
Part of Speech Tags: e.g. Clubbing consecutive nouns for Named Entity Recognition
Disadvantage: Sentence Level Long Segments can’t be identified
Semantic Tagging
Hierarchical Parsing Structures
Trained a semi-Markov CRF on segments
Features
Syntactic Features
Parse tree of sentence
Plot
Semantic Tagging
Semantic Dependency Features
leverage the information about dependencies among different segments
Ex. “show me a funny movie starring Johnny and featuring Carribbean Pirates”
‘Featuring’ takes arguments – “funny movie” and “Carribbean Pirates”
long distance semantic dependency between the object “movie” and attribute <Plot>
Conclusion & Future Work
End-to-End Discussion of Query Understanding Module Tasks
Semantic Understanding of queries for intent detection has lot of scope
Use of NL (grammatically correct) queries rising
Understanding at the structure level
User community detection for its application in Query Suggestion
Based on search behavior
Community/Topic specific temporal trending of search query
References
[1] Makoto P. Kato, Takehiro Yamamoto, Hiroaki Ohshima and Katsumi Tanaka, "Cognitive Search Intents Hidden Behind Queries: A User Study on Query Formulations," in WWW Companion, Seoul, Korea, 2014.
[2] Milad Shokouhi, "Learning to Personalize Query Auto-Completion," in SIGIR, Dublin, Ireland, 2013.
[3] Weotta, "Deep Search," 10 6 2014. [Online]. Available: http://streamhacker.com/2014/06/10/deepsearch/. [Accessed 6 8 2014].
[4] W. Bruce Croft, Michael Bendersky, Hang Li and Gu Xu, "Query Understanding and Representation," SIGIR Forum, vol. 44, no. 2, pp. 48-53, 2010.
[5] Jingjing Liu, Panupong Pasupat, Yining Wang, Scott Cyphers and Jim Glass, "Query Understanding Enhanced by Hierarchical Parsing Structures," in ASRU, 2013.
[6] Huanhuan Cao, Derek Hao Hu, Dou Shen and Daxin Jiang, "Context-Aware Query Classification," in SIGIR, Boston, Massachusetts, USA, 2009.
References (Continued…)
[7] Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, Hang Li, "Context-Aware Query Suggestion by Mining Click-Through and Session Data," in KDD, Las Vegas, Nevada, USA, 2008.
[8] Zhen Liao, Yang Song, Li-wei He and Yalou Huang, "Evaluating the Effectiveness of Search Task Trails," in WWW, Lyon, France, 2012.
[9] Allan, Henry Feild and James, "Task-Aware Query Recommendation," in SIGIR, Dublin, Ireland, 2013.
[10] Jiafeng Guo, Gu Xu, Hang Li and Xueqi Cheng, "A Unified and Discriminative Model for Query Refinement,“ in SIGIR, Singapore, 2008.
[11] Jianfeng Gao, Gu Xu and Jinxi Xu, "Query Expansion Using Path-Constrained Random Walks," in SIGIR, Dublin, Ireland, 2013.