predicting information seeker satisfaction in community question answering
DESCRIPTION
SIGIR 2008. Predicting information seeker satisfaction in community question answering. Yandong Liu, Jiang Bian , Eugene Agichtein from Emory & Georgia Tech University. CQA - Community Question Answering 問答 社 群 近來成為一個尋找資訊的有效方法. IS ASKER IN QA COMMUNITY SATISFIED?. - PowerPoint PPT PresentationTRANSCRIPT
Predicting information seeker satisfaction incommunity question answering
SIGIR 2008
Yandong Liu, Jiang Bian, Eugene Agichtein from Emory & Georgia Tech University
CQA -Community Question Answering
問答社群近來成為一個尋找資訊的有效方法
IS ASKER IN QA
COMMUNITY SATISFIED?
IS ASKER IN QA
COMMUNITY SATISFIED?發問者是否得到滿意的回答主宰著問答社群的發展或沒落
Asker use case
Lifecycle of a question in CQA
Asker use case
Lifecycle of a question in CQA
DefinitionAn asker in a CQAis considered satisfied iff:
The asker personally has closed the question, selected the best answer, and provided a rating of at least 3 stars for best answer quality.
DefinitionThe Asker Satisfaction Problem:
Given a question submitted by an asker is CQA, predict whether the user will be satisfied with the answers contributed by the community.
Features
Question Question-Answer Relationship Asker User History Answerer User History Category Features Textual Features
Formally a two-class classification problembutprimarily focus on the satisfied class.
Human judgment
Amazon’s paid rater service Mechanical Turk
Datasets
•Random 5000 from top 10000•Skewed distribution•Satisfaction varies by category
Setting
Methods Human Heuristic Baseline
Evaluation Metrics Precision Recall F1 Accuracy
ASP ASP_SVM ASP_RandomForest ASP_C4.5 ASP_Boosting ASP_NaiveBayes
Result
Selected features
Analysis
Online vs. Offline
Analysis
Feature Ablation
Analysis
Textual Features
Analysis
Past Experience
Conclusion
First to quantify and predict asker satisfaction
Shown importance of asker history is this
Our system outperform human assessors