Multiple Aspect Ranking Multiple Aspect Ranking using the Good Grief using the Good Grief AlgorithmAlgorithmBenjamin Snyder and Regina Barzilay at Benjamin Snyder and Regina Barzilay at MIT MIT

4.1.2010 Elizabeth Kierstead4.1.2010 Elizabeth Kierstead

IntroductionIntroductionWant to create a system that accounts for many Want to create a system that accounts for many aspects of a user’s satisfaction, and use aspects of a user’s satisfaction, and use agreement across aspects to better model their agreement across aspects to better model their reviewsreviews

Sentiment analysis started out as a binary Sentiment analysis started out as a binary classification task (“Good Restaurant”/ “Bad classification task (“Good Restaurant”/ “Bad Restaurant”) (Pang et al. 2002)Restaurant”) (Pang et al. 2002)

(Pang and Lee, 2005) expanded this to account (Pang and Lee, 2005) expanded this to account for polarity and a multipoint scale for modeling for polarity and a multipoint scale for modeling sentimentssentiments

Other work (Crammer and Singer 2001) allowed Other work (Crammer and Singer 2001) allowed for ranking multiple aspects of a review, but only for ranking multiple aspects of a review, but only addressed the aspects independently, failing to addressed the aspects independently, failing to capture important relations across aspectscapture important relations across aspects

Example: Restaurant review may rate food, decor Example: Restaurant review may rate food, decor and value, and if a user says that “The food was and value, and if a user says that “The food was good but the value was better” independently good but the value was better” independently ranking aspects fails to exploit dependencies ranking aspects fails to exploit dependencies across aspects, and key information is lostacross aspects, and key information is lost

The authors’ algorithm uses the The authors’ algorithm uses the Agreement Agreement Relation Relation to model dependencies across aspectsto model dependencies across aspects

Good Grief algorithm predicts a set of ranks (one Good Grief algorithm predicts a set of ranks (one for each aspect) to minimize the difference for each aspect) to minimize the difference between the individual rankers and the agreement between the individual rankers and the agreement modelmodel

Their method uses the Good Grief decoder to Their method uses the Good Grief decoder to predict a set of ranks based on predict a set of ranks based on bothboth agreement and agreement and individual ranking models, and they find that their individual ranking models, and they find that their joint model significantly outperforms individual joint model significantly outperforms individual ranking modelsranking models

The AlgorithmThe Algorithmm-aspect ranking model m-aspect ranking model with m + 1 components: with m + 1 components: (<(<ww[1], [1], bb[1]>, ... , <[1]>, ... , <ww[m], [m], bb[m]>, [m]>, aa) - First m ) - First m components are the individual ranking models, components are the individual ranking models, one per aspect, and the final vector is the one per aspect, and the final vector is the agreement modelagreement model

ww[i]: a vector of weights on the input features for [i]: a vector of weights on the input features for the the iith aspect th aspect bb[i]: a vector of boundaries dividing [i]: a vector of boundaries dividing the real line into the real line into k k intervals, corresponding to intervals, corresponding to k k ranks of the aspectranks of the aspect

default ranking using PRank (Crammer and default ranking using PRank (Crammer and Singer 2001), which performs rank predictions for Singer 2001), which performs rank predictions for individual aspects of a reviewindividual aspects of a review

agreement model- vector of weights agreement model- vector of weights aa- If all m - If all m aspects are equal, aspects are equal, a x a x > 0, otherwise > 0, otherwise a x a x < 0< 0

|| a x a x| indicates the confidence of the | indicates the confidence of the agreement predictionagreement prediction

The authors use a joint prediction criterion that The authors use a joint prediction criterion that simultaneously takes into account simultaneously takes into account all all model model components, assessing the level of grief components, assessing the level of grief associated with associated with iith aspect ranking model th aspect ranking model g_i(g_i(x, x, r[i]) and the grief of the agreement r[i]) and the grief of the agreement model model g_ag_a((x, r)x, r)

Decoder picks the ranks that minimize overall Decoder picks the ranks that minimize overall griefgrief

Ranking Model- Ranking Model- following previous work on following previous work on sentiment classification (Pang et al., 2002), sentiment classification (Pang et al., 2002), they extract unigrams and bigrams, discarding they extract unigrams and bigrams, discarding those that occur less than three times (30,000 those that occur less than three times (30,000 features extracted)features extracted)

Agreement Model- Agreement Model- also use lexicalized also use lexicalized features like unigrams and bigrams, but features like unigrams and bigrams, but introduce a new feature to quantify the introduce a new feature to quantify the contrastive distance between a pair of wordscontrastive distance between a pair of words

Ex: “delicious” and “dirty” would have high Ex: “delicious” and “dirty” would have high contrast, while “expensive” and “slow” would contrast, while “expensive” and “slow” would have low contrasthave low contrast

Feature Feature RepresentationRepresentation

ResultsResultsThe Good Grief algorithm can rank a training set The Good Grief algorithm can rank a training set perfectly if the independent ranking models can do so perfectly if the independent ranking models can do so

The Good Grief algorithm can also perfectly rank The Good Grief algorithm can also perfectly rank some training sets that the independent ranking some training sets that the independent ranking models could not rank, because of the benefits of models could not rank, because of the benefits of using the agreement modelusing the agreement model

Ex: Ex: The food was The food was good, but not good, but not the ambience. the ambience. The food was The food was goodgood, and so was the , and so was the ambience. The food was ambience. The food was badbad, , but not but not the the ambience. The food was ambience. The food was badbad, and so was , and so was the ambience.the ambience.

ResultsResults

ResultsResults