A Multimedia Interface for Facilitating Comparisons of Opinions

February 11, 2009

Giuseppe CareniniSupervisorUniversity of British Columbia

Lucas Rizoli

2

Opinion data isabundant and useful,but analysis is expensive and difficult

3

Our interfacesupports analysis of opinion data,particularly comparison across entities

4

It supports analysis byvisualizing the data andsummarizing notable comparisons

5

Our user study showsthe visualization is usable,the summarizer’s choices are human-like

6

7

Text mining(Carenini, Ng, & E. Zwart, 2006)

8

9

10

11

12

13

+2

+2

+1

–3

14

Spore

Gameplay Technology

Cell phaseGraphicsCreature creation

Civilization phaseSoundSystem requirements

DRM

15

vs.

16

17

Applications● Consumers● Intelligence

– Competitive analysis– Forecasts

● Research– Survey, questionnaire

18

19

20

Controversiality(Carenini & Cheung, 2008)

21

22

23

24

(Dis)similarity of comparisons● Adapt stats to pairs of distributions● Aspects of a comparison

– counts– means– contros– dists

25

26

27

28

29

30

31

32

33

34

35

Interpreting aspects● Aspects in range 0–1● Interpretation

– means 0.6 != contros 0.6 != dists 0.6

36

37

38

Selecting comparisons

1. Filter out low-count comparisons2. Rank by # of extreme aspects (VD, VS)3. Realize top comparisons

• Sentence templates

39

40

Study goals● Evaluate content selection strategy

– Matches human selections?● Better than baseline?

– Humans like selections?● Usability of visualization

41

42

43

Baseline selection strategies● Naïve

– Randomly select which and how many● Semi-informed

– Likely select same how many as subjects– Randomly select which

44

Matches human selections?

45

Humans like selections?

0.500 0.449 0.454

46

Humans like selections?

0.500 0.449 0.454

● Roughly 20% difference● 60% of system selections marked as “good”

47

Usability

48

Opinion data isabundant and useful,but analysis is expensive and difficult

Our interfacesupports analysis of opinion data,

particularly comparison across entities, by

visualizing the data and

summarizing notable comparisons

Our user study showsthe visualization is usable,

the summarizer’s choices are human-like

49

Thank you

50

Future work● Tune thresholds and aspects● Analysis of human reasoning● Machine learning for selection

51

Future work● More evaluation of visualization

– Interaction– Deeper heirarchies– Different data– Insight

● Multiple entities● Improved summarization

– Visual cues

52

Future work● Machine learning–based selection

– Trained on study data– Which

● Regression on comparison selection scores– How many

● Max # of comparisons (2)

53

Sentence construction● Always main claim: counts● All other aspects relate to counts

– Support: same (dis)similarity as counts– Contrast: different

● Always mention means● Mention contros, dists when they are extreme

54

counts

supp

ortcontrast

means contros dists

55

counts

supp

ortcontrast

means contros dists

56

counts

supp

ortcontrast

meanscontros

57

Study● 36 subjects

– 24 female– 19–43 years old

● 22 different pairs of entities– Subjects saw ~4 each

58

Data generation● No existing dataset● Generated similar to existing datasets

– Distribution, modality● Explore space of possible data

– Too large– Representative of larger space

59

Generated data● Generic camera features

– Consistent with scenario● Simple heirarchy

– Reduce visual and task complication

60

Generated data● 9 comparison types

– Constraints on aspects– Range of support/contrast

● 22 summary cases– Summaries by type– Range of

● overall,● how many,● which,● others.

61

62

63

64

65

66

67

68

69

70

71

72

73

74