lee sung eob mastersthesisproposal03

KAIST GSCT / Cultural Management & Policy Lab / Lee Sung Eob / Master Degree’s Thesis Theme Presentation03

Expanded Reproduction ofSocially Shared Opinions via Qtag

A presentation about a Master Thesis

KAIST Graduate School of Culture TechnologyAffiliation

Lee, Sung EobWritten & Presented by

Han, ‘Steve’ SangKiAdvisor Professor

Qtag: Tagging as a Means of Self-expressionand Aggregation of Collective Opinion for VLSCQtag: Tagging as a Means of Self-expressionand Aggregation of Collective Opinion for VLSC


General Perspective

Qtag: Tagging as a means of rating, Self-Expression & sharing

• Introducing a new system

Published for two ACM sponsored conferences• ACM SIGWEB Hypertext 2007 (Manchester, U.K. / 2007.09.02)• ACM SIGDOC 2007 (El Paso, U.S. / 2007.10.22)• Currently Nominated for a future publication of Communications of ACM

How to extract and represent public opinions from VLSC (Very Large Scale Conversation)

• Solving a real-life problemSociology

Computer

Science

My research consists of two parts

Design

A Multidisciplinary area called “Social Computing”


Contents

Key Words: Collaborative Tagging / Folksonomy /Collective Intelligence / Wisdom of Crowds

Part1. Qtag1. Preliminaries

2. Objectives

3. Proposed Scheme

4. Experiments

5. Results

6. Conclusion

The contents of This Presentation

Part2. Collective Opinion1. Preliminaries

2. Objectives

3. Proposed Scheme

4. Experiments

5. Results

6. Conclusion

The contents of Sung Eob’s Master’s Thesis

1. Introduction

2. Related Work 2.1. Collaborative tagging system

2.2. Collective Intelligence

3. Introducing Qtag, a new means of rating & self-expression

3.1. Qtag Conceptual Model

3.2. Implementation and experimental result

4. Aggregation and Representation of Collective Opinion 4.1. Collective Opinion

4.2. Qtag guided VLSC model

4.3. Implementation and experimental result

5. Implication of result

6. Conclusion and future work 6.1. Conclusion

6.2. Outlook


Part 1

Qtag: Tagging as a means of rating & Self-Expression•Introducing a new system


About Tagging

However, users use tags for rating & self-expression

Tags are short Freeform labels describe items

• Meta-Data generated by Folks (users) a.k.a. Folksonomy• One of major means of categorization of Web contents• The leading fields where tags are deployed are IF & IR

Preliminaries


From Previous Studies

People tag to rate & express their opinions

However, this kind of tags hardly shared among usersBecause, users tend to use diversified expressions

Preliminaries


Tags for ratings and opinion-expressionIs a common phenomenon

Tagging Revisited

Marlow et al. provided a comprehensible taxonomy of tagging incentives, assigning six different incentives. He noted incentives such as Self Presentation and Opinion Expression.

• Marlow et al.

They classified tags from del.icio.us into seven different classes according to their functions. One of the classes was Identifying Qualities or Characteristics.

• Golder & Huberman

classified tags from the MovieLens Community into three classes based upon a study by Golder and Huberman They classified Subjective Tags, which represent users’ opinions and ratings.

• Sen et al.

Preliminaries


The Problem

Tags tagged for rating and opinion-expression scarcely useful as meta-data as people use diversified vocabulary to express their opinions.

• Golder & Huberman

Terrible

Awful

LaughableTalentless

Tags hardly shared…

Preliminaries


We propose Qtag

Qtag expands user experience of collaborative tagging

Q stands for Qualitative, Qtag is made to…

1. To Rate and to express Opinions

2. To produce more sharable tags

3. To provide fast & intuitive interpretation

Objective


Simple Idea of Qtag

Qtag is simple augmentation of Plus(+) and Minus(-) signs

Positive Tags

Positive Rating& Opinions

Negative Tags

Negative Rating& Opinions

Neutral Tags

ConventionalTags

Style+Sound+KFed+

Talent-Life-

music-

Pop Music

KFedBritney

Simple

Proposed Scheme


How Qtag Makes Tags more Sharable

Qtag will reduce diversification of expressionvia formulated expressions (Qtag)

Music-

Proposed Scheme


Qtag Experiments

In order to evaluate Qtag, we built a conceptual model

1042 distinct tags were tagged 4083 times by 126 participantsA series of questionnaire is also conducted

in order to elaborate Adoption and Satisfaction of Qtag

In order to aggregate Qtags,four digital product reviews were chosen from ZDNet Korea (http://www.zdnet.co.kr/)and four articles concerning celebrities published on Joins (http://www.joins.com/),

one of the major internet newspapers in Korea.

1st Experiment

Each participant is asked to tag conventionally & Qtag 2 articles & 2

reviews

2nd Experiment

Participants answered a series of questionnaires

The same tagging interface is applied as del.icio.us( Free form / Bag-model / *Blind Tagging )

Experiments


Qtag Cloud (1)

Total Tag Count(709) / Positive Tag Count(344) / Negative Tag Count(210)

Total Tag Count Rating: +189

Experiments

After conducting experiments, we visualized Qtag clouds

A review article about Sanyo Xacti


Qtag Cloud (2)

Total Tag Count(414) / Positive Tag Count(35) / Negative Tag Count(176)

Total Tag Count Rating: -141

Experiments

A celebrity article about Britney Spears

After conducting experiments, we visualized Qtag clouds


Tagging vs. Qtagging (1)

Changes in quantity and quality of tags

Shared tags increased,It elaborates that Qtag filters meta-noise

Total Distinct Tags

Total Shared

400 128 1841Conventional Tagging

642 219 2242Qtagging

Increased By

TaggingFrequency

60.0% 71.1% 21.8%1 12

Entropy of tag data increased However, shared tags increased1 2

Results


Tagging vs. Qtagging (3)Results

The difference of probabilities between QT & CT is calculated

Calculating Normal Distributionfor ‘Conventional Tagging’

(400,0.958) ~ (383.3,16.1)B N

Calculation for normal distributionCTZ

128 383.363.67

16.1CTZ

Calculating Normal Distribution for ‘Qtagging’

(642,0.900) ~ (577.8,57.8)B N

Calculation for normal distributionQTZ

219 577.847.21

57.8QTZ

1 2

Qtagging has absolutely higher 'Shared Tags'

Since ,16.46Q CT QTZ Z Z 21

Qtag improves tag sharing


Implementation & Contribution Conclusion


2. Qtag encourages users to tag more sharable tags in case of rating & self-expression

3. Participants generally accepted & Qtag system

1. Proposed a new formulated means of tagging for rating and self-expression


Part 2

How to extract and represent public opinions from a very large scale conversation (comments)

• Solving a real-life problem


Collective IntelligencePreliminaries

Nobody knows everything, but everybody knows something

Collaborative Tagging & Qtagging is also tools forharnessing Collective Intelligence or Wisdom of crowds

Collective Intelligence is a form of intelligence that emerges from the collaboration and competition of many individuals.

Appears in a wide variety of forms of consensusdecision making in Bacteria, Animals, Human and Computers


New Domain of Qtag

Qtag can be a tool of extracting public opinions from comments

We define this data aggregating as “Collective Opinion”Qtag will expend the domain of collective intelligence

Preliminaries

There must be dominant opinions about this article

If public opinions could be revealed,more productive debate via VLSC would be possible

However, readers put their eyes on comment lists about 3 to 15 secondsCurrent interface makes hard to extract dominant opinions among VLSC


Collective Opinion vs. Intelligence

Collective Opinion is subsidiary to Collective Intelligence

According to degree of interactivity, Collective Opinion isthe link between Collective Intelligence & Wisdom of Crowds

CollectiveIntelligence

CollectiveOpinion

Wisdom ofCrowds

Abstraction & Enhancement

Abstraction & Grouping

Social ProofingProblem Solving

High degree of Interactivity

Diversified degree of interactivity

Minimum degree of Interactivity

IndependencyAmong

Participants

SimilarityData size matters / Shares similar process

(Data Aggregation & Mining)

Knowledge Production(Wikipedia)

Aggregation of public opinions

(Comments)

Problem Solving(Recommendation)

Applications

Preliminaries


Collective OpinionCollective Intelligence

What is Collective Opinion

Collective opinion is mining public opinionsfrom a very large scale conversation (Comments)

Qtag will harness Collective Opinion & encourage debatesamong a Very Large Scale Conversation (VLSC)

Problem Solving / Data Producing OrientedOmni-directional Approach

Data Aggregation OrientedMulti-directional Approach

Preliminaries


Sharing Opinions via Qtag


Qtag is applied in order to enhance VLSC environment

1. To provide breadcrumbs to access old comments

2. To aggregate & represent public opinions

3. To provide guides for writing new comments

Objective


User Scenario

Massing process for dominant public opinionsmay be similar to following example

QtagsJYPWondergirls-가창력 -

Qtags

박진영Wondergirls+

완소희 +

Comments난 상관없다 . 원더걸스 너무 좋아 .특히 완소희…

CommentsJYP 의 원더걸스 노래 너무 못한다 .

Proposed Scheme

Comments원더걸스가 뭐가 좋다고…짜증해서 태깅한다

QtagsJYPWondergirls-Tell Me-

Qtag may encourage debates & represent dominant public opinionsfor a very large scale conversation


2nd Set of Qtag Experiments

This set of experiments will evaluate Qtag foraccumulating & representing Collective Opinion

In order to aggregate Qtags,A number of internet articles which can trigger debates will be picked from renowned sources

Experiments is openly deployed to attract wide range of internet users

1st Experiment

Qtag will be collected and a set of questionnaire will be conducted

2nd Experiment

Participants will answer a series of questionnaires about usability

Tagging interface will be different from the first set of experiments( Free form / Bag-model / *Open Tagging / synonym)

1st Experiments conducted for one week from 25 Nov to 2 Dec12 popular contents are posted, 401 participants participated

1531 comments & 5014 times tagged

Experiments


Implementation & Contribution Conclusion

VLSC environment can be improved via Qtag

2. Public opinions among VLSC can be aggregated & represented by Qtag

3. We speculate that comment writing may assisted by Qtag

1. Access to old comments can be improved via Qtag (breadcrumbs)


Thank You for Paying Attention

Q&A


Appendix

Extra Information about Part 1


An Extreme Case



Tag Sharing Status

We analyzed tag data from mar.gar.in. To elaboratewhether tags for rating can be shared or not

Introducing mar.gar.in, Korean replica of del.icio.us

3000Registered

Users

73,001bookmarks

24,000distinct tags


Tag Data Analysis

Our tag classification is based on Golder & Huberman’s

Sen et al. customized Golder & Huberman’stag classifications for their own use. So did we

Functions Tag Perform for bookmarks Classified by Golder and Huberman

Types of Tags optimizedfor social shopping and networking

Identifying What (or Who) it is about

Identifying What It is

Identifying Who Owns It

Refining Categories

Identifying Qualities or Characteristics

Self Reference

Task Organizing

Identifying Tags

Descriptive Tags

Personal Tags


Sharable tag distribution

We analyzed tag data which is shared by two or more usersto elaborate whether tags for ratings and expressing opinion

can be shared or not

We assigned two coders, and they distributed tags manuallyand reached consensus for tags failed to yield agreement

Identifying Tags93.42%

Personal Tags0.92% Others

0.86%Descriptive Tags

4.79%


Answer for 1st Research Question

RQ1. Are tags for rating and expressing opinions shared easily?

Tags for rating and expression are hardly shared

4.8%Of sharable tags are “Descriptive Tags”


Qtag Design

Basically Qtag system is not much differentfrom current tagging system

Qtag is consisted of four following components

Featured Item or People

A homogeneous kinds of products or people which shares the same qualities

Related Contents

Reviews or Articles about featured items (Actual Tagging Source)

Qtags

KFed -Music-

Talent-

Life-

Dance+

Pop Music

Fashion+

Dance Music

Positive / Negative / Neutral Tags

Qtag Counts

Demonstrate overall ratings and reputation

Positive Tag

Neutral Tag

Negative Tag

10

5

20

-10 (35)


Domains of Qtag

A homogeneous product line-up & peoplewhich & who shares the same qualities for comparison

Qtag may perform well for Social Shopping & SNS

A homogeneous product line-up shares the same specifications

CCD+ Grip- Dslr Weight-

Possible Qtags

People who shares similar qualities

Look+ personality- Students

Possible Qtags


Qtag May Control Wild Expressions of Tags

The causes of expression may be limited

Music

Privacy

Appearance

Dance

GeneralReputation

Qtag will provide a guideline offormulated expression via augmenting signs

Proposed Scheme


When there are 642 distinct tags When there are 400 distinct tags


A Simple speculation about the probability of sharing tags

Results

Qtag SystemConventional

Tagging System

When the number of distinct tag increases,the probability of sharing tags among users decreases.

The same number of participants

126 Personnel

Although there were more distinct tags, There were more shared tags among Qtag dataset

This proves Qtag provides more chance of sharing tags


Research Questions

Will Qtag work? We will answer following research questions

RQ1. Are tags for rating and expressing opinions shared easily?

RQ2. Are users able to apply the Qtag without difficulties?

RQ3. Are users able to interpret valid information from Qtag?

RQ4. Are there changes in quantity and quality of tags?

We analyzed conventional tag data& conducted experiments with a Qtag Conceptual model


Answer for 2nd Research Question

RQ2. Are users able to apply the Qtag without difficulties?

From both aspects of understanding the concept of Qtag& the actual tag count, participants applied Qtag without difficulties

The answerbased on Questionnaire

After a text-based introduction of Qtag was provided, participants responded on a five-point scale as to whether the concept of Qtag was easy to understand.

4.15 stars

The answerbased on aggregated tag data

Comparison of tag counts between conventional neutral tags and tags augmented with positive(+) or negative(-) signs

1231 > 9211231 > 921AugmentedTag Count

NeutralTag Count


Answer for 3rd Research Question

RQ3. Are users able to interpret valid information from Qtag?

Participants generally interpretvaluable information from Qtag Clouds

Participants responded on a five-point scaleas to whether Qtag clouds convey valuable information or not

3.59 stars

Participants requested for better visualization of Qtag clouds


Answer for 4th Research Question

RQ4. Are there changes in quantity and quality of tags?

Shared tags increased,It elaborates that Qtag filters meta-noise

Total Distinct Tags

Total Shared

400 128 1841Conventional Tagging

642 219 2242Qtagging

Increased By

TaggingFrequency

60.0% 71.1% 21.8%1 12

Entropy of tag data increased However, shared tags increased1 2


Extreme Tags


We often tagto rate & express


Conclusion

We have elaborated & speculated following hypotheses

• Users can easily adopt & apply Qtag for rating & self-expression • Qtag improves sharability of tags tagged for rating & self-expression

• From Part1

Qtag outperform conventional tagging in domains of rating & self-expression

• Qtag aggregates dominant public opinions among VLSC• Qtag improves debating condition of VLSC

• From Part2

Qtag outperform conventional comments writing

Qtag has values for research & application for real-life Web services


Massing Snow into Balls

Qtag is the tool for making dominant public opinion standout

Snow = A Very Large Scale Conversation

snowballs = Dominant Public Opinions

Dominant public opinions can be more noticeable via Qtag(Qtag: Tagging as Means of Massing Public Opinions)

PreliminariesProposed Scheme


Let be a total set such that each element (tagging object) has at least tagskN k

1 2 3N N N on it. Then we have a nested property such that

1N

2N

3N

4N

5N


In our case, the 'Total distinct Tag' set is , and 'Shared Tag' set is . And from the nested property, .

1N 2N

1 2N N

The number of total distinct tags : (tags)The number of participants : (person)The average number of each participants tagging frequency : (tags/person)

1( )n N nm

b

Results


Problem DefinitionQ: What is the probability of any random object is also a element in ?P 1x N 2N

SolutionWe will think about the complimentary case.What is the probability of with ?q

1x N1 2

cx N N

Then only one person should pick selected object and others don't.

1( )(1 )1

mm b bq

n n

1 11 ( )(1 ) 1 ( )(1 )1

m mm b b b bp m

n n n n



Extreme Case CheckingIf ,then , : it means that if gets larger,less chance of 'Shared' case.If , then , : it means that if gets smaller,high chance of 'Shared'

n 1q n

n b 0q 1p

0p

From the extreme cases, we could see that the derivation of probability is reasonable. And since is constant for each selection in , we can see that distribution of number of shared tagging follows 'Binomial Distribution' and for large , it can be approximated by 'Normal Distribution‘ .

qp

1N( , )B n p

n ( , )N np npq

n



Checking ‘Conventional Tagging’

400, 126, 1841/126 14.6n m b Then 12514.6 14.61 126( )(1 ) 0.958

400 400p

( ) 400 0.958 383.3E X np ( ) 383.3 0.042 16.1V X npq

1 2

Participants tagged average 14.6 tags for ‘Conventional Tagging Model’The unshared tag production probability reaches near 1.0. This result means that in the case of random tag selection, most tags should be shared.

In the random tag selection case, the number of shared tags supposed to be 383.3.Variance is calculated to calculate normal distribution.

3 4

If , then , : it means that if gets smaller, high chance of 'Shared'n b 0q 1p nIf , then , : it means that if gets smaller, high chance of 'Shared'n b 0q 1p n

1

2

3

p

4



Checking ‘Qtagging’

Then642, 126, 2242 /126 17.6n m b 12517.8 17.81 126( )(1 ) 0.900

642 642p

( ) 642 0.900 577.8E X np ( ) 577.8 0.100 57.8V X npq

1 2

3 4

Participants tagged average 17.6 tags for ‘Conventional Tagging Model’The unshared tag production probability reaches near 1.0. This result means that in the case of random tag selection, 90% of tags should be shared.

In the random tag selection case, the number of shared tags supposed to be 577.8.Variance is calculated to calculate normal distribution.

If , then , : it means that if gets smaller, high chance of 'Shared'n b 0q 1p nIf , then , : it means that if gets smaller, high chance of 'Shared'n b 0q 1p n

1

2

3

p

4



Calculating Normal Distributionfor ‘Conventional Tagging’

Calculating Normal Distribution for ‘Qtagging’

(400,0.958) ~ (383.3,16.1)B N (642,0.900) ~ (577.8,57.8)B N

Calculation for normal distributionCTZ Calculation for normal distributionQTZ

128 383.363.67

16.1CTZ

219 577.847.21

57.8QTZ

1 2

Qtagging has absolutely higher 'Shared Tags'

Since ,16.46Q CT QTZ Z Z 21


lee sung eob mastersthesisproposal03

Technology

qtag objective qtag

qtag experiments experiments

qtag clouds

satisfaction of qtag

quality of tags shared

rating selfexpression

sharable tags

qtag preliminaries objectives