augmented social cognition cognition: the ability to remember, think, and reason; the faculty of...

Augmented Social Cognition

• Cognition: the ability to remember, think, and reason; the faculty of knowing.

• Social Cognition: the ability of a group to remember, think, and reason; the construction of knowledge structures by a group.

• Augmented Social Cognition: Supported by systems, the enhancement of the ability of a group to remember, think, and reason; the system-supported construction of knowledge structures by a group.

Citation: Ed H. Chi. The Social Web: Opportunities for Research. IEEE Computer, Sept 2008

1

Collective Intelligence

Augmented Social Cognition

Higher Productivity via Collective Intelligence

Intelligence that emerges from the collaboration and competition of many individuals

search

sharing

foraging

TagSearch: Mining social data for automatic data clustering and organization:

• Better organization via user-assigned tags

• Better UI for browsing interesting contents

• Recommendation instead of just search

Social Transparency create trust and attribution:

• Increase participation via attribution

• Increase credibility and trust with community feedback

• Reduce wiki risks

SparTag.us: sharing of interesting contents:

• A notebook that automatically organizes your reading

• Social sharing of important and interesting tidbits

• Viral sharing of highlighted and tagged paragraphs

Foundation:• Understanding of human

cognition and behavior• Data mining of social data

Generic benefits:• Greater trust• Better decision-making• Useful sharing of info• Auto-organization thru

social data

2

• Bier, E. A. ; Good, L. ; Popat, K. ; Newberger, A. F. A document corpus browser for in-depth reading. Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries (JCDL 2004); 2004 June 7-11; Tucson; AZ; USA. NY: ACM; 2004; 87-96.

• Budiu, R.; Pirolli, P. L.; Hong, L. Remembrance of things tagged: How tagging effort affects tag production and human memory. 27th Annual CHI Conference on Human Factors in Computing Systems (CHI 2009); 2009 April 4-9; Boston MA.

• Ed H. Chi, Rowan Nairn, Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser, ACM International Conference on Intelligent User Interfaces Workshop on Social Recommender Systems, 2010

• Held, C., & Cress, U. (in press). Learning by Foraging: The Impact of Social Tags on Knowledge Acquisition. In U. Cress, V. Dimitrova, & M. Specht (Eds.), Learning in the Synergy of Multiple Disciplines, Proceedings of the EC-TEL 2009, LNCS Vol. 5794. Berlin/Heidelberg: Springer.

• Hong, L. ; Chi, E. H. ; Budiu, R. ; Pirolli, P. L. ; Nelson, L. SparTag.us: a low cost tagging system for foraging of web content. Proceedings of 2008 International Working Conference on Advanced Visual Interfaces (AVI'08); 2008 May 28-30; Naples; Italy.

• Hong, L.; Chi, E. H. Annotate once, appear anywhere: collective foraging for snippets of interest using paragraph fingerprinting. 27th Annual CHI Conference on Human Factors in Computing Systems (CHI 2009). 2009 April 4-9; Boston, MA.

• Nelson, L.; Convertino, G.; Pirolli, P. L.; Hong, L.; Chi, E. H. Impact on process by a social annotation system: a social reading experiment. Thirteenth International Conference on Human-Computer Interaction (HCI International 2009); 2009 July 19-24; San Diego, CA.

3

Annotate Once, Appear Anywhere: Collective Foraging for Snippets of Interest Using

Paragraph Fingerprinting

Lichan Hong and Ed H. Chi

Palo Alto Research Center

2009 ACM International Conference on Human Factors in Computing SystemsWorkshop on Social Search and Sensemaking (cited by 6)

Presented by Jun-Ming Chen 4/14/2010

Outline

• Introduction• Paragraph fingerprinting• Benefits of paragraph fingerprinting• Implementation and evaluation• Conclusion

5

Introduction

• The passages in these web pages are often cut-and-pasted• How we explore the idea of paragraph fingerprinting to achieve the goal of

“annotate once, appear anywhere”

• SparTag.us– attaches users’ annotations to the contents of paragraphs– enabling annotations to move along with the paragraphs within dynamic live

pages – travel across page boundary to other pages as long as the paragraph contents

remain intact

6

IntroductionIntroduction Paragraph fingerprintingParagraph fingerprinting Benefits of paragraph fingerprintingBenefits of paragraph fingerprinting Implementation and evaluationImplementation and evaluation ConclusionConclusion

Paragraph fingerprinting

• When a user loads a web page in his browser, – we traverse the Document Object Model (DOM) tree of the page in a top-

down manner to extract paragraphs– A DOM element is considered to be a paragraph if its HTML tag matches one

of a predefined list • <p> or <h2>

• For each paragraph, we remove the HTML tags from its textual content and compute a fingerprint from the remaining text.

• For each word of the paragraphs, – we modify the DOM tree to enclose it with the HTML tag <span>, making the

word live and clickable [8]

7


[8] Hong, L., Chi, E., Budiu, R., Pirolli, P., and Nelson, L. SparTag.us: A Low Cost Tagging System for Foraging of Web Content. Proc. AVI’08, 65-72.


• Subsequently, the user can click on words in a paragraph to tag the paragraph, or highlight phrases and sentences with mouse click-and-drag – Figure 1 shows a paragraph that has been tagged with keywords “CHI 2009

submission”. Two different parts of the paragraph have also been highlighted in yellow

• When the user annotates a paragraph, the tags or highlights, in conjunction with the paragraph fingerprint, are sent to our web server and stored in a database

8

Benefits of paragraph fingerprintingBenefits of paragraph fingerprinting Implementation and evaluationImplementation and evaluation ConclusionConclusionIntroductionIntroduction Paragraph fingerprintingParagraph fingerprinting


• As the user visits a web page, – we use the fingerprints of those paragraphs appearing on the page to query

the database to see if some of those paragraphs have been annotated by the user or his friends

• If so, those annotations are fetched from our server and displayed with their associated paragraphs.

9

Benefits of paragraph fingerprintingBenefits of paragraph fingerprinting Implementation and evaluationImplementation and evaluation ConclusionConclusionIntroductionIntroduction Paragraph fingerprintingParagraph fingerprinting

Benefits of paragraph fingerprinting

11


A paragraph of a blog site was annotated when it was placed at the top of the page (see the scroll bar).

A few hours later, the same annotated paragraph moved down the page as new stories had been added to the top of the page

Handle dynamic web pages


12


A paragraph of the CHI2008 authors’ guides was annotated

The same paragraph appears in the CHI2009 authors’ guides. Our annotations show up even though the new guides are located in a different URL

Those annotations travel across page boundary to other pages


13


A portion of a friend’s notebook as viewed by user hong. The first paragraph was so interesting to user hong that he decided to annotate the paragraph himself

Those annotations travel across users

Implementation and evaluation

• SparTag.us consists of two parts: a Firefox extension and a server• Firefox extension

– GreaseMonkey [7] script and a browser toolbar– The toolbar

• providing shortcuts to key functionalities such as turning on/off SparTag.us, editing the list of friends, and opening the notebook, etc.

14


[7] GreaseMonkey. https://addons.mozilla.org/en-US/fire fox/addon/748. Retrieved Dec, 2008.

https://addons.mozilla.org/en-US/fire


• To understand the impact of annotation sharing enabled by paragraph fingerprinting, we have conducted a lab study– no SparTag.us, – SparTag.us with no friends– SparTag.us with an expert friend

• Our result shows that– people with access to social annotations obtained significant learning gains [9]– users found SparTag.us intuitive and easy to use, and liked the combination of

tagging and highlighting features [8]– examining the comprehension and memory effects of the Click2Tag interface

vs. the type-to-tag interface [3]

15


[3] Budiu, R., Pirolli, P., and Hong, L. Remembrance of Things: How Tagging Effort Affects Tag Production and Human Memory. To appear in Proc. CHI’09. [8] Hong, L., Chi, E., Budiu, R., Pirolli, P., and Nelson, L. SparTag.us: A Low Cost Tagging System for Foraging of Web Content. Proc. AVI’08, 65-72. [9] Nelson, L., Held, C., Pirolli, P., Hong, L., Schiano, D., and Chi, E. With a Little Help from My Friends: Examining the Impact of Social Annotations in Sensemaking Tasks. To appear in Proc. CHI’09.


• Click2Tag: – Participants had to tag the passage with relevant words by clicking on words

from the passage. The tags were displayed in a box under the passage and could not be modified by the participants

• people fixated more on particular words in the text• led them to rehearse the text content more• perform bottom-up, content-driven tagging

• Type-to-tag: – Participants had to tag the passage with any relevant tags that they could

generate, and type those tags in a box under the passage• people tended to fixate less on any specific words• used their background knowledge to generate tags for the text• the tagging process was top-down, knowledge driven and entailed more

elaboration and connection with prior knowledge16


Conclusion

• In this paper we propose to use paragraph fingerprinting to achieve the goal of “annotate once, appear anywhere” in SparTag.us– not only eliminates the cost and annoyance of re-producing the annotations,

but also encourages the social sharing of information nuggets– We are interested in learning more about these cases and plan to address

them in future work

17


Information Seeking with Social Signals: Anatomy of a

Social Tag-based Exploratory Search Browser

Ed H. Chi, Rowan Nairn

Palo Alto Research Center

2010 ACM International Conference on Intelligent User InterfacesWorkshop on Social Recommender Systems

Presented by Jun-Ming Chen 4/9/2010

Outline

• Introduction• The TagSearch algorithm• MrTaggy browsing / Search interface• Experiment Design• Conclusion

19

Social Search Survey

[Evans & Chi, CSCW2008]

• 150 user surveys• Help understand the importance of:

– social cues and information exchanges– vocabulary problems– distribution and organization

20Brynn Evans, Ed H. Chi. Towards a Model of Understanding Social Search. In Proc. of Computer-Supported Cooperative Work (CSCW),ACM Press, 2008.

TagSearch Exploratory Focus

3 kinds of search

navigational transactional

28% 13%

You know what you want and where it is You know what you want to do

Existing search engines are OK

informational

59%

You roughly know what you want

but don’t know how to find it

Difficult for existing search engines

Opportunity

21

Introduction

22[10] Furnas, G.W., Landauer, T.K., Gomez, L.M. And Dumais, S.T.. The vocabulary problem in human-system communication. Communications of the ACM , 30 (1987), 964-971.

IntroductionIntroduction The TagSearch algorithmThe TagSearch algorithm MrTaggy interfaceMrTaggy interface Experiment DesignExperiment Design ConclusionConclusion

Introduction

• Using social tagging data as “navigational advice” and suggestions for additional vocabulary terms

• To combat noisy patterns in tags, we have designed a system using probabilistic networks to model relationships between tags, which are treated as topic keywords– MrTaggy.com

23


24

The system enables users to quickly give relevance feedbacks to the system to narrow down to related concepts and relevant URLsThe system enables users to quickly give relevance feedbacks to the system to narrow down to related concepts and relevant URLs

A tag-based exploratory search system


Introduction

• TagSearch algorithm– performs tag normalizations that reduces the noise and finds the patterns of

co-occurrence between tags to offer recommendations of related tags and contents [15]

• Experiment Design– provide a quick overview of the user study reported previously

25

[15] Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems CHI '09. ACM, New York, NY, 625-634.


The TagSearch algorithm

• Here we describe an algorithm called TagSearch that uses the relationships between tags and documents to suggest other tags and documents– First form a bi-graph between document and tagging pairs

(Bi-graph between document/tag)– Steps– TagSearch Architecture

26


Bi-graph between document/tag

• Spreading Activation in a bi-graph– For a URL, the probability p(Tag|URL)

can be roughly estimated by the number of times a particular tag is applied by users divided by total number of times all tags are used for a URL

• Spreading activation have been used in many other systems for modeling concepts that might be related, or to model traffic flow through a website [5]

27

Tags URLs

P(URL|Tag)

P(Tag|URL)

1/3

1/3

1/3

1/2

1/2


[5] Ed H. Chi, Peter Pirolli, Kim Chen, James Pitkow. Using Information Scent to Model User Information Needs and Actions on the Web. In Proc. of ACM CHI 2001 Conference on Human Factors in Computing Systems, pp. 490--497. ACM Press, April 2001. Seattle, WA.

蔓延激發（ Spreading Activation ）

• 激發現象：就是提取訊息 / 概念意義性的處理過程

• 關鍵特徵：– 字彙與語意的激發會蔓延至其他相關連

的字彙和概念

– 激發沿著儲存的路徑蔓延到整個網路中只要有一個概念被激發，則激發會從這個概念蔓延到全部與其有連接的概念上

28概念圖的原理：語意網路與命題敘述國立臺灣師範大學教育與心輔導學系陳學志教授

• 當一個語義節點處在激發狀態，其激發程度會傳送給與之相聯結的其他語義節點– 激發節點與少數的語義節點相聯結，則每一個相鄰節點平均所能分享的

激發量也就越強• 產生的語意連結越強

– 激發節點與很多的語義節點相聯結，則每一個相鄰節點平均所能分享的激發量也就越少

• 產生的語意連結越弱– 此即所謂的「發散效果」（ fan effect; Anderson, 1984 ）

• Bi-graph between document/tag• A sketch of the idea behind the algorithm is as follows

29

John R. Anderson and Peter L. Pirolli, Spread of Activation. Journal of Experimental Psychology: Learning, Memory, and Cognition1984, Vol. 10, No. 4, 791-798

To suggest tags To suggest documents

• form a “tag profile” for a tag– which is the set of other tags that are

related to the tag– To compute the tag profiles, we use the

bi-graph to perform a spreading activation to find a pattern of other tags that are related to a set of tags

– Once we have the tag profiles, we can find other tags that are related by comparing these tag profiles

– That is, for a given tag, we can compare its tag profile to other tag profiles in the system to find the top most related tags

• form a “document profile” for a tag, – which is the set of other documents that

are related to the tag, similarly using spreading activation

– then find other tags that are related using these document profiles

• form “tag profiles” for a document– which is the set of other tags that are

related to that document, – again using the spreading activation

method– then compare these tag profiles for

documents to other document tag profiles to find similar documents

• form “document profiles” for a document– Using the spreading activation method

over the bigraph – compare these document profiles for

documents to find similar documents

30

• construct a bi-graph between URLs and tagging keywords

• form [url, tag1, tag2, tag3, tag4, ….] [url, tag1], [url, tag2], and so on

• Given tuples in the form [url, tag], we can form a bi-graph of URLs linked to tags

31

Step 1Step 1 Step 2Step 2 Step 3Step 3

• construct “tag profiles” and “document profiles” for each URL and each tag in the system

• In this case, we use spreading activation to model tag and concept co-occurrences

32


• Specifically, the tag profiles and document are computed using spreading activation iteratively as vectors A as follows:

33


• After “n” steps (which can be varied based on experimentation),– depending on whether the spreading activation was stopped on the tag side of

the bi-graph or the document side of the bi-graph– we will have a pattern of weights on tags or documents

• These patterns of weights form the “tag profiles” or “document profiles”

34


• Having constructed these profiles, we now have several options for retrieval. These profiles form the basis for doing similarity computations and lookups for retrieval, search, and recommendations

• For example, – For a given document,

• if we want to find more related document to it, we have three options• if we want to look for related tags to it, we have three options

– For a given tag, • if we want to find related documents or related tags to it

35


if we want to find more related document if we want to find for related tags

a. Lookup the corresponding document profile for that document, and choose the top highest weighted documents in that profile and return that set

b. Use the corresponding document/tag profile for that document and compare it against all other document/tag profiles for other documents in the system, and find the most similar profiles and return the matching documents

c. If the document is not already in the bi-graph, we can first use standard information retrieval techniques (for example, cosine similarity of the document word vectors) to find the most similar document that is in our bi-graph, and use method (a) or (b) above to find related documents in our bi-graph

a. Lookup the corresponding tag profile for that document, and choose the top highest weighted documents in that profile and return that set

b. Use the corresponding document/tag profile for that tag and compare it against all other document/tag profiles for other tags in the system, and find the most similar profiles and return the matching tags

c. If the document is not already in the bi-graph, we can first use a standard information retrieval technique to find the most similar document that is in our bi-graph, and use method (a) or (b) above to find related documents in our bi-graph

36

For a given document,For a given document,

if we want to find more related document if we want to find for related tags

a. Lookup the corresponding document profile for that document, and choose the top highest weighted documents in that profile and return that set

b. Use the corresponding document/tag profile for that document and compare it against all other document/tag profiles for other documents in the system, and find the most similar profiles and return the matching documents

c. If the document is not already in the bi-graph, we can first use standard information retrieval techniques (for example, cosine similarity of the document word vectors) to find the most similar document that is in our bi-graph, and use method (a) or (b) above to find related documents in our bi-graph

37

For a given document,For a given document,

a. We can again use similar methods (a) or (b) as described above, if the tag already exists in our bi-graph

b. If the given tagging keyword is not in the bi-graph, we can first perform a standard keyword search to find the first initial related documents and tags

c. We can then further refine the result set by the above methods

38

For a given tag,For a given tag,

if we want to find related documents or related tags to it

TagSearch Architecture

• MapReduce computation over a large data set • 150 Million+ bookmarks

39


Baseline Interface

40

MrTaggy browsing / Search interfaceMrTaggy browsing / Search interface

Exploratory Interface

41

MrTaggy browsing / Search interfaceMrTaggy browsing / Search interface

Related Tag FeedbackRelated Tag Feedback

Related Tag FeedbackRelated Tag Feedback

Experiment Design

• We recently completed a 30-subject study of MrTaggy and Kammerer et al. describes the study in detail [15]– In this study, we analyzed the interaction and UI design– The main aim was to understand whether and how MrTaggy is beneficial for

domain learning

• We compared the full exploratory MrTaggy interface to a baseline version of MrTaggy that only supported traditional query-based search

• In a learning experiment, we tested participants’ performance in three different topic domains and three different task types

43[15] Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems CHI '09. ACM, New York, NY, 625-634.


Experiment Design• 2 interface x 3 task domain design

– 2 Interface (between-subjects)

• Exploratory vs. Baseline– 3 task domains (within-subjects)

• Future Architecture, Global Warming, Web Mashups• 30 Subjects (22 male, 8 female)

– Intermediate or advanced computer and web search skills– Half assigned Exploratory, half Baseline.

• For each domain, single block with 3 task types:– Easy and Difficult Page Collection Task– Summarization Task– Keyword Generation Task

44


Page Collection

45


Summarization Tasks

46


Procedure

• Prior Knowledge Test• Task Domain

– Interaction Behaviors– summarization– keyword generation task

47


Results: Interaction Behaviors

• Number of Queries– Effect of Interface on number of queries (p < .01)

• Exploratory (M=7.81) > Baseline (M=3.77)

• Time Taken– Effect of Interface on time taken (p < .01)

• Exploratory (7.7min) > Baseline (6.6min)

Subjects using the MrTaggy full exploratory interface took advantage of the additional features provided by relevance feedback,

– without giving up their usual manual query typing behavior– They also spent more time on tasks and appear to be more engaged in

exploration than the participants using the baseline system

48


0

1

2

3

4

5

futurearchitecture globalwarming mashups

nu

mb

er

of

sum

mar

y a

sp

ect

s

baseline exploratory For learning outcomes, subjects using the full exploratory system generally wrote summaries of higher quality compared to baseline system users

For learning outcomes, subjects using the full exploratory system generally wrote summaries of higher quality compared to baseline system users

Results: Summarization Tasks

– Quality of summarization scored (Cohen’s Kappa=0.7)

– ANCOVA with Prior Knowledge as covariate

– Exploratory Interface scored higher in Future Architecture (p<.05) and Global Warming (p<.05)

– For Web Mashup, Prior Knowledge correlated positively with performance (r=.51)

49


Results: Keyword Generation Tasks

0

2

4

6

8

10

12

14

16

18

globalwarming futurearchitecture mashups

nu

mb

er o

f ke

yw

ord

s

baseline exploratory

– Subjects using the exploratory system were able to generate more reasonable keywords than the baseline system users for topic domains of medium and high ambiguity

51


Summary of the evaluation

• Exploratory interface users:– performed more queries, – took more time, – wrote better summaries (in 2/3 domains), – generated more relevant keywords (in 2/3 domains)

• Suggestive of deeper engagement and better learning• Some evidence of scaffolding for novices in the keyword generation and

summarization tasks

52


Conclusion

• In this paper, we described the detailed implementation of the TagSearch algorithm. We also summarized a past study on the effectiveness of the exploratory tool

• Harnessing user-generated tags to enrich content for social search• The results of this project point to the promise of social search to fulfill a

need in providing navigational signposts to the best contents

53


augmented social cognition cognition: the ability to remember, think, and reason; the faculty of...

Documents

social web

social signals

social cognitioncognition

augmented social cognition

social recommender systems

social annotation system

impact of social tags

social reading experiment