csn11-johan bollen- indiana university of informatics and computing

Introduction Methods Results Ongoing research: Mood contagion Conclusions

Twitter sentiment voorspelt aandelenkoersen

Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot

Smith, Peter Todd, and Ishani Banerji

[email protected] of Informatics and Computing

Center for Complex Networks and Systems ResearchIndiana University

February 9, 2011

Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen


Objective

Public mood states and the markets

Do societies experience varying mood states like individuals?If so, can we assess such mood states from online materials anddetermine its socio-economic correlates?



Outline

1 IntroductionCollective Intelligence: from mobs to crowdsSentiment analysis: from mood to behavior

2 MethodsDataSentiment tracking instrument

3 ResultsCase-studiesCross-validation

4 Ongoing research: Mood contagion

5 ConclusionsDiscussionLiterature


Introduction Methods Results Ongoing research: Mood contagion ConclusionsCollective Intelligence: from mobs to crowds Sentiment analysis: from mood to behavior

Outline








Crowds and mobs: reasons to be pessimistic

H. L. Mencken

“Democracy is the art of running the circusfrom the monkey cage.”

Assumption: large groups = random = bad decisions



Condorcet jury theorem: large groups can make betterdecisions

1785, jury votes, by majority rule, on binary decision:

Definition (Condorcet theorem)

PN =�

N

i=m

�N!

(N−i)!i!

�pi (1− p)N−i

1 where

N = the number of jurors

p = the probability of an individual juror being right

m = the number of jurors required for a majority

1First and last equation.Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen


Condorcet theorem: bigger crowds do better!

Conclusion: large groups makebetter decisions!BUT:

Same individualprobability {right,wrong}Optimization problems

Independent judgements:often not true!



From collective intelligence to crowd sourcing

Emergence

Complex systems and behavior can emerge from multiplicity ofinteractions between simple units

Large groups of networkedindividuals can exhibitcollective intelligence: anthills, fish schooling, bees,routing, humans...

Open Source software

Mechanical Turk

Folksonomies

Wikipedia

and... web optimization(Bollen, 1996)



Not even trying?

The Naughts: Rise of social networking and micro-blogging

Stigmergy: medium coordinates collective intelligence

Facebook: +500M users

Twitter: +150M users

BUT!

Not trying to solve particular problems.Objective: recreation, socializing, sharing



Twitter!

tweets and updates

users broadcast brief text updates to the public or to a limitedgroup of contacts: 140 characters or lessTwitter, Facebook, Myspace

Examples

“Our Rights from Creator(h/t @JLocke). Life,Liberty, PoH FTW! Yourtransgressions = FAIL.GTFO, @GeorgeIII.-HANCOCK et al.”

“at work feeling lousy”



Analyzing the chatter

Predicting the present from side-effects

Mapping online traffic to real-world outcomes

+70M tweets per day +20GB of text per day the EEG of theglobal brain

Box office receipts from Twitter chatter: Asur (2010)

Google trends: flu (verbal autopsies)

Predicting consumer behavior from search query volume(Goel, 2010)

Contagion of “Loneliness” and happiness in social networks(Cacioppo, 2010 - Bollen, 2011)



Link between sentiment, mood and behavior

Behavior is shaped not just by rational, conscious considerations

Emotion plays a significant role in human decision-making(behavioral economics, behavioral finance, social psychology).Emotion plays a tremendously important role online andconsequently in the “real” world, cf. Tunesia, Egypt.

Extract indicators of individual and collective sentiment fromonline media feeds?

Predict not just the present, but the future?



Extracting sentiment indicators from text

Happy tweets.

So...nothing quite feels like a good shower, shave and haircut...love itMy beautiful friend. i love you sweet smile and your amazing souli am very happy. People in Chicago loved my conference. Love you, my sweetfriends@anonymous thanks for your follow I am following you back, great group amazingpeopleUnhappy tweets.

She doesn’t deserve the tears but i cry them anywayI’m sick and my body decides to attack my face and make me break out!! WTF:(I think my headphones are electrocuting me.My mom almost killed me this morning. I don’t know how much longer i can behere.

Different Approaches: Natural Language processing (n-grams) for reviews

(Nasukawa, 2003), topics (Yi, 2003), Support Vector Machines: text

classification (positive vs. negative) using pre-classified learning sets: Gamon

(2004), Pang (2008), Blogs, web sites: mixed approaches. Mishne (2006),

Balog (2006), Gruhl (2005),...Huina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen


Sentiment and mood analysis is difficult for tweets

Individual tweets

Length: 140 characters, lack of text content

Diversity:no standardized training sets, dimensions of mood?

Lack of topic specificity

Public mood from tweet collections and other microblog contents?

We Feel Fine http://www.wefeelfine.org/

Moodviews http://moodviews.com

Myspace: Thelwall (2009), FB: United States Gross NationalHappiness http://apps.facebook.com/usa_gnh/, MichaelJackson (Kim, 2009)


http://www.wefeelfine.org/

http://moodviews.com

http://apps.facebook.com/usa_gnh/


What we did:

Trends in general public mood from a large-scale collection oftweets

Each tweet= patient taking psychometric instrument (POMS)

Large-scale collection of tweets: 10M, 2006-2008

Daily public mood assessment: Time series depictingfluctuations of public mood

Correlations to socio-economic indicators?

Contagion of mood in social network, cf. Haiti disaster?


Introduction Methods Results Ongoing research: Mood contagion ConclusionsData Sentiment tracking instrument

Outline








Data sets

Collection of tweets:

April 29, 2006 to December 20, 2008

2.7M users

Subset: August 1, 2008 to December 2008 - 9,664,952 tweets

2008

log

(n t

we

ets

)

Aug 1 Sep 1 Oct 1 Nov 1 Dec 1 Dec 20

2e

+0

21

e+

03

5e

+0

32

e+

04

1e

+0

5



Each tweet:ID date-time type text1 2008-

11-2802:35:48

web Getting ready for Black Friday. Sleep-ing out at Circuit City or Walmart notsure which. So cold out.

2 2008-11-2802:35:48

web @dane I didn’t know I had an un-cle named Bob :-P I am going to bechecking out the new Flip sometimesoon

· · ·



Collective mood: Profile of Mood States (POMS-bi)

Definition

72 term questionnaire (McNair, 1971) to assess individual moodstates. Each mood term maps into one of 6 mood dimensions:

composed/anxious composed (+1), tense (-1), untroubled (+1), ...

clearheaded/confused confused (-1), clearheaded (+1), mixed-up (-1), ...

confident/unsure weak (-1), strong (+1), timid (-1), ...

energetic/tired vigorous (+1), fatigued (-1), energetic (+1), ...

agreeable/hostile sympathetic (+1), bad tempered (-1), agreeable (+1), ...

elated/depressed mad (-1), good-natured (+1), annoyed (-1), ...

About 6 minutes to complete by individual.Designed for repeated within-subjects testing.



Tweet:

I am so not bored. way too busy! I feel really great!

composed/anxiousclearheaded/confusedconfident/unsureenergetic/tiredagreeable/hostileelated/depressed

0.017250.051250.7256250.666625

0.3610.53175



Aggregating daily tweets into a mood time series

Twitterfeed

~

Calm

Mood indicators (daily)

textanalysis

Happy

Confident

...


Introduction Methods Results Ongoing research: Mood contagion ConclusionsCase-studies Cross-validation

Outline








Ratio of emotional tweets, over time.

23

45

67

89

ratio of # tweets with mood expressions over all tweets%

moo

d ex

pres

sion

s

Aug 08 Sep 08 Oct 08 Nov 08 Dec 08

−1.5

0.0

1.0

resi

dual

(%)

Aug 08 Oct 08 Dec 08 −1.5 −0.5 0.5

0.0

0.4

0.8

residual (%)

prob

abilit

y

Ratio of tweets containing mood expressions vs. all tweets on agiven day, including residuals from trendline.



Public mood trends: overview

composed/anxious

clearheaded/confused

confident/unsure

energetic/tired

agreeable/hostile

elated/depressed

+2sd

+2sd

+2sd

+2sd

+2sd

+2sd

−2sd

−2sd

−2sd

−2sd

−2sd

−2sd

Election08 Thanksgiving08

08/01 09/01 10/01 11/01 12/01 12/20

Figure: Sparklines for G-POMS measured public mood states in August2008 to December 2008 period highlight long-term changes.



Case study 1: November 4th, 2008 - the presidentialelection

composed/anxious


confident/unsure

energetic/tired

agreeable/hostile

elated/depressed

+2sd

+2sd

+2sd

+2sd

+2sd

+2sd

−2sd

−2sd

−2sd

−2sd

−2sd

−2sd

Election08

10/20 11/04 11/19

Figure: Sparklines for public mood before, during and after the USpresidential election on November 4th, 2008.



TFIDF scoring of tweet terms

2008 U.S. Presidential Election

Nov 03 Nov 04 Nov 05robocal poll historibusiness plumber wonvoter result barackcleanser absente propgrandmoth ballot speechrussert turnout resultsocialist barack president-electhalloween citizen hologramacknowledg joe victorirace thoughtfulli ecstat

Table: Top 10 TF-IDF ranking terms 1 day before, on and 1 day afterelection day.



Case study 2: November 27th, 2008 - Thanksgiving

composed/anxious


confident/unsure

energetic/tired

agreeable/hostile

elated/depressed

+2sd

+2sd

+2sd

+2sd

+2sd

+2sd

−2sd

−2sd

−2sd

−2sd

−2sd

−2sd

Thanksgiving08

11/12 11/27 12/12

Figure: Sparklines for public mood before, during and after Thanksgivingon November 27th, 2008.



Long-term changes in public mood: statistical significance

Mood dimension Period 1 Period 2 p-valueAgreeable/Hostile 08/01-20 12/01-20 0.0001338

Mean 1= Mean 2= Difference-0.007sd 1.286sd 1.292sd

Confident/Unsure 08/01-20 12/01-20 0.002381Mean 1= Mean 2= Difference-0.120sd 0.785sd 0.905sd

Composed/Anxious 08/01-20 12/01-20 0.0272Mean 1= Mean 2= Difference

0.162 0.897 0.736

Table: T-tests to compare mood levels in two 20-day periods (August1-20 and December 1-20, 2008) show statistically significant elevatedz-scores for Agreeable, Confident and Composed mood.



Outline








Comparison to existing sentiment tracking tools:OpinionFinder

1.2

51.7

5

OpinionFinder day afterelection

Thanksgiving

!1

1

pre!electionanxiety

CALM

!1

1

ALERT

!1

1

electionresults

SURE

!1

1

pre!electionenergy

VITAL

!1

1 KIND

!1

1

Thanksgivinghappiness

HAPPY

Oct 22 Oct 29 Nov 05 Nov 12 Nov 19 Nov 26

http://www.cs.pitt.edu/mpqa/Theresa Wilson, Janyce Wiebe, and PaulHoffmann (2005). Recognizing ContextualPolarity in Phrase-Level SentimentAnalysis. Proc. of HLT-EMNLP-2005.


http://www.cs.pitt.edu/mpqa/


Table: Multiple Regression Results for OpinionFinder vs. GPOMSdimensions.

Parameters Coeff. Std.Err. t pCalm (X1) 1.731 1.348 1.284 0.20460Alert (X2) 0.199 2.319 0.086 0.932Sure (X3) 3.897 0.613 6.356 4.25e-08 ��Vital (X4) 1.763 0.595 2.965 0.004�Kind (X5) 1.687 1.377 1.226 0.226

Happy (X6) 2.770 0.578 4.790 1.30e-05 ��Summary Residual Std.Err Adj.R2 F6,55 p

0.078 0.683 22.93 2.382e-13



Comparison to DJIA

DJIA daily closing value (March 2008−December 2008

Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2008

8000

9000

10000

11000

12000

13000



Comparison to DJIA

Twitterfeed ~

(1) OpinionFinder

(2) G-POMS (6 dim.)

Mood indicators (daily)

DJIA ~

Stock market (daily)

(3) DJIA

Grangercausality

-n (lag)

F-statisticp-value

textanalysis

normalization

SOFNN

predictedvalue MAPE

Direction %

1

2

t-1t-2t-3

3

t=0value

Figure: Methodological diagram outlining use of Granger causalityanalysis and Self-Organizing Fuzzy Neural Network to predict daily DJIAvalues from (1) past DJIA values at t − 1, t − 2, t − 3, and variouspermutations of Twitter mood values (OpinionFinder and GPOMS).



bivariate-causal analysis: DJIA vs. public mood

Table: Calm (X1), Alert (X2),Sure (X3), Vital (X4), Kind (X5), Happy(X6)

lag XOF X1 X2 X3 X4 X5 X6

1 0.703 0.080� 0.521 0.422 0.679 0.712 0.300

2 0.633 0.004�� 0.777 0.828 0.996 0.935 0.697

3 0.928 0.009�� 0.920 0.563 0.897 0.995 0.652

4 0.657 0.03�� 0.54 0.61 0.87 0.78 0.68

5 0.235 0.053� 0.753 0.703 0.246 0.837 0.05

�



Calm vs. DJIA

-2

-1

0

1

2D

JIA

z-s

core

Aug 09 Aug 29 Sep 18 Oct 08 Oct 28

-2

-1

0

1

2

-2

-1

0

1

2

-2

-1

0

1

2

DJIA

z-s

core

Calm

z-s

core

Calm

z-s

core

bank

bail-out



Table: DJIA Daily Prediction Using SOFNN

Evaluation IOF I0 I1 I1,2 I1,3 I1,4 I1,5 I1,6

MAPE (%) 1.95 1.94 1.83 2.03 2.13 2.05 1.85 1.79�

Direction (%) 73.3 73.3 86.7� 60.0 46.7 60.0 73.3 80.0



Citation:

Johan Bollen, Huina Mao, and Xiao-Jun Zeng. Twitter moodpredicts the stock market. Journal of Computational Science,2010, http://arxiv.org/abs/1010.3003.



Outline








Data set to study mood contagion in Twitter socialnetwork

Nov. 20, 2008 until May 29, 2009

4,844,430 users

129M tweets

Each tweet:

ID, date:time, user ID, text, type, referred



Turning followers into friends

Twitter 2: Nov08-May09, 129M tweets

1 undirected network: Friend= A follows B, B follows A

2 sufficient tweet coverage: More than 1 tweet per day onaverage.

edge weight

Jaccard index of node’s friend list:

wij =Ci ∩ Cj

Ci ∪ Cj

where Ci is k=1 neighborhood of i



Twitter social network: users=130,636, edges=7,586,895



Preliminary sentiment tracking via emoticons

Emoticon ratio

En =tnT

Ep =tpT

where tn and tp number of negative or positive tweets, and T totalnumber of tweets submitted in 188 days

user 1

user 2

;-) ;-) ;-) ;-{ :->

;-/ ;-( ;-( ;-{ :-\ :-\ :-\

N=189

N=210

http://en.wikipedia.org/wiki/Emoticon


http://en.wikipedia.org/wiki/Emoticon


Identifying sour and happy users

On the way home. Yay. Having pizza tonight. I think I got a cold.Poof! Just like that! :-/ - http://bkite.com/02yfl Morning so far:woke up to awful smell - 2 doggie ’presents’ in living room. :-/Then a nearly flat rear tire. :-/ (squared) - http://... Well, Ishould be in bed ... so I guess I should try that. Work at 10 in themorning. :-/ - http://bkite.com/02Q2L Chance of Snow thisweekend. :-/ - http://bkite.com/02V98

I’ve been hearing the words ”last” and ”final” a lot lately. :-)@HipMamaB what? I don’t understand that. why not?? ;-)ScarfWatch’09 - the theater lost & found has it!! Things arelooking up. ;-) It’s Crafty Mama’s night! Yay for CM night. :-)cast-iron skillets are the best. And having a pre-loved one is evenbetter - already seasoned. :-)



Assortativity vs. disassortativity

From: Sid Redner (2008) Networks: Teasing out the missing links, Nature 453, 47-48

See also: M. E. J. Newman (2003) Mixing patterns in networks, Phys. Rev. E 67, 026126



Assortativity vs. disassortativity

PS Bearman (2004). ”Chains of affection”, American Journal of Sociology, 100(1)

Not me ③ ✍✌✎☞



Example twitter mood assortativity

low SWB

high SWB

Red=happy, blue=sad white=neutralHuina Mao (IU) and Johan Bollen (IU)Acknowledgement: Bruno Goncalves, Alex Vespignani, Eliot Smith, Peter Todd, and Ishani BanerjiTwitter sentiment voorspelt aandelenkoersen


Assortativity of positive and negative sentiment



ego network of world’s most depressed user


Introduction Methods Results Ongoing research: Mood contagion ConclusionsDiscussion Literature

Outline








Discussion

Power of collective intelligence:Wisdom of crowds extends to their mood state?Predictive power?Research front: growing support

BusinessCan present models be translated to business applications?Prediction → control?

Issues:User privacy, rightsDependencies on proprietary social networking systemsBusiness vs. science



References

Johan Bollen, Huina Mao, and Xiao-Jun Zeng. Twitter moodpredicts the stock market. Journal of Computational Science,2010, http://arxiv.org/abs/1010.3003. (featured on CNBC,CNN International and Bloomberg News!)

Johan Bollen, Huina Mao, and Alberto Pepe. Determining thepublic mood state by analysis of microblogging posts.Proceedings of the Proc. of the Alife XII Conference, Odense,Denmark, MIT Press, August 2010.

Huina Mao, Alberto Pepe, and Johan Bollen. Structure andevolution of mood contagion in the Twitter social network.Proceedings of the International Sunbelt Social NetworkConference XXX, Riva del Garda, Italy, July 2010

Johan Bollen, Alberto Pepe, and Huina Mao. Modeling publicmood and emotion: Twitter sentiment and socio-economicphenomena. arXiv:0911.1583 (November 2009)



THANK [email protected]


csn11-johan bollen- indiana university of informatics and computing

Technology