Sentiment Analysis

Applied Advertising & Public Relations Research

JOMC 279

"Listening is the study of naturally occurring

conversations, behaviors, and signals—information that may or may not be guided—that brings the

voice of people's lives in to a brand."

Why Do Brands Listen?

• Insights (wants, unmet needs, challenges)• Voice of consumer• Redefine relationships• Understand shifts in perspectives• Understand context & reasons why

Where Do Brands Listen?

• Offline– Comment cards– Trade-show notes– CRM / sales mgmt. systems

• Online– Brand backyard– Customer backyard

Whom Do Brands Listen To?

• Customers• Prospects• Business partners• Friends, contacts, followers• Others

How Do Brands Make Senseof What They Hear?

• Search & Monitoring• Text Analytics• Full-Service Listening Platforms• Private Communities

Measuring whatyour customers say about youwhen they're talking to each

other.

LISTENING

Advantages (Online)

• Unobtrusiveness• Immediate / Real-time• Natural, rich, unfiltered WOM• BIG data

Disadvantages (Online)

• Ethics• Representativeness / Accuracy• WOM Noise• BIG data

Sentiment Analysis

• aka “opinion mining”• Measurement of emotion in texts– Polarity– Strength

• Human coding vs. NLP• Methodological standards / transparency

Project 2 Results

• Data set: You were provided with 200 Tweets related to pizza. (2 sets)

• Code each Tweet as – Positive, Negative, Mixed, or Neutral.

• When coded as Positive, Negative, or Mixed, identify the portion of the Tweet that resulted in that decision.

• Evaluate the difficulty of the coding decision.

Natural Language Processing

• SocialRadar vs. SentiStrength

• Observed agreement = .315 – Both data sets

• Why would computing kappa be inappropriate in this situation?

OA kappaSentistrength 0.680 0.502Sentistrength 0.600 0.424Sentistrength 0.585 0.374Sentistrength 0.510 0.314Sentistrength 0.500 0.309Sentistrength 0.485 0.307Sentistrength 0.415 0.150

Social Radar 0.460 0.211Social Radar 0.450 0.151Social Radar 0.445 0.203Social Radar 0.400 0.207Social Radar 0.380 0.184Social Radar 0.365 0.188Social Radar 0.320 0.172

“After coding these tweets, it is easy to see why computers might

not be the most effective way for a brand or company to decipher

customers’ tweets about a product or service.”

“I have come to admire people who are professional coders.”

But are humans better?

OA kappa0.660 0.4870.555 0.2950.525 0.310

0.810 0.7120.700 0.5560.670 0.5190.665 0.5130.665 0.5260.665 0.512

Difficulty correlations

0.3970.3810.3580.3440.3380.2380.2290.1600.078