Powering forward. Together.

December 2014

A Sentiment-al Journey: Text and Sentiment Analysis of Online

Discussion Responses

Susan Corbelli and Anya Suneson

Powering forward. Together.

Home: Sacramento Municipal Utility District

Electric Utility

Located in Northern California

Have about 610,000 Customers– 540,000 Residential– 70,000 Commercial

Covers territory:• Sacramento County• Part of Placer County

Why an electric utility does customer research?

• Monitor existing programs• Develop new programs• Improve customer experience

Traditional methods:• Telephone, mail, online surveys• Focus groups

Now venturing out into new methods…

Travel Bucket List

“Bucket list” of Innovative Research Methods

Ethnography

Usability testing

Online Research Community

Discussion boards

Text analytics

The “Vacation Home” – Online Research Communities

• Spent lots of time researching and planning

• Big investment

• Use regularly

• Requires regular maintenance

• Lots of people want to use

SMUD’s Online Research Community

The “Time Share” – Discussion Boards

• Use occasionally

• Short term use

• No long term commitment

• Great way to try something different!

Case Study: Analysis of Online Discussion Board Responses

Research Objectives and Methodology

• Objective: Determine customer energy-related needs and ways SMUD can address them

• Data Collection Method: Online Discussion Boards• Invitation sent out to 386 SMUD Plugged In participants• Online discussion was open for 2 weeks• Facilitated by SMUD research specialist• About 100 respondents participated in discussion

Online Discussion Board Example

Positive Customer Feedback

I appreciate the opportunity to be able to share my opinions and ideas with SMUD.

Since I consider SMUD to be a highly ethical company, I would like to be a participant in helping it flourish, in any way I can.

I really love that my opinion matters to you.

Tons of love for SMUD. That is all =)

I appreciate the opportunity to give feedback and voice my concerns, as well as suggestions for future development of my energy provider.

Text Analysis Tools Utilized

• Traditional Verbatim Coding• Word clouds• Semantic score• Sentiment Analysis

Text Analysis: Traditional Coding

• Pros: Human coder gets nuances machine can’t• Cons: Labor- and time-intensive

Provide unbiased energy advise

Household energy efficiency, more ways to save

Offer discounts to disadvantaged groups

Use more solar, renewable energy

Provide safe, reliable power

Keep energy affordable, keep bill low

SMUD meets all needs already

0% 10% 20% 30% 40% 50%

7%

10%

13%

23%

25%

41%

42%

The “Date Night” – Word Clouds

• Inexpensive

• Don’t require much planning or preparation

• Quick look

• Fun

Text Analysis: Word Clouds

• Pros: Quick, easy, free, visually-appealing

• Cons: Not sophisticated, word-level and not phrase-level

The “Day in Port” – Sentiment Analysis

• Heard “buzz” about SA• Cost ranges from free to very expensive• Quick look• But soon realized need more time in this “destination”

What is Sentiment Analysis?

• Also called Opinion Mining

• Is a systematic analysis of online expressions. Focuses on evaluating attitudes and opinions on a topic of interest using machine learning techniques (Rombocas)

• Classifies words, text, documents according to the opinion, emotion or sentiment that they express

• Determines:

– Subjective vs. Objective polarity

– Positive vs. Negative polarity

– Strength of the opinion

Benefits of Sentiment Analysis

• Can analyze large sets of data quickly

• Can use on many different sources of unstructured data

• Measures emotions which are important for marketing

• Data less influenced by researcher

• Less subjective than human coding

• Allows real-time analysis

Text Analysis: Sentiment Score

• Tool: Python NLTK

• Pros: Easy to use, fast, free

• Cons: Harder to explain, less actionable, further analysis requires Python programming knowledge

Text Analysis: Semantic Analysis

• Tool: Semantria

• Pros: Sophisticated, can churn big volumes, combines sentiment score

and phrase-level categorization

• Cons: Costly, takes time to learn

Semantic Analysis

• Facets are the most important ideas with their accompanying attributes. Facets rely on Subject Verb Object (SVO) parsing to find trends even when there are weak or no noun phrases in your text.

• Affordable energy, reliable energy, fixed income, low income and solar power were some of the facets identified in our text

• Discussion boards were a great way to have more of a conversation with our customers

• Text analysis methods can help save time and offer new insights

• Sentiment analysis appears to be useful for finding themes and important ideas in verbatims – especially for certain types of questions

• Need more time to better understand and utilize Sentiment Analysis and available tools– May need to learn a new language?– Maybe hire a guide?– Plan to visit again!

What did we learn on our journey?

Questions?

Thank you!

Resources

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts Bo Pang and Lillian Lee

http://www.cs.cornell.edu/home/llee/papers/cutsent.pdf

LingPipe Sentiment Tutorial

http://alias-i.com/lingpipe/demos/tutorial/sentiment/read-me.html

Professor Dan Jurafsky & Chris Manning are offering a free online course on Natural Language Processing.

http://www.nlp-class.org/

https://www.coursera.org/course/textanalytics

Resources

• http://www.online-utility.org/text/analyzer.jsp

Free software utility which allows you to find the most frequent phrases and frequencies of words. Non-English

language texts are supported. It also counts number of words, characters, sentences and syllables. Also calculates

lexical density.

• Text Mining package in R: http://cran.r-project.org/web/packages/tm/vignettes/tm.pdf

• Data Mining (including Text) Examples:

http://cran.r-project.org/doc/contrib/Zhao_R_and_data_mining.pdf

• MAXQDA, free 30-day trial available at http://www.maxqda.com/downloads/demo

• UCINET, free 30-day trial available at http://www.analytictech.com/downloaduc6.htm

Resources - Word Cloud Tools

• www.wordle.net

TIP: Double click on word to remove, can use “~” to combine words, customize color

• www.tagcrowd.com

TIP: Does stemming

• www.tagxedo.com

• TIP: Does stemming, many shapes

• https://tagul.com/

Free Sentiment Analysis Tools

1. Twitrratr – www.twitrratr.com

2. Sentiment 140 - http://www.sentiment140.com

3. Tweetfeel – www.tweetfeel.com

4. Opinmind – www.opinmind.com

5. Social Mention – www.socialmention.com