New Directions in

Social MediaTom Reamy

Chief Knowledge ArchitectKAPS Group

http://www.kapsgroup.com

2

Agenda Introduction – Social Media and Text Analytics

Deeper than Positive-Negative Building a Foundation for Social Media

– Adding intelligence to BI, CI, and Sentiment New Dimensions for Social Media

– New taxonomies– Cognitive Science of Emotion

Applications– Expertise Analysis, Behavior Prediction, Crowd Sourcing– Integration with Predictive Analytics, Social Analytics

Conclusions

3

KAPS Group: General Knowledge Architecture Professional Services – Network of Consultants Partners – SAS, SAP, IBM, FAST, Smart Logic, Concept Searching

– Attensity, Clarabridge, Lexalytics, Strategy – IM & KM - Text Analytics, Social Media, Integration Services:

– Taxonomy/Text Analytics development, consulting, customization– Text Analytics Fast Start – Audit, Evaluation, Pilot– Social Media: Text based applications – design & development

Clients: – Genentech, Novartis, Northwestern Mutual Life, Financial Times,

Hyatt, Home Depot, Harvard Business Library, British Parliament, Battelle, Amdocs, FDA, GAO, World Bank, etc.

Applied Theory – Faceted taxonomies, complexity theory, natural categories, emotion taxonomies

Presentations, Articles, White Papers – http://www.kapsgroup.com

4

Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment Beyond Good and Evil (positive and negative)

– Social Media is approaching next stage (growing up)– Where is the value? How get better results?

Importance of Context – around positive and negative words– Rhetorical reversals – “I was expecting to love it”– Issues of sarcasm, (“Really Great Product”), slanguage

Granularity of Application– Early Categorization – Politics or Sports

Limited value of Positive and Negative– Degrees of intensity, complexity of emotions and documents

Addition of focus on behaviors – why someone calls a support center – and likely outcomes

5

Introduction – Social Media & Text AnalyticsBeyond Simple Sentiment Two basic approaches:

– Statistical Signature of Bag of Words – Dictionary of positive & negative words

Beware automatic solutions – Accuracy, Depth Essential – need full categorization and concept extraction to get

full value from social media Categorization - Adds intelligence to all other components –

extraction, sentiment, and beyond Categorization/extraction rules – not just topical or sentiment

Combination with advanced social media analysis Opens up whole new worlds of applications

6

Text Analytics Foundation for Social MediaText Analytics Features Noun Phrase Extraction

– Catalogs with variants, rule based dynamic Sentiment Analysis

– Objects and phrases – statistics & rules – Positive and Negative Auto-categorization

– Training sets, Terms, Semantic Networks– Rules: Boolean - AND, OR, NOT– Advanced – DIST(#), ORDDIST#, PARAGRAPH, SENTENCE

Text Analytics as Foundation– Disambiguation - Identification of objects, events, context– Build rules based, not simply Bag of Individual Words

Case Study – Categorization & Sentiment

7

Case Study – Categorization & Sentiment

8

9

New Dimensions for Social Media New Taxonomies – Appraisal

– Appraisal Groups – Adjective and modifiers – “not very good”– Four types – Attitude, Orientation, Graduation, Polarity– Supports more subtle distinctions than positive or negative

Emotion taxonomies – Joy, Sadness, Fear, Anger, Surprise, Disgust– New Complex – pride, shame, embarrassment, love, awe– New situational/transient – confusion, concentration, skepticism

Beyond Keywords– Analysis of phrases, multiple contexts – conditionals, oblique – Analysis of conversations – dynamic of exchange, private language

10

New Applications in Social Media Expertise Analysis

– Experts think & write differently – process, chunks– Categorization rules for documents, authors, communities

Applications:– Business & Customer intelligence, Voice of the Customer– Deeper understanding of communities, customers – better models– Security, threat detection – behavior prediction, Are they experts?– Expertise location- Generate automatic expertise characterization

Behavior Prediction–TA and Predictive Analytics, Social Analytics Crowd Sourcing – technical support to Wiki’s Political – conservative and liberal minds/texts

– Disgust, shame, cooperation, openness

11

12

New Applications in Social Media Analysis of Conversations

– Techniques: self-revelation, humor, sharing of secrets, establishment of informal agreements, private language

– Detect relationships among speakers and changes over time– Strength of social ties, informal hierarchies

Combination with other techniques– Expertise Analysis – plus Influencers– Quality of communication (strength of social ties, extent of private

language, amount and nature of epistemic emotions – confusion+) Experiments - Pronoun Analysis – personality types Essay Evaluation Software - Apply to expertise characterization

– Model levels of chunking, procedure words over content

13

New Applications in Social MediaBehavior Prediction – Telecom Customer Service Problem – distinguish customers likely to cancel from mere threats Analyze customer support notes General issues – creative spelling, second hand reports Develop categorization rules

– First – distinguish cancellation calls – not simple– Second - distinguish cancel what – one line or all– Third – distinguish real threats

14

New Applications in Social MediaBehavior Prediction – Telecom Customer Service

Basic Rule– (START_20, (AND,  – (DIST_7,"[cancel]", "[cancel-what-cust]"),– (NOT,(DIST_10, "[cancel]", (OR, "[one-line]", "[restore]", “[if]”)))))

Examples:– customer called to say he will cancell his account if the does not stop receiving

a call from the ad agency. – cci and is upset that he has the asl charge and wants it off or her is going to

cancel his act– ask about the contract expiration date as she wanted to cxl teh acct

Combine sophisticated rules with sentiment statistical training and Predictive Analytics and behavior monitoring

15

New Applications: Wisdom of CrowdsCrowd Sourcing Technical Support Example – Android User Forum Develop a taxonomy of products, features, problem areas Develop Categorization Rules:

– “I use the SDK method and it isn't to bad a all. I'll get some pics up later, I am still trying to get the time to update from fresh 1.0 to 1.1.”

– Find product & feature – forum structure– Find problem areas in response, nearby text for solution

Automatic – simply expose lists of “solutions”– Search Based application

Human mediated – experts scan and clean up solutions

16

New Directions in Social MediaText Analytics, Text Mining, Semantic Technology Two Systems of the Brain

– Fast, System 1, Immediate patterns (TM)– Slow, System 2, Conceptual, reasoning (TA)

Text Analytics – pre-processing for TM, ST– Discover additional structure in unstructured text– Behavior Prediction – adding depth in individual documents – New variables for Predictive Analytics, Social Media Analytics– New dimensions – 90% of information – converting text to data

Text Mining for TA– Semi-automated taxonomy development – Bottom Up- terms in documents – frequency, date, clustering– Improve speed and quality – semi-automatic

17

New Directions in Social MediaConclusions Social Media Analysis requires a hybrid approach

– Software, text analytics, human judgment – Contexts are essential

Text Analytics needs new techniques and structures – Smaller, more dynamic taxonomies– Focus – verbs, adjectives, broader contexts, activity

Value from Social Media Analysis requires new structures– Appraisal taxonomies, emotion taxonomies Plus– Better models of documents and authors – multi-dimensional

Result:– Enhanced sentiment analysis / social media applications– Develop whole range of new applications– Combine with semantic technology, predictive analytics

Questions? Tom Reamy

tomr@kapsgroup.comKAPS Group

http://www.kapsgroup.comUpcoming: Text Analytics World – Boston, October 3-4

SAS A2012 – Las Vegas, Oct 8-9Taxonomy Boot Camp – Washington DC, Oct 16-17

Gilbane – Boston, November 27-29

19

New Applications in Social Media Two tasks – emotion detection (distinguish emotion

words ), and emotion classification Get references from this article Primary and complex emotions Many states that people call emotion are not in standard

taxonomy – ex. Love, confusion, concentration, worry, flirtatiousness, skepticism, indifference

Analyzing political trends – augment opinion polls Identifying political bias

20

New Applications in Social Media IDEA – TA needs to grow too – not just nouns, but verbs,

adjectives and adverbs, phrases, etc.  Text to Emotion Engine  Uses basic emotion taxonomy – happy, sad, fear, surprise,

anger, disgust Identify conditional sentences – “I’m happy when he is here” Emotion intensity – adjectives – very, eetc.

21

New Applications in Social Media “Most expressions of feeling can be more properly situated within

a context of exchange,” People use a variety of techniques to establish intimacy in

epistolary exchanges: Self-revelation, humor, sharing of secrets, establishment of

informal agreements, private language Can be applied to bodies of documents like corporate email

exchanges to determine such features: De facto hierarchy of an organization (Me – add expertise?),

strength or weakness of social ties, changing relationships among co-workers over time