marketing profs boston june 09

People talk, We Listen

Tales from the Trenches, How Organizations are Measuring Value in Social MediaA presentation for Marketing Profs B2BKatie Delahaye PaineCEOKDPaine & Partners [email protected], IPR Measurement Commissionwww.instituteforpr.org

mailto:[email protected]

http://www.kdpaine.com/


Why Measure?

“The main reason to measure objectives is not so much to reward or punish

individual communications manager for success or failure as it is to learn from the

research whether a program should be continued as is, revised, or dropped in favor of another approach ”

James E. Grunig, Professor Emeritus, University of Maryland “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”

Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis

2


Conquering your fears• If it’s not working, why

keep doing it?

I’m afraid that metrics will reveal that my program

isn’t working • If you’re deaf to the

conversation, only your enemies will hear it

I’m afraid of what I’ll hear

• It’s not about justifying, it’s about improving

I’m afraid I won’t be able to justify my

program/existence

• You should be fired for not showing any numbers

I’m afraid I’ll be fired for not showing the

right numbers

• Learn the language of business first, measurement will follow

I’m afraid to admit that I don’t know how to measure



MSM Online Social Media

Eyeball counti

ngHITS Engag

ement

A measurement timeline


You can’t talk when the marketing department is talking sales, leads and revenueImpressions are impossible to count in social media Who cares about impressions when you can measure brand engagement ?Who cares about reach when you can measure revenue? Wouldn’t you rather be closer to your customers? Better measurement tools are available

Old School PR 21st Century Role of PR

Social Media renders everything you know about measurement obsolete


Why Traditional Metrics (AVEs) are like buggy whips

They both confuse activity with outcomeThe goal is to arrive at your destination, not to have a faster horse

They both were based on a flawed premise

Neither PR people nor horses respond well to beatingsComparing PR to Advertising is like comparing the Surrey With a Fringe On Top to the Space Shuttle – they serve different purposes

They both were rendered obsolete by technology



12 Signs that it’s the end of measurement as we know it 1. The Dept of Defense considers Twittering and other

forms of social media critical to national security2. BMC Software measures communications

effectiveness based on contribution to EPS 3. BestBuy measures 85% lower turnover as a result of

its Blue Shirt community4. State Farm uses an internal blog to measurably

improve morale5. ASPCA correlates increases in on-line donations and

increased membership with its social media efforts6. HSUS generated $650,000 in new donations from an

on-line photo contest on Flickr7. NWF increased wildlife spotting as well as members

with its Twitter account 8. The Red Cross measures the effectiveness of Twitter

via lives saved and harm avoided9. ImmunizeBC measures success in terms of vaccines

given, awareness AND traffic 10. IBM receives more leads, sales and exposure from a

$500 podcast than it does from an ad11. 11 Mom’s turned around Walmart’s image and

delivered measureable increases in sales. 12. @comcastcares turned around Comcast’s customer

service reputation 13. A social media campaign for War Child delivered a

38% increase in donations and 300 new volunteers

http://twitter.com/comcastcares

http://twitter.com/comcastcares


The Engagement Decision Tree

Awareness

Consideration

Preference Trial Purchas

e

FindObserv

e/Lurk

Participate

Engagement

Purchase/Act/Link/WOM


The measurement fork in the road

Marketing/leads/sales

Reputation/relationships

To fix this Or get to this


Goals drive metrics, metrics drive results

10

Reputation/

RelationshipsRelation

ship scores

Recommend-

ations

Positioning

Engagement

Sales

Engagement IndexCost per

customer

acquisition

Web analytics

Sales leads

Marketing Mix

Modeling

Goal

Metrics


Listen, learn, change

2 4 4 4 2 26 5

2 4 2 2 22

8 85 9 9

9

24

16

27

10

20

15

4

2 13

2

4

30

5

2

12

16

17

0

10

20

30

40

50

60

Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

2007 2008

Positive

Neutral

Negative

Mentions

Tone of Conversation over time


Negative coverage over time

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr2006 2007 2008

0

5

10

15

20

25

21

2

5

21 1

4 42

1 12 2 2

1

10

9

4

1418

21

12

10

15

14

7

26

2

10

4

12

2

3 1

1

1

4

2

2

1

2

5

3

2

2

2

Entr

ies



Correlation exists between traffic to the ASPCA web site and the organization’s overall media

exposure

-

100,000

200,000

300,000

400,000

500,000

600,000

700,000

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Web

Sit

e Vi

sito

rs

Expo

sure

Overall Exposure

Web Traffic


Tying activity to development/marketing goals

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Exposure

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

$1,400,000

$1,600,000

$1,800,000

Donations

Overall exposure

Online donations

14


What do you need to measure?Impact

(Outputs/Outtakes)

ROI (Outcom

es)

Did audience behavior change?

Did the right people show up?

Did your relationship change?

Did sales or revenue or profits increase?

Did you get the exposure you wanted?

Were your messages communicated

Did your relationships improve?


Goals, Actions and Metrics Goal Action Output

Metric Outtake Metric

Outcome Metric

Recruitment

Twitter campaign

Number of tweets

% inclined to support% improvement in perception of organization

% increase in online requests for information % increase traffic to recruitment site

Increase in support

Flickr photo contest

Number of entrants

% likely to support the cause

Improvement in relationship scores

Message consistency

Blogger/Speaker training

% postings/articles containing one or more key messages

Share of positioning on key issues

% hearing message% believing message

% consistency in messaging between external and internal communicationsMarket shareRatio of on-message to off-message quotes

16


The 7 steps to Social Media ROI

1. Define the “R” – Define the expected results?

2. Define the “I” -- What’s the investment?

3. Understand your audiences and what motivates them

4. Define the metrics (what you want to become)

5. Determine what you are benchmarking against

6. Pick a tool and undertake research7. Analyze results and glean insight,

take action, measure again



Step 1: Define the “R”

What return is expected?

What will be different when the program has launched?

What were you hired to do?

If you are celebrating complete 100% success a year from now, what is different about the organization?

If your department was eliminated, what would be different?

18


Step 2: Define the “I”

What is the investment? PersonnelAgency compensationSenior Staff time Opportunity cost

19



Step 3: Define your audiences and how you impact them

There is no “audience.” There are multiple constituencies List every stakeholder

Where do they go for information? Don’t ask me, ask your customers. What’s important to them?What is the benefit of having a good relationship with that stakeholder group?

Understand your role in getting the audience to do what you want it to do

Raise awarenessIncrease preferenceIncrease engagement

20


Step 4: Define your Key Performance Indicators (KPIs) carefully because you become what you measure

Cost savingsEfficiency

Cost per message communicatedCost per new lead/customer acquired

Productivity: Increase in employee engagement/moraleLower turnover/recruitment costs

Engagement: Ratio of posts to comments% of repeat visitors% of 5+min visitors% of registrations

Trust:Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)

Thought leadership: Share of quotesShare of opportunities

Message penetrationPositioning on key issuesImprovement in favorable/unfavorable ratioImprovement in Optimal Content Score (OCS)

21


What makes a perfect communications KPI?

Gets you where you want to go (achieves corporate goals) Is actionable by individuals as well as departmentsContinuously improves your processes Is there when you need it


Why an Optimal Content Score?

You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive:

Mentions of the brandKey messagesPositioningVisibility

Negative OmittedNegative toneNo key message

23


How to calculate Optimal ContentQuality score +1 0 -1

Score Score ScoreTonality Positive 3 Neutral 0 Negative -3

Positioning Contains 2 Doesn't contain 0

Positions the competition favorably or positions Sargento negatively -2

Messaging Contains 3 partially contains 0

Does not contain or miscommunicates key message (neg mess) -1

Quotes Contains 1 Does not contain -1Competitive mention

Does not mention Competition 1

Competition mentioned prominently -3

Total Score 10 0 -10

Visibility Score+1 0 -1

Score Score Score

Brand Photo Contains 3 Doesn't contain 0Contains competitive photo -5

Dominance Focal point 3 Not a focal point -1Visibility Headline mention 2 Top -20 % of story 0 Minor mention -2Target publication Top Tier 2 2nd tier 0 Not on target list -2

Total Score 10 0 -10

Optimal Content Score


Standard classifications of discussion

• Acknowledging receipt of information

• Advertising something• Answering a question• Asking a question• Augmenting a previous

post• Calling for action• Disclosing personal

information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise• Giving a heads up

• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll• Validating a position


For all institutions, most postings were simply making an observation or distributing media.

3

6

1

1

7

36

1

29

5

15

14

2

16

1

2

12

7

2

6

2

24

787

3

2

203

12

12

46

11

1

3

2

1

4

1

4

3

6

2

1

13

2

2

1

13

2

6

18

4

1

1

5

35

3

17

2

8

9

1

1

1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing criticism

Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion

Making an observation

Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Conversation Types

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

44.2%

6.5%

30.9%

49.5%

100.0%

100.0%

100.0%

1.6%

53.9%

100.0%

26.9%

23.1%

10.8%

38.7%

72.7%

10.9%

15.5%

46.1%

66.6%

27.3%

35.1%

39.7%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing criticism

Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion

Making an observation

Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Engagement by Conversation Type - Institutional Blogs

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

cx


Past PerformanceThink 3

PeerUnderdog nipping at your heelsStretch goal

Whatever keeps the C-suite up at night

Step 5: Define your benchmarks

27


Internet dominated in 5 out of 6 months


Stanford dominates in broadcast coverage


Benchmarks put numbers in perspective

Twitter grade

Followers

Twitter rank

98.3

1295

43395

96.4

622

93718

99.99

15291

398

Shel Israel U of Penn Carnegie Mellon


MIT Berkeley see highest visibility in Social Media


Overview of Key Metrics

Bookmark.

Facebook

Ext. Blogs

Inst. Blogs

YouTube

MSM

SOV 2% — 8% 9% 11% 7%

Popularity

230 bkmks

500/mo.

—20

links150k views

—

Engagement

59 cmts

1 day13

cmts2-12 cmts

2 cmts —

% Positive

20% 32% 54% 50% 15% 15%

% Negative

0% 0% 4% 0% 1% 2%

Strat. Mess.

40%† 18%† 42% 42%† 18% 38%

Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.

† Small base size. Findings are directional only.


First: find out what already existsWeb analyticsCustomer Satisfaction dataCustomer loyalty data

Second: Decide what research is needed to give you the information you need:

Message content analysisRelationship surveys

Step 6: Conduct research (if necessary)

33


Step5: Selecting a measurement tool based on your KPIs

Objective Metric Tool

Increase inquiries, web traffic, recruitment

% increase in traffic#s of clickthrus or downloads

Google Analytics, Omniture

Increase awareness/preference

% of audience preferring your brand to the competition

Survey Monkey, Zoomerang, PollDaddy, Vizu

Engage marketplace Conversation index greater than .8Rankings

Type pad, Technorati , Xinureturns

Communicate messages

% of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages

Media content analysis –Dashboards

% aware of or believing in key message

Survey Monkey, Zoomerang,Vizu, PollDaddy

34


Your tool box needs:

1. A content source: Google News/Google BlogsTechnorati, Social Media Mention, Twazzup, Cyberalert, CustomScoop, e-WatchRadian 6, Techrigy, Visible TechnologiesRSS feedsSurvey Monkey/Zoomerang

35


Your tool box also needs to include: 2. A way to analyze that content

Automated vs. Manual Census vs random sampleThe 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisionsDashboards to aggregate data

Tools:• Woopra• Net promoter score

• Hubspot Grader

• Xinureturns• Twinfluence• SPSS• Excel• Crimson Hexagon

• www.tealium.com

36


Diversity dominates C-M discussions in Social Media


Your tool box also needs to include: 3. A way to measure

engagementThe conversation index=• Ratio of posts to

comments Relationship studiesThe engagement index

38



Managing engagement on your own property

% increase or decrease in unique visits In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email address

Courtesy of Eric Peterson


Share of conversation vs share of engagement

2

2

1

2

1

6

5

3

1

1

1

1

1

1

1

1

1

1

2

2

2

1

2

1

2

1

4

2

1

4

2

1

1

4

1

6

7

6

2

2

2

2

1

3

2

3

1

0 2 4 6 8 10 12 14 16 18 20

Faculty

Students

Research, Physical Sciences

Courses

Research, Earth Sciences

Projects, Non -Research

Financials

Alumni Topics

Research, Life Sciences

Staff

Admissions

Legal News

Other

Research, Agriculture

Policies

Institution, Overall

Campus Life

Research, Social Sciences

Share of Subject

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4

15.3%

68.7%

100.0%

4.4%

33.3%

96.8%

28.6%

34.9%

12.5%

43.3%

28.6%

13.0%

38.3%

100.0%

23.6%

66.7%

6.3%

28.6%

20.8%

2.3%

95.6%

33.2%

5.8%

28.6%

100.0%

86.8%

13.0%

31.0%

22.1%

3.2%

71.4%

43.5%

18.8%

94.2%

56.7%

14.2%

13.2%

53.2%

28.4%

21.1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Admissions

Alumni Topics

Campus Life

Community Relations

Courses

Events

Faculty

Financials

Institution, Overall

Inventions

Legal News

Other

Partnerships

Policies

Projects, Non - Research

Research, Agriculture

Research, Earth Sciences

Research, Life Sciences

Research, Other

Research, Physical Sciences

Research, Social Sciences

Staff

Students

Share of Engagement by Subject - ,External Blogs

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4


The vast majority of discussion in external blogs is neutral.

23

29

12

14

20

5

8

4

1

4

0

5

10

15

20

25

30

University of Michigan Purdue University Penn State Michigan State Arizona State

Share of Tone

Negative

Neutral

Positive

71%

3%

29%

94%

83%

42%

58%

6%

14%

58%

42%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Arizona State Michigan State Penn State Purdue University University of Michigan

Share of Engagement by Tone - External Blogs

Negative

Neutral

Positive


Aspects of relationships

Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship

42


Components of a Relationship IndexControl mutuality

In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.

TrustThis organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it will do.

SatisfactionGenerally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.

CommitmentThere is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization more

Exchange relationshipEven though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.

Communal relationshipThis organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other people. (Reversed)

43


How to implement relationship metrics

Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed


Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, 13-month averageFigure out what worked and what didn’t workMove resources from what isn’t working to what is

Step 7: Analysis - -Research without insight is just trivia

45


Thank You!

For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard:

www.themeasurementstandard.comFollow me on Twitter: KDPaineFind me on Facebook: Katie PaineReach me on Skype: KDPaine Or call me at 1-603-868-1550Check out our website www.kdpaine.com

http://kdpaine.blogs.com/

http://www.themeasurementstandard.com/

marketing profs boston june 09

Technology

pr people

traffic talk

kwom people

different purposes people

measurement fork

end of measurement

social media efforts

social media renders