Measuring What Matters In PR

A presentation to the Applied Public Relations and Public Affairs Research CourseGeorge Washington University  September 17, 2009Katie Delahaye PaineCEOkdpaine@kdpaine.comwww.kdpaine.comhttp:/kdpaine.blogs.comMember, IPR Measurement Commissionwww.instituteforpr.org

Why Measure?

“The main reason to measure objectives is not so much to reward or punish

individual communications manager for success or failure as it is to learn from the

research whether a program should be continued as is, revised, or dropped in favor of another approach ”

James E. Grunig, Professor Emeritus, University of Maryland “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”

Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis

What Matters?

To P&G: EngagementTo the Humane Society:

DonationsTo ComCast: Happier

customers To Best Buy: Better informed

employeesTo WMUR: Faster, more

complete, more relevant stories To Dell: SalesTo Molson: Better messaging

What Doesn’t Matter?

AVEsEyeballsHITS (How Idiots Track Success)Couch Potatoes# of Twitter Followers (unless you’re a celebrity)# of Facebook Friends/Fans (unless they donate money)

Page 4

Eyeball counti

ngHITS Engag

ement

MSM Online Social Media

A measurement timeline

Page 6

Old School 21st Century

You are a party planner, not a communicator

The definition of timely has changedThe definition of reach has changed

GRPs & Impressions are impossible to count (an irrevelvant) in social media

The definition of success has changedThe answer isn’t how many you’ve reached, but how those you’ve reached have responded Page 7

Old School PR 21st Century Role of PR

Social Media renders everything you know about measurement obsolete

Signs that it’s the end of measurement as we know it 1. Procter & Gamble is now paying for

engagement, not eyeballs 2. Sodexo cut $300K out of its recruitment

budget using Twitter3. Facebook USERS translated the site from

English to Spanish via a Wiki in less than 4 weeks and cost Facebook $0

4. BMC Software measures communications effectiveness based on contribution to EPS

5. HSUS generated $650,000 in new donations from an on-line photo contest on Flickr

6. The Red Cross measures the effectiveness of Twitter via lives saved and harm avoided

7. IBM 1000+ people tweeting & receives more leads, sales and exposure from a $500 podcast than it does from an ad

8. 11 Mom’s turned around Walmart’s image and delivered measureable increases in sales.

The New Rules of Communications

You aren’t in control and never have beenThere is no market for your message You become what you measureShe/he with the most data winsBehind every Tweet or Post is a person Empower employees, rely on customersEnable the conversations—it’s going on, with or without youSpin is dead, long live transparency – you can’t fake it so be who you are and see who is pleased Crowdsourcing will beat outsourcing every time

The Engagement Decision Tree

Awareness

Consideration

Preference Trial Purchas

e

FindObserv

e/Lurk

Participate

Engagement

Purchase/Act/Link/WOM

The measurement forks in the road

Marketing/leads/sales/mission

Reputation/relationships

To fix this Or get to this

Goals drive metrics, metrics drive results

12

Reputation/Relationships

Relationship scores

Recommendations

Positioning

Engagement

Get the word out

% hearing

% believing

% acting

Sales

Engagement Index

Cost per customer

acquisition

Web analytics

Sales leads

Marketing Mix Modeling

Goal

Metrics

Change the conversation, improve your reputation

Improve your reputation

Listen first, then respondStop doing stupid things

Negative coverage over time

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr2006 2007 2008

0

5

10

15

20

25

21

2

5

21 1

4 42

1 12 2 2

1

10

9

4

1418

21

12

10

15

14

7

26

2

10

4

12

2

3 1

1

1

4

2

2

1

2

5

3

2

2

2

Entr

ies

Correlation exists between traffic to the ASPCA web site and the organization’s

overall media exposure

-

100,000

200,000

300,000

400,000

500,000

600,000

700,000

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Web

Sit

e Vi

sito

rs

Expo

sure

Overall Exposure

Web Traffic

Tying activity to development/marketing goals

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Exposure

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

$1,400,000

$1,600,000

$1,800,000

Donations

Overall exposure

Online donations

16

What do you need to measure?Outputs?

Did you get the coverage you wanted?

Did you produce the promised materials on time and on budget?

Outtakes?Did your target audience see the messages?

Did they believe the messages?

Outcomes?Did audience behavior change?

Did the right people show up?

Did your relationship change?

Did sales increase?

Goals, Actions and Metrics Goal Action Output Metric Outtake

MetricOutcome Metric

Increased on-line reservations

Revamp website

Amount of content on web site

% perceiving state as a destination

% increase in web traffic and reservations

#1site for visitors to NH

Increase staffing and resources for communications

Increased exposure of “visit NH” message

Increased perception of NH as an an extreme destination

% increase in agreement with the statement

Website is preferred site for information

Add content, features to web site, keep up to date

% increase in traffic

% agreeing with the statement

# 1 rankings, and time spent on site

The 7 steps to Social Media ROI

1. Define the “R” – Define the expected results?

2. Define the “I” -- What’s the investment?

3. Understand your audiences and what motivates them

4. Define the metrics (what you want to become)

5. Determine what you are benchmarking against

6. Pick a tool and undertake research7. Analyze results and glean insight,

take action, measure again

Step 1: Define the “R”

What return is expected?

What were you hired to do?

If you are celebrating complete 100% success a year from now, what is different about the organization?

If your department was eliminated, what would be different?

20

Step 2: Define the “I”

What is the investment? PersonnelAgency compensationSenior Staff time Opportunity costRaw costs/hr costs vs material costs.

21

Step 3: Define your audiences and how you impact them

There is no “audience.” There are multiple constituencies Should you blog or Twitter? Don’t ask me, ask your customers List every stakeholder

Where do they go for information?What’s important to them?What is the benefit of having a good relationship with that stakeholder group?What’s important to them?Where do they go for information?What do you want them to know?

Understand your role in getting the audience to do what you want it to do

Raise awarenessIncrease preferenceIncrease engagement

22

Step 4: Define your Key Performance Indicators (KPIs)

23

The Perfect KPIGets you where you want to go (achieves corporate goals)Is actionableContinuously improves your processesIs there when you need it

KPIs should be developed for: Your own propertiesDifferent tacticsOther influential sites

Revenue KPIs

Cost savingsCost per click thru, downloads, engagement vs other marketing channelsCost per message communicated vs other channels

Lifetime value of engagementCost per customer acquisition

Engagement metrics

% increase or decrease in unique visits In the past  month,  what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email address

Courtesy of Eric Peterson

KPIs for External blogs and other Consumer Generated Media

Share of positioningShare of rants vs. ravesShare of positives/negativesShare of visibilityShare of quotesShare of brand benefits

mentionedTypes of conversationsOptimal content score

Emerging benchmarks Past PerformanceThink 3

PeerUnderdog nipping at your heelsStretch goal

Whatever keeps the C-suite up at night

Step 5: Define your benchmarks

27

First: find out what already existsWeb analyticsCustomer Satisfaction dataCustomer loyalty data

Second: Decide what research is needed to give you the information you need

Step 5: Conduct research (if necessary)

Step 6: Selecting a measurement tool

Objective KPI Tool

Increase inquiries, web traffic, recruitment

% increase in traffic#s of clickthrus or downloads

Google Analytics, Omniture, Web trends

Increase awareness/preference

% of audience preferring your brand to the competition

SurveyMonkey, Zoomerang

Engage marketplace Conversation index greater than .8Rankings % increase in engagement

TypePad, Technorati Omniture, Google Analytics

Communicate messages

% of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages

Media content analysis –Dashboards

% aware of or believing in key message

Survey

29

Your tool box needs:

1. A content source: Google News/Google Blogs, RSS feedsTechnorati, Social Mention, Twazzup, Cyberalert, CustomScoop, e-WatchRadian 6, Techrigy, Visible Technologies, Scout LabsSurvey Monkey/Zoomerang

30

Your tool box also needs to include: 2. A way to analyze that content

Automated vs. Manual Census vs random sampleThe 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisionsDashboards to aggregate data

Tools:• Woopra• Net promoter score

• Hubspot Grader

• Xinureturns• Twinfluence• SPSS• Excel• Crimson Hexagon

• www.tealium.com

31

Why an Optimal Content Score?

You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive:

Mentions of the brandKey messagesPositioningVisibility

Negative OmittedNegative toneNo key message

32

How to calculate Optimal ContentQuality score +1 0 -1

Score Score ScoreTonality Positive 3 Neutral 0 Negative -3

Positioning Contains 2 Doesn't contain 0

Positions the competition favorably or positions Sargento negatively -2

Messaging Contains 3 partially contains 0

Does not contain or miscommunicates key message (neg mess) -1

Quotes Contains 1 Does not contain -1Competitive mention

Does not mention Competition 1

Competition mentioned prominently -3

Total Score 10 0 -10

Visibility Score+1 0 -1

Score Score Score

Brand Photo Contains 3 Doesn't contain 0Contains competitive photo -5

Dominance Focal point 3 Not a focal point -1Visibility Headline mention 2 Top -20 % of story 0 Minor mention -2Target publication Top Tier 2 2nd tier 0 Not on target list -2

Total Score 10 0 -10

Optimal Content Score

Standard classifications of discussion

• Acknowledging receipt of information

• Advertising something• Answering a question• Asking a question• Augmenting a previous

post• Calling for action• Disclosing personal

information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise• Giving a heads up

• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll• Validating a position

Standard classifications of videos

AdvertisementAnimationDemonstrationEvent/PerformanceFictionFilmHome VideoInstructional VideoInterviewLecture

MontageMusic VideoNews BroadcastPromotional VideoSightseeing/TourSlideshowSpeechTelevision ShowVideo Log

Your tool box also needs to include: 3. A way to measure

engagementThe conversation index=• Ratio of posts to

comments Relationship studiesThe engagement index

36

For all institutions, most postings were simply making an observation or distributing media.

Page 37

3

6

1

1

7

36

1

29

5

15

14

2

16

1

2

12

7

2

6

2

24

787

3

2

203

12

12

46

11

1

3

2

1

4

1

4

3

6

2

1

13

2

2

1

13

2

6

18

4

1

1

5

35

3

17

2

8

9

1

1

1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing criticism

Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion

Making an observation

Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Conversation Types

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

44.2%

6.5%

30.9%

49.5%

100.0%

100.0%

100.0%

1.6%

53.9%

100.0%

26.9%

23.1%

10.8%

38.7%

72.7%

10.9%

15.5%

46.1%

66.6%

27.3%

35.1%

39.7%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing criticism

Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion

Making an observation

Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Engagement by Conversation Type - Institutional Blogs

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

cx

Share of conversation vs share of engagement

Page 38

2

2

1

2

1

6

5

3

1

1

1

1

1

1

1

1

1

1

2

2

2

1

2

1

2

1

4

2

1

4

2

1

1

4

1

6

7

6

2

2

2

2

1

3

2

3

1

0 2 4 6 8 10 12 14 16 18 20

Faculty

Students

Research, Physical Sciences

Courses

Research, Earth Sciences

Projects, Non -Research

Financials

Alumni Topics

Research, Life Sciences

Staff

Admissions

Legal News

Other

Research, Agriculture

Policies

Institution, Overall

Campus Life

Research, Social Sciences

Share of Subject

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4

15.3%

68.7%

100.0%

4.4%

33.3%

96.8%

28.6%

34.9%

12.5%

43.3%

28.6%

13.0%

38.3%

100.0%

23.6%

66.7%

6.3%

28.6%

20.8%

2.3%

95.6%

33.2%

5.8%

28.6%

100.0%

86.8%

13.0%

31.0%

22.1%

3.2%

71.4%

43.5%

18.8%

94.2%

56.7%

14.2%

13.2%

53.2%

28.4%

21.1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Admissions

Alumni Topics

Campus Life

Community Relations

Courses

Events

Faculty

Financials

Institution, Overall

Inventions

Legal News

Other

Partnerships

Policies

Projects, Non - Research

Research, Agriculture

Research, Earth Sciences

Research, Life Sciences

Research, Other

Research, Physical Sciences

Research, Social Sciences

Staff

Students

Share of Engagement by Subject - ,External Blogs

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4

The vast majority of discussion in external blogs is neutral.

Page 39

23

29

12

14

20

5

8

4

1

4

0

5

10

15

20

25

30

University of Michigan Purdue University Penn State Michigan State Arizona State

Share of Tone

Negative

Neutral

Positive

71%

3%

29%

94%

83%

42%

58%

6%

14%

58%

42%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Arizona State Michigan State Penn State Purdue University University of Michigan

Share of Engagement by Tone - External Blogs

Negative

Neutral

Positive

Aspects of relationships

Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship

40

Components of a Relationship IndexControl mutuality

In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.

TrustThis organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it will do.

SatisfactionGenerally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.

CommitmentThere is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization more

Exchange relationshipEven though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.

Communal relationshipThis organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other people. (Reversed)

How to implement relationship metrics

Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed

Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, 13-month averageFigure out what worked and what didn’t workMove resources from what isn’t working to what is

Step 7: Analysis - -Research without insight is just trivia

43

Best Practices:

Correlations to bottom-line impact

DonationsMembershipsSign-upsLeads

Using SMM for planning

Define the time frame, market/topic you want to studyUse Google News, Technorati or Radian6 to identify the conversations around the topic Analyze the conversations for type, tone and positioningLook at share of positioning, tone or conversation

Benchmarking against your peers

Looking at what the best doSetting goals accordinglyUse data to persuade recalcitrant spokespeople

Social Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product

Improve your reputation

Listen first, then respondStop doing stupid things

The competitive landscape

Technorati mentions with high authority

Cingular7%

Sprint7%

Verizon10%

T-Mobile75%

US Cellular1%

Company "sucks" mentions in Technorati with high authority

US Cellular2% Cingular

16%

Sprint12%

Verizon19%

T-Mobile51%

Using SMM for planning

The environmental scanDefining issues in a marketSelecting a positioning that works

Percent of impressions containing messages by product

0% 20% 40% 60% 80% 100%

TAC

Manuscript

One Source

HAL

Positive Messages No Messages Negative Messages

Measuring the impact of messaging

47

$0.00 $0.50 $1.00 $1.50 $2.00 $2.50

Pressconference

Event/party

Press tour

Event/presstour

Metric: Cost per message communicated

The press tour was clearly the most efficient for communicating key messages and the big party was least efficient.

Measuring which tactic was most efficient

48

Lesson learned, you need the PR department

Share of exposure vs the competition over time

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jan

MarMay Ju

lSep Nov Ja

nMar

May Jul

Sep Nov Jan

Intel

TI

Moto

National

49

Facebook: Correlating MSM, CGM and signups

Strong correlationNon-negative discussion only

0

100

200

300

400

500

600

700

800

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

7,000,000

8,000,000

9,000,000

10,000,000

Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov

2006 2007

User Registrations and Media CoverageDecember 2006-November 2007

New Accounts

FB Mentions

50

PR is shown to deliver more value

A major consumer company found that PR delivered 8 times the value of TV and 4 times the value of trade advertising. 0

1

2

3

4

5

6

7

8

Trade TV ad PR

$ return oninvestment

51

Proof of PR’s impact on sales

P&G found that PR drives sales

Three of the six products showed PR with the highest ROI of any marketing tactic

Overall PR delivered a 275% ROI

AT&T found that PR delivered customers at a fraction of the cost

95

63

15$0

$50

$100

Advertising Outboundtelemarketing

PR

Cost per customer acquisition

52

PR delivers more results for less money

Miller discovered that PR campaigns generate 4% of incremental sales compared to 17.3% of incremental sales for TV.

However, PR delivered that 4% for less than 1% of the budget.

01020304050607080

Trade TV PR

% of Spend vs % of incremental revenue

% of incrementalrevenue

% of spend

53

Overview of Key Metrics

Bookmark.

Facebook

Ext. Blogs

Inst. Blogs YouTube MSM

SOV 2% — 8% 9% 11% 7%

Popularity

230 bkmks

500/mo. — 20 links

150k views —

Engagement 59 cmts 1 day 13 cmts

2-12 cmts 2 cmts —

% Positive 20% 32% 54% 50% 15% 15%

% Negative 0% 0% 4% 0% 1% 2%

Strat. Mess. 40%† 18%† 42% 42%† 18% 38%

Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.

Rank Order

Facebook YouTube Social Bookmarking

External Blogs

Institutional Blogs

1 Campus Life

Events Courses Faculty Campus Life

2 Sports Campus Life

Projects, Non-Research

Research, Physical Sciences

Events

3 Technology Faculty Research, Physical Sciences

Institution Overall

Institution Overall

4 Product Services

Courses Events Expert Commentary

Institution Sub-Groups

5 Events Institution Overall

Faculty Events Admissions

Few subjects appear across all forms of social media, so tailor outreach accordingly

Thank You!

For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard:

www.themeasurementstandard.comFor a copy of this presentation

go to: http://www.kdpaine.comFollow me on Twitter: KDPaineFriend me on Facebook: Katie

Paine Or call me at 1-603-868-1550