wpaa measuring naked conversations
DESCRIPTION
TRANSCRIPT
People talk, We Listen
Measuring Naked Conversations
A presentation to Worldwide Public Affairs ConferenceCrystal City, VA
May, 2009 Katie Delahaye Paine
www.kdpaine.comkdpaine.blogs.com
Member, IPR Measurement Commissionwww.instituteforpr.org
People talk, We Listen
Why Measure?
“The main reason to measure objectives is not so much to reward or punish
individual communications manager for success or failure as it is to learn from the
research whether a program should be continued as is, revised, or dropped in favor of another approach ”
James E. Grunig, Professor Emeritus, University of Maryland “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”
Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis
2
People talk, We Listen
Conquering your fears
I’m afraid that metrics will reveal that my
program isn’t working •If it’s not working, why keep doing it?
I’m afraid of what I’ll hear
•If you’re deaf to the conversation, only your enemies will hear it
I’m afraid I won’t be able to justify my
program/existence
•It’s not about justifying, it’s about improving
I’m afraid I’ll be fired for not showing
the right numbers
•You should be fired for not showing any numbers
3
People talk, We Listen
Communications then and now
Traditional role of Marketing & Communications
21st Century Role of PR
People talk, We Listen
People talk, We Listen
Page 5
MSM
Online
Social
Media
Eyeball
counting
HITS
Engagement
A measurement timeline
People talk, We Listen
The changing “Holy Grail” of measurement
Engageme
nt
On your property
?
With your b
rand?
Relationsh
i
ps
With w
hat
audience?
Must be
competiti
ve?
ROI
AVE does not
equal ROI
ROI =Desir
ed
Return minus
Investment
6
People talk, We Listen
The New Rules of Communications
You aren’t in controlEmpower employees, rely on customersEnable the conversations—it’s going on, with or without youSpin is dead, long live transparency – you can’t fake it so be who you are and see who is pleased Crowdsourcing will beat outsourcing every time There is no market for your message You become what you measureShe/he with the most data winsSocial media must be a core competency for all employees
7
People talk, We Listen
The measurement fork in the road
Marketing/recruitment
Reputation/relationships
To fix this Or get to this
People talk, We Listen
Goals drive metrics, metrics drive results
9
Reputation/Relationships
Relationship scores
Recommend-ations
Positioning
Engagement
Goal
Metrics
People talk, We Listen
Changing reputation via metrics
2 4 4 4 2 26 5
2 4 2 2 22
8 85 9 9
9
24
16
27
10
20
15
4
2 13
2
4
30
5
2
12
16
17
0
10
20
30
40
50
60
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2007 2008
Positive
Neutral
Negative
Mentions
Tone of Conversation over time
People talk, We Listen
Negative coverage over time
Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr2006 2007 2008
0
5
10
15
20
25
21
2
5
21 1
4 42
1 12 2 2
1
10
9
4
1418
21
12
10
15
14
7
26
2
10
4
12
2
3 1
1
1
4
2
2
1
2
5
3
2
2
2
Entr
ies
People talk, We Listen
The 7 steps to Social Media
1. Define the “R” – Define the expected results?
2. Define the “I” -- What’s the investment?
3. Understand your audiences and what motivates them
4. Define the metrics (what you want to become)
5. Determine what you are benchmarking against
6. Pick a tool and undertake research7. Analyze results and glean insight,
take action, measure again
People talk, We Listen
People talk, We Listen
Page 13
Step 1: Define the “R”
What return is expected?
What were you hired to do?
If you are celebrating complete 100% success a year from now, what is different about the organization?
If your department was eliminated, what would be different?
13
People talk, We Listen
Step 2: Define the “I”
What is the investment? PersonnelAgency compensationSenior Staff time Opportunity cost
14
People talk, We Listen
People talk, We Listen
Page 15
Step 3: Define your audiences and how you impact them
You audience is never “anyone with a pulse” There are multiple constituencies List every stakeholder
Where do they go for information?What’s important to them?What is the benefit of having a good relationship with that stakeholder group?
Understand your role in getting the audience to do what you want it to do
Raise awarenessIncrease preferenceIncrease engagement15
People talk, We Listen
Step 4: Define your Key Performance Indicators (KPIs)
Cost savingsEfficiency
Cost per message communicatedCost per new lead/customer acquired
Productivity: Increase in employee engagement/moraleLower turnover/recruitment costs
Engagement: Ratio of posts to comments% of repeat visitors% of 5+min visitors% of registrations
Trust:Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)
Thought leadership: Share of quotesShare of opportunities
Message penetrationPositioning on key issuesImprovement in favorable/unfavorable ratioImprovement in Optimal Content Score (OCS)
16
People talk, We Listen
What makes a perfect communications KPI?
Gets you where you want to go (achieves corporate goals) Is actionable by individuals as well as departmentsContinuously improves your processes Is there when you need it
People talk, We Listen
Why an Optimal Content Score?
You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive:
Mentions of the brandKey messagesPositioningVisibility
Negative OmittedNegative toneNo key message
18
People talk, We Listen
How to calculate Optimal ContentQuality score +1 0 -1
Score Score ScoreTonality Positive 3 Neutral 0 Negative -3
Positioning Contains 2 Doesn't contain 0
Positions the competition favorably or positions Sargento negatively -2
Messaging Contains 3 partially contains 0
Does not contain or miscommunicates key message (neg mess) -1
Quotes Contains 1 Does not contain -1Competitive mention
Does not mention Competition 1
Competition mentioned prominently -3
Total Score 10 0 -10
Visibility Score+1 0 -1
Score Score Score
Brand Photo Contains 3 Doesn't contain 0Contains competitive photo -5
Dominance Focal point 3 Not a focal point -1Visibility Headline mention 2 Top -20 % of story 0 Minor mention -2Target publication Top Tier 2 2nd tier 0 Not on target list -2
Total Score 10 0 -10
Optimal Content Score
People talk, We Listen
Emerging benchmarksEngaged = 3-13 comments per postHyper-engaged = 15-35 comments per postAfter 3 days most comments are done, 14 days maxSocial Bookmarking momentum = 1 submitted item every other dayMessage should be communicated in 2 out of 5 blogs
Past PerformanceThink 3
PeerUnderdog nipping at your heelsStretch goal
Whatever keeps the C-suite up at night
Step 5: Define your benchmarks
20
People talk, We Listen
Overview of Key Metrics
Bookmark.
Ext. Blogs
Inst. Blogs
YouTube
MSM
SOV 2% — 8% 9% 11% 7%
Popularity
230 bkmks
500/mo.
—20
links150k views
—
Engagement
59 cmts
1 day13
cmts2-12 cmts
2 cmts —
% Positive
20% 32% 54% 50% 15% 15%
% Negative
0% 0% 4% 0% 1% 2%
Strat. Mess.
40%† 18%† 42% 42%† 18% 38%
Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.
† Small base size. Findings are directional only.
People talk, We Listen
Top 5 Subjects of discussion in each channel
Rank Order
Facebook YouTube Social Bookmarking
External Blogs
Institutional Blogs
1 Campus Life
Events Courses Faculty Campus Life
2 Sports Campus Life
Projects, Non-Research
Research, Physical Sciences
Events
3 Technology Faculty Research, Physical Sciences
Institution Overall
Institution Overall
4 Product Services
Courses Events Expert Commentary
Institution Sub-Groups
5 Events Institution Overall
Faculty Events Admissions
Few subjects appear across all forms of social media, so tailor outreach accordingly
People talk, We Listen
First: find out what already exists
Web analyticsSurvey data Third party data
Then figure out what is missing and what research you need
Step 5: Conduct research (if necessary)
23
People talk, We Listen
People talk, We Listen
Page 24
What do you need to measure (cont.)
Output Outtake Outcome
+ +Unique monthly visitorsTime on site, repeat visitsSNS connectionsRSS subscribersForwards/ linksComments/ postCommunity descriptivesUse of discussion featuresMass media influence
Relationship measuresReputation dimensionsAttitudeAwarenessMessages communicated by stakeholdersMeme creation
Click-through rateDonations/ ordersStock priceSignups / membershipSupport callsEvent participationVotesQualified applicantsRetention
People talk, We Listen
Step5: Selecting a measurement tool based on your KPIs
Objective Metric Tool
Increase inquiries, web traffic, recruitment
% increase in traffic#s of clickthrus or downloads
Clicktrax, Web trends, WebSide Story
Increase awareness/preference
% of audience preferring your brand to the competition
Survey Monkey, Zoomerang,
Engage marketplace Conversation index greater than .8Rankings
Type pad, Technorati
Communicate messages
% of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages
Media content analysis –Dashboards
% aware of or believing in key message
Survey Monkey, Zoomerang,Vizu 25
People talk, We Listen
Your tool box needs:
1. A content source: Google AlertsTechnorati, Ice RocketCyberalert, CustomScoop, e-WatchRadian 6, TechrigyRSS feedsSocial Mention eNR, MeltwaterSurvey Monkey/Zoomerang
26
People talk, We Listen
Your tool box also needs to include: 2. A way to analyze that content
Automated vs. Manual Census vs random sampleThe 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisionsDashboards aggregate data
Tools:• Woopra• Hubspot Grader• Xinureturns• Twinfluence• SPSS• Excel• Crimson
Hexagon• www.tealium.co
m27
People talk, We Listen
Standard classifications of discussion• Acknowledging receipt of
information• Advertising something• Answering a question• Asking a question• Augmenting a previous
post• Calling for action• Disclosing personal
information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise• Giving a heads up
• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll• Validating a position
People talk, We Listen
Standard classifications of videos
AdvertisementAnimationDemonstrationEvent/PerformanceFictionFilmHome VideoInstructional VideoInterviewLecture
MontageMusic VideoNews BroadcastPromotional VideoSightseeing/TourSlideshowSpeechTelevision ShowVideo Log
People talk, We Listen
People talk, We Listen
Page 30
Measuring conversations on social networking sites
To record Total connections/
members Stakeholder
classification Active connections Energy of
discussion Topicality of
discussion Signals of
reputation or relationship
Uses of discussion features
Size of threads Transmission of
logos, information and links
Questions and feedback
Key metricsGrowth of network,
membershipComposition of active and
passive connections by stakeholder group
Top uses of discussion features
Discussion per minute/hour/day
Key information transmitted, re-transmitted
Number of questions answered
Amount of feedback, suggestions received
Alignment of profile or group with identity
Illuminating cases of fulfillment of purpose
People talk, We Listen
People talk, We Listen
Page 31
Measuring conversations on video sharing sites
To record Alignment of video
with brand (tone, messaging, benefits)
Type of user posting video
Views Ratings Number of times
embedded Types of sites
embedding Total comments Number of users
commenting Use of comments Comments that
become threaded Size of threaded
comments
Key metricsType, tone, messaging of
videoOwnership of video,
classified by stakeholder group, organization
Total views vs. comments, ratings and links
Median/ mean comments per video
Distance between post and first comment
Shows of support, feedback received
Classification of organizational subjects by average comments, average tone and average views
Re-transmission of videos on other channels
People talk, We Listen
Your tool box needs to include:
4. A way to quantify it all
HITS= How Idiots Track SuccessEyeballs – CompeteGoogle AnalyticsPanelsSurveys
32
People talk, We Listen
What works, what doesn’t?What needs to be done? What are you communicating?What tools work best?Connect the results to what you already
knowLink your data to how your organization
understands the world—stakeholder segmentations, decision-making models, and branding
Step 7: Analysis – Research without insight is just trivia
33
People talk, We Listen
Research without insight is just triviaWhat works, what doesn’t?What needs to be done? What are you communicating?Which tools work best?
Connect the results to what you already know
Link your data to how your organization understands the world—stakeholder segmentations, decision-making models, and branding
Step 6: Analysis
B Cbecause you know if B then C
AYou’ve measured Anow connect it to B
People talk, We Listen
Data mining the numbers you have
Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, last yearFigure out what worked and what didn’t work B C
Because you know if B then C
AYou’ve measured ANow connect it to B
People talk, We Listen
Best Practices:
Correlations to bottom-line impact
DonationsMembershipsSign-upsLeads
Using SMM for planning
Define the time frame, market/topic you want to studyUse Google News, Technorati or Radian6 to identify the conversations around the topic Analyze the conversations for type, tone and positioningLook at share of positioning, tone or conversation
Benchmarking against your peers
Looking at what the best doSetting goals accordinglyUse data to persuade recalcitrant spokespeople
Social Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product
Improve your reputation
Listen first, then respondStop doing stupid things
People talk, We Listen
Thank You!
For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard: www.themeasurementstandard.comFor a copy of this presentation go to: http://www.kdpaine.comOr call me at 1-603-868-1550Or email me at [email protected]
Where to find me:
Follow me on Twitter: @kdpaineFriend me on Facebook: Katie PaineSkype: KDPaineLinked In: Katie Delahaye PaineFlickr: kdpaineandpartnersYouTube: kdpaineandpartners
37