why man is mightier than the machine - jimdo

Why Man is Mightier than the Machine

Feature-Based Social Media Opinion Mining using traditional research techniques

- A Hospitality Case Study -

www.kinsightanalytics.com

Extending Traditional Market Research & Analytics

Traditional MR techniques are increasingly applicable to a wide variety of analytics problems

One of the areas where we see this applicability is in the area of social media analytics

Analyzing the wealth of customer feedback via social media is becoming a n engaging MR problem


Hospitality Industry Trends

• In the past, hotel choice decisions were guided by travel agents or friends & family

• Now, dramatic changes in information availability

– Audience is much wider and less controlled

– Traveler options for hotel recommendations & reviews have increased exponentially

– User-Generated Content on a variety of forums - review sites and social networks like Facebook and Twitter significantly influence customer decisions


The Hospitality Context

• Hotel marketers now need to track online conversations about their property so as to:

– Understand Traveler preferences

– Influence traveler choice decisions

– Address pain points and highlight advantages of their properties

• Key brand messaging is no longer just the marketer’s prerogative – consumer has a voice, and marketer has to listen


Hospitality players

struggle to cater to local

needs

Business Intelligence

reports from corporate lack

actionable data for local

properties

Turnaround time on data requirements takes too long for property Owners/GMs

Frustration for Owners/GMs due to lack of

corporate support for

their need to grow the business

Hospitality - Demand for Actionable Social Media Analytics Data


Analytics Objectives for Hotel Properties

• To understand where a hotel property stands in comparison to its geographically closest competitors – To address property and process issues

– To build on and highlight perceived superiority

• To track and monitor key competition and category for trends, movements and initiatives

• To generate positive word of mouth for the brand; to develop a customer centric focus

• To address and resolve customer issues before they affect reputation

• To provide inputs for messaging and communication

• To mine the online space for category and brand drivers and needs – To drive customer-centered product development and marketing efforts

• To drive ROI on online marketing and outreach programs



The Travel Social Media Space

Blogs Twitter Product Review Sites

Microblogs TripAdvisor

Expedia

Google Hotels.com

Orbitz.com

priceline.com Facebook

Social Media Analytics - A Framework -

Analytics Dimensions

Volume Analysis

(Descriptive Statistics - # of hits, brand

mentions; sites driving traffic; time series

analysis)

Content Analysis & Insights

(Feature Based Opinion Mining using

best practices from traditional market

research)

Direction Analysis

(Sentiment, influence, Mood & Emotions)


Social Media Feedback at Hotel Property Level & Analytics Recommendations

Twitter Google, Bing TripAdvisor, igougo et al

Facebook Expedia,

Priceline et al

Less volume to track

Short term, event-based high volume

Deep user generated

content

Use automated

tools

Monitor for customer

satisfaction, influence

Insights from human analysis


Overall Approach

• Created & validated the Brand Context - brand names/search tags for both client and competition

• Mined hotel brand data from travel destination sites, travel booking engines, and standard search and web 2.0 review sites using social media aggregator tools

• Mapped volume trends, brand mentions and traffic sources to identify key online vehicles for our brand

• Deep-dive Content Analysis of User-Generated Content and reviews on key identified online vehicles (hotel review sites like TripAdvisor, Priceline, Hotels.com in this case)


Content Analysis Methodology

Developed a framework which took free flow text in reviews and classified them into various categories (For eg. room/service/ambience/…)

Using an iterative process, this Master Code Frame was refined with further reviews, across competitors and multiple sites till further refinements became superfluous or repetitive.

Finalized Code Frame was then applied back on to the aggregated (across sites/competition) data to categorize all unstructured review data for the brand

Leading to incisive insights for the brand set in a meaningful competitive context across online social media


HOSPITALITY CASE STUDY - 1

Looking beyond volume trends


Overall, Hotel C generating more mentions –

but this evens out in the high-influence review

sites

374

9 0 13 6 8 0 1 0 0 11

# of brand mentions – Hotel C

Sept’2010

Google

Facebook

Twitter

Blogs

Microblogs

TripAdvisor

Hotels.com

Expedia 374

9 0 13 6 8 0 1 0 0 11

272

10 8 13 1 11 0 3 0 1 8

0

50

100

150

200

250

300

350

400

# of brand mentions

Competitive Context – Sep’2010

Hotel C

Hotel X



TripAdvisor & Priceline are where the action

is - Hotel C & Hotel X generate equal traction

on these sites

Source: Google, Respective Sites,

SocialMention

8

0

1

0 0

11 11

0

3

0

1

8

0

2

4

6

8

10

12

TripAdvisor Hotels.com Expedia igougo Travelocity Priceline Orbitz

# of brand mentions

Hotel C

Hotel X


Hotel C with a net positive disposition; Hotel

X generating significant negative sentiment

Source: Google, Respective Sites,

SocialMention

-30

-20

-10

0

10

20

30

40

50

Positive

Sentiment

Neutral

Sentiment

Negative

Sentiment

Net Effect

20

0

15

5

21

0

46

-25

Hotel C

Hotel X


Room, Staff and Ambience are key drivers of

customer reviews; significant negatives on these

for Hotel X

-20

-15

-10

-5

0

5

10

Room

Locati

on

Pri

ce

Sta

ff

Build

ing A

esth

eti

cs

In-ro

om

Am

enit

ies

Pro

cesses

Facilit

ies

Am

bie

nce

6 3 3 4

1 1 2

-4 -1

-3 -2 -1 -4

4

1 2

5

1 1 1

4 2

-14

-1 -1

-8 -3 -7

-2

-10

Hotel C Hotel 'C Hotel X Hotel 'X

Significant drivers for Hotel C

Social Media Analytics Case Study 1 - Insights

Volume Trends:

• Hotel C with greater overall reach in the online medium

• However, review sites were key influence sites driving significant traffic.

• And on review sites, Hotel X drew more mentions than Hotel C.

Hence, Hotel X - highly focused with brand mentions where it counted – on the review sites

Sentiment Direction:

• A different spin ! Hotel X nursing a significant net negative sentiment

• Hotel C, though more muted, with overall positive sentiment.

An intriguing twist in the tale! Hotel X generating significantly negative word of mouth. In Hotel X's interest to track and fix this to prevent a damaging cascading

effect.

Hotel C, perhaps, could sit pretty for the moment?


Social Media Analytics Case Study 1 - Insights

Feature Based Sentiment Mining:

• Room, Staff & Ambience were key category drivers of customer reviews

• On all 3 parameters, Hotel X drew significant negative sentiments – a clear call for action for Hotel X to prioritize and address these

• Hotel C - while locally perceived superior to Hotel X, – This was driven not by the key drivers of customer reviews (room, staff or ambience) but by price

and location.

For Hotel C, while this could work as a potentially short-term tactical position, it needed to address room and service issues so as to build a strategically sustainable long-term position besides looking for key differentiators..


HOSPITALITY CASE STUDY - 2

Mining customer reviews for strategic and tactical directions


LuxH3 customers more voluble in voicing negative sentiment; While LuxH1 generates less buzz (perhaps due to its smaller size), the ratio of

positive to negative buzz is on par with LuxH2

LuxH1 LuxH2 LuxH3

#.of customers 31 45 69

# of review themes 154 402 667

Buzz per customer 5 9 10

# of positive reviews 96 274 352

Positive buzz per customer

3 6 5

# of negative review

58 128 312

Negative buzz per customer

2 3 5

Nov 2010 – Feb 8th 2011


010203040506070

% Contribution

Travel Review Site Total Reviews Analysis – By Feature Nov.2010 – February 8th 2011 (3 months)

Base: Reviews of all 3 properties; n=1223

0

20

40

60

LuxH1

% Positive % Negative

-10

10

30

50

70LuxH2

% Positive % Negative

Property and Food are the key themes driving customer buzz

followed by room and service.

For LuxH1, room is a significant negative sentiment generator –

need to address!


Travel Review Site Sentiment Analysis – By Feature

Feature Positive

Sentiment Negative

Sentiment Total Reviews

Room Thin Walls 6 6

Maintenance 5 5

Cleanliness 4 1 5

Bed 2 2 4

Amenities - TV 4 4

Smell 3 3

Bath 3 3

Amenities - Water 1 2 3

Insects 2 2

Lighting 2 2

Others 2 2

Amenities - Snacks 1 1

Storage 1 1

Style 1 1

Size 1 1

Nov.2010 – February

8th 2011 (3 months)

Base: Reviews for

LuxH1 LuxH1; n=154

“The LuxH1 had very disappointing accommodation; They looked lovely from the outside but the interiors were VERY basic and in need of a refurb”

“The location is perfect - along the ocean within Location, a gated community right in Location 2”

“ The maid service was exceptional”

“You get water in your room if you are lucky; After 2 days of no water and going up to the front desk 4 times, we were finally given 3 small bottles”

“The toilets, bathtub/shower and sink drainage are all connected to the sewer system Thus, you sometimes get sewage "smell" backup”

Clear maintenance issues emerging on the rooms in LuxH1 including

TV, bath and smell issues.

The ‘Thin Walls’ issue may need a creative solution.


Conclusions & Insights – LuxH1

• As the smallest hotel in this competitive set, LuxH is able to maintain an appropriate amount of positive buzz

• Maintain & build presence

Buzz Volume & Sentiment

• Property, food, room & service are key category buzz themes

• Food is a key brand driver for LuxH1

• LuxH1 also delivering on Property & Staff

Category & Brand Drivers

• LuxH1 generating significant negative buzz around its rooms

• Specific issues pertain to thin walls – this may need a creative solution

• However, LuxH1 should address immediately other issues of maintenance of the room and bath, bad odor and outdated TVs.

Key Concern - Rooms

Conclusions & Insights – LuxH2

• The hotel with the best Travel Review Site ranking but more muted than LuxH3 in the volume of buzz created

• Need to raise buzz levels focusing on the positives

Buzz Volume & Sentiment

• Property – specially the pool and the grounds are key strengths; addressing the pool temperature and seating will raise delight levels

• But room & service need attention – focusing on the quality of the bed and combating insects

Category & Brand Drivers

• While LuxH2 has the best Travel Review Site ranking, the muted level of review rhemes is a concern

• Need to generate excitement and raise buzz levels – beauty of the property and pool could be the messaging focus

Key Concern – Muted Buzz;

Leadership under Threat

The Analyst Edge


The Human Edge - Analyst over Automation -

• Superior to current Sentiment Mining Options:

– Automated approaches based on NLP/Machine Processing

– Based on ‘text rules’; unable to understand context

– Leading to error rates of 30-40%!

• Attuned to Domain/Cultural/Language Sensitivities

– Web 2.0 is all about User-Generated Content; about people; about listening to people

– Social Media Analysts are able to tune in to cultural, language and domain sensitivities for killer insights which a machine is unable to deliver.

• Mining Tangential Insights

– Analysts listening in to brand buzz, category chatter, trends and opinions are able to bring a wider perspective to Insights Mining

– Eg. Tactical promotional ideas emerging from a routine reputation audit


The Human Edge - 2 - Analyst over Automation -

• The Need:

– The need to mine insights from customer review data, while ongoing, is relevant mainly in its aggregate. For eg. 1 customer panning the service of a hotel drives a different response from a consistent, decrying of food quality by n customers over 6 months.

– Analysis of data in aggregate drives systemic process and product resolutions and it is critical that no functionality is lost while conducting this aggregate analysis – a need for researcher involvement and intervention.

• Analytic Freedom & Depth:

– Finally, we believe that automation stops short of providing the required analytic freedom nor depth to provide actionable insights for businesses.


Applications


Applications - 1

Across categories:

• While the above case study relates to a hospitality example, the framework developed is easily applicable to all forms of customer review data.

– Product-centric reviews for automobiles, consumer electronics and even consumer packaged goods

– Review of after-sales services/processes for the above as well as service oriented businesses like financial services, restaurants, entertainment and utilities.

Spin-off Insights:

• We found that the significant advantage of the framework was in utilizing discretionary judgement to mine insights that were domain-specific or which needed a cultural/language sensitivity – an advantage over machine recognition of comparative adjectives/rules.


Applications - 2

Exploratory Research:

• We recommend the above approach additionally when businesses are interested in exploratory research and idea generation – when customer verbatims are of significant importance.

Optimizing Costs:

• A further critical learning from this study was on the issue of optimizing costs for a labor-intensive framework. We found that businesses would need to analyze review data in periodic aggregates in order for it to be actionable. This would usually work out to a quarterly reporting format for most businesses to allow for a sufficient enough sample of reviews to accumulate. For such periodic deep-dive analysis, the return on investment on a bespoke custom analysis is very clearly tangible and justifiable.


why man is mightier than the machine - jimdo

Documents