actionability of insights

Intended for Knowledge Sharing only

Actionability of Insights

Text Analytics Summit

Text Analytics Summit | June 2015


Disclaimer: Participation in this summit is purely on personal basis and not representing VISA in any form or matter. The talk is based on learnings from work across industries and firms. Care has been taken to ensure no proprietary or work related info of any firm is used in any material.

Director, Insights at Visa, Inc.

Help Executives/Product/Marketing

with actionable insights

RAMKUMAR RAVICHANDRAN


Quick recap of what is it?Quick recap of what it is


What makes an insight actionable?

NEITHER ACTIONABLE, NOR INSIGHT!


WHAT IS IT AFTER ALL??


Specific answer to the question

Easy to understand

Timely & available (whenever, wherever & however needed)

Trustworthy & reliable

Scalable & Repeatable




Seems easy enough?

SEEMS EASY, SO WHERE IS THE PROBLEM?


Sky high expectations (Data Scientists the new “Alchemists”)

Philosophical differences

Vague Questions/Undefined

Constraints (Data, Time, Resources unavailable)

Heartbreak Syndrome “not what I expected you know, answer”

8

DATA SCIENTISTS, THE ANSWER TO MY PRAYERS…


9

CUSTOMER EDUCATION ON MULTIPLE VALUE PROP WOULD HELP


Size behaviors

with KPIs and high level drilldowns (Sizing)

InformInvestiga

tePredict Optimize Mine

Root cause analysis:

Hypotheses testing via

data drilldowns (Business Analytics)

Determine Causal

relationships (Advanced Analytics)

Experiments on options to verify which one works

(A/B Testing)

Automated relationship

discovery and Data Products

(Machine Learning)

10








PHILOSOPHICAL? WHAT DO YA MEAN?


"Analysts are from Mars and their customers are from Venus…

ANALYSTS CUSTOMERS

Language they speak

Numbers, graphs, lifts, accuracy, insights

If we do this, then we get this...

GRAPHS, OH SO MANY OF THEM!


CAUSES THIS




Analysts are from Mars and their customers are from Venus…

ANALYSTS CUSTOMERS

Language they speak



What excites them Brilliance of approach Simplicity of the answer

THE SEARCH FOR TRUTH…


WHEN ALL THEY WANT IS…





ANALYSTS CUSTOMERS

Language they speak




How they think Detail oriented Big picture

HMMM, WHAT IF WE DO THIS?


WHY?





ANALYSTS CUSTOMERS

Language they speak





What they can compromise on Time for accuracy Perfection for timely

action

OUR MISSION OF LIFE


BUT SHE IS LIKE





ANALYSTS CUSTOMERS

Language they speak





What they can compromise on Time for accuracy Perfection for timely

action

Biggest difference Scientists who deal with facts Artists who deal with gut

RING A BELL?


WAIT, WHAT?


HOW TO MANAGE CONSTRAINTS


•Tactical Prioritization: Classify the requests into “Firefights, Urgent and important, Important but not urgent, Good to have” based on factors like Requestor, Urgency, Impact and availability of resources solve it.

•Pre-Analysis work: Strategic prioritization (Outcome Focused), Gap analysis on data/proxy, Various approaches and the Sizing of ETA for each, final output templates.

•Expectations setting: Discuss with requestors, the output from Pre-Analysis and decide together on next steps. Set up Milestones/regular check-ins.

•Execution, Communication, Fine tuning & Course-correction (if necessary)

•Automate if necessary

STRATEGIC PRIORITIZATION (ILLUSTRATIVE)


Sl. No.

Ask Why is it needed? How will it be used?

Fit with the high level business Strategy/KPI

impactedPlan B?

1What are the Consumers

saying?

Redesigned the website and need

to know the customer reaction

Roll back/ Ramp up based on feedback

Customer Satisfaction & Engagement via better website UX

A/B Test findings only

2Thematic

Extraction of Tweets

Investigate why Site Engagement

(#Page Views/Visit) down WoW

Feedback will inform where issues were and need to

be addressed

Maintain Product uptime for the users

Pathing Analysis/ Heatmap

Analysis only

OUTPUT CUSTOMIZED TO CONTEXT OF THE USER (NEED, TIME, MINDSET)


“In mail”

Recommendations with supporting

graphs, tables, etc.

“Story Deck”

Full deck with the pitch and supporting arguments, numbers,

graphs, charts

“On-the-go”

-Mobile App, On the Cloud,

Subscriptions-Reports,

Dashboards, Infographics

Algorithm/Model

Ready to be deployed

How to decide? Customer needs; Turnaround Speed;

One time/reuse; Deployment on Front end; Strategic Doc;

Quick read/research doc

CUSTOMER DRIVEN ANALYTICS


Pre-work & Kickoff1 Analyst, Customer

Translation to Analytical Framework2 Analyst, Researcher, Data Instrumentation, & Data Manager,

Developer, Data Scientist

Data Collection and Preparation3 Analyst, Data Manager, Data Scientist

Analysis, Validation & Verification4 Analyst, Data Scientist, Customer and SME, Researcher

Actionable insights and impact sizing5 Analyst, Customer, Leader

A/B Testing6 Analyst, A/B Testing, Customer, Developer

Rollouts7 Customer, Leadership & Executives

ResponsibleSteps

COME ON BOY, DO IT NO…


BUT WAIT, WHAT…


FINALLY SOMETHING ON TEXT ANALYTICS


Complex and takes time to execute(Data collection/standardization/ cleaning up/Preparation too long the first time)Chance of Model/Analysis not revealing anything (junk data)Chance of Model/Analysis giving false positives (since sparse data issue)Low RoI Exercise (too much effort for little incremental benefit)Dependency on others – researchers, instrumentation, etc.

Specific Risks with Text Analytics…

Rigorous pre Analytics assessment and expectations setting.Success Criteria to be changed from Significance to Consistence/raw counts.Impact Sizing and Vetting the impact with A/B Testing.

…so what can be done about it




Wrapping it up

WHY IS THE TOPIC SO POPULAR NOW?

Evolution in the value prop of Analysts: What/where/how much -> what can happen ->what should we do ?

Audience has broadened (A numbers middle man -> Front line Managers)Luxury of time has evaporated

Nature of questions have drastically changed (Expectation of being able to connect the dots in “Data Lake” world).

Overselling potential before getting “there”

REALLY WRAPPING IT UP, I PROMISE…

• “Know” that not all Analytics is supposed to be actionable.

• “Must have” User Experience Design (UED) Strategist for the Analytics practice

• “Ensure” Deeper Stakeholder involvement in Analytics development & Test & Learn approach must

• “Develop” Outcome Focused Approach for Analytics

• “Prepare” for ever more increasing ask for analytics and related actionability issues

Putting it all together…




Appendix

Intended for Knowledge Sharing only 4040

THANK YOU!


Would love to hear from you on any of the following forums…

https://twitter.com/decisions_2_0

http://www.slideshare.net/RamkumarRavichandran

https://www.youtube.com/channel/UCODSVC0WQws607clv0k8mQA/videos

http://www.odbms.org/2015/01/ramkumar-ravichandran-visa/

https://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a

https://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a

http://www.slideshare.net/RamkumarRavichandran

http://www.odbms.org/2015/01/ramkumar-ravichandran-visa/

https://www.youtube.com/channel/UCODSVC0WQws607clv0k8mQA/videos

actionability of insights

Data & Analytics

knowledge sharing

quick recap

specific answer

ya mean

data products machine

question easy

insight actionable

high level drilldowns