lean analytics 40 minute intro
DESCRIPTION
TRANSCRIPT
Upcoming Networking Events
1
Detailed Schedule and registrations as well as past session videos now available www.productleadership.in
“5 things every Product Leader should know about UXJuly 20th, BangaloreSpeaker: Jay, Head of User Experience Design Adobe
“Webinar: Global Customer Insights from India Product Management – Can it be done?Aug 7th, OnlineSpeaker: Muthu. R, Director Product Management, SAP Labs
“5 ways in how Product Management is different in AgileJuly 25th, PuneSpeaker: Dr. Adrian (Hong Kong), Faculty Institute of Product Leadership
Implementing Lean AnalyticsBen Yoskovitz@byosko
1111
2222
Basics of Analytics
One Metric That Matters
3333 Lean Analytics Cycle
What We’ll Cover
1The Basics of Analytics
Analytics is the measurement of movement towards business goals.
In a startup, the purpose of analytics is to iterate to product/market fit
before the money runs out.
http://www.flickr.com/photos/pseudonomad/5955268183/sizes/l/in/photostream/
WhatWhat’’s a good metric?s a good metric?WhatWhat’’s a good metric?s a good metric?
Good Metrics are:
1111 Understandable
4444 Behavior Changing
2222 ComparativeActive Users vs. Active Users/month
3333 Ratio / Rate% Monthly Active Users
If it won’t changehow you behave,it’s a
badmetric.
If a metric won’t change how you behave, it’s a
http://www.flickr.com
/pho
tos/circasassy/785
8155
676/
http://www.flickr.com/photos/mattimattila/3822631755/
Types of MetricsTypes of MetricsTypes of MetricsTypes of Metrics
Warm and fuzzy. Cold and hard.
Unstructured, Unstructured, anecdotal, anecdotal,
revealing, hard to revealing, hard to aggregate.aggregate.
Unstructured, Unstructured, anecdotal, anecdotal,
revealing, hard to revealing, hard to aggregate.aggregate.
Numbers and stats; Numbers and stats; hard facts but less hard facts but less
insight.insight.
Numbers and stats; Numbers and stats; hard facts but less hard facts but less
insight.insight.
Qualitative
vs.
Quantitative
Discover qualitatively.Prove quantitatively.
Quantitative vs. qualitative data
Do hosts with Do hosts with professional professional
photography get photography get more business?more business?
Do hosts with Do hosts with professional professional
photography get photography get more business?more business?
Airbnb experiments...
Professional photography helps Airbnb’s business
Gut instinct
Concierge MVP20 photographers in the field
Test resultsTwo to three times more bookings!
Back to the beginningUse additional data to keep experimenting
5,000 shoots / month in Feb. 2012
Historical metric Historical metric that shows you how that shows you how
youyou’’re doing: re doing: reports the news.reports the news.
Historical metric Historical metric that shows you how that shows you how
youyou’’re doing: re doing: reports the news.reports the news.
Number today that Number today that shows a metric shows a metric
tomorrow: tomorrow: makes makes the news.the news.
Number today that Number today that shows a metric shows a metric
tomorrow: tomorrow: makes makes the news.the news.
Try and get here.Start here.
Lagging Leading
vs.
2 The One Metric That Matters
Pick one metric that matters to your business right now.
The metric will change over time, but right now you need to FOCUS.
So how do you do it?So how do you do it?So how do you do it?So how do you do it?
http://www.flickr.com/photos/horiavarlan/4290549806/sizes/l/in/photostream/
Your Business + Stage
What business are What business are you in?you in?
What business are What business are you in?you in?
What stage are What stage are you at?you at?
What stage are What stage are you at?you at?
• E-Commerce• SaaS• Free Mobile App• 2-Sided Marketplace• Media• User-Generated Content
• Empathy• Stickiness• Virality• Revenue• Scale
GR
OW
TH
RA
TE
GR
OW
TH
RA
TE
GR
OW
TH
RA
TE
GR
OW
TH
RA
TE
Lean Analytics Stages
II ’’ve found a real, poorly-met need that a ve found a real, poorly-met need that a reachable market faces.reachable market faces.
II ’’ve found a real, poorly-met need that a ve found a real, poorly-met need that a reachable market faces.reachable market faces.
II ’’ve figured out how to solve the problem ve figured out how to solve the problem in a way they will adopt and pay for.in a way they will adopt and pay for.
II ’’ve figured out how to solve the problem ve figured out how to solve the problem in a way they will adopt and pay for.in a way they will adopt and pay for.
II ’’ve built the right ve built the right product/features/functionality that keeps product/features/functionality that keeps
users around.users around.
II ’’ve built the right ve built the right product/features/functionality that keeps product/features/functionality that keeps
users around.users around.
The users and features fuel growth The users and features fuel growth organically and artificially.organically and artificially.
The users and features fuel growth The users and features fuel growth organically and artificially.organically and artificially.
II ’’ve found a sustainable, scalable ve found a sustainable, scalable business with the right margins in a business with the right margins in a
healthy ecosystem.healthy ecosystem.
II ’’ve found a sustainable, scalable ve found a sustainable, scalable business with the right margins in a business with the right margins in a
healthy ecosystem.healthy ecosystem.
“Gates” needed to move forward
EMPATHYEMPATHYEMPATHYEMPATHY
STICKINESSSTICKINESSSTICKINESSSTICKINESS
VIRALITYVIRALITYVIRALITYVIRALITY
REVENUEREVENUEREVENUEREVENUE
SCALESCALESCALESCALE
• Stage: Empathy / Stickiness
• Model: E-Commerce
• Eco-friendly prints of your photos
• Originally tied specifically to Instagram and built a custom InstaOrder feature
Jumping the gun on product development
Optimize for 1st time orders or repeat purchases?
2x transactions lower bounce rate sign-in goals increased
with InstaOrder
Click checkoutClick checkoutClick checkoutClick checkout
Sign into PayPalSign into PayPalSign into PayPalSign into PayPal
Confirmation pageConfirmation pageConfirmation pageConfirmation page
Confirm orderConfirm orderConfirm orderConfirm order
Back to PayPalBack to PayPalBack to PayPalBack to PayPal
Authorize pre-Authorize pre-approved paymentsapproved payments
Authorize pre-Authorize pre-approved paymentsapproved payments
Success pageSuccess pageSuccess pageSuccess page
without InstaOrder
Click checkoutClick checkoutClick checkoutClick checkout
Sign into PayPalSign into PayPalSign into PayPalSign into PayPal
Confirmation pageConfirmation pageConfirmation pageConfirmation page
Confirm orderConfirm orderConfirm orderConfirm order
Success pageSuccess pageSuccess pageSuccess page
skip steps at your skip steps at your own riskown risk
skip steps at your skip steps at your own riskown risk
Six sample business models
• E-commerce• SaaS (freemium?)• Mobile app (gaming)• Two sided marketplace• Media• User generated content
The SaaS Customer Lifecycle
Customer Acquisition Cost
paidpaid directdirect searchsearch womwom inherent virality
inherent virality
VISITORVISITORVISITORVISITOR
Freemium/trial offerFreemium/trial offerFreemium/trial offerFreemium/trial offer
EnrollmentEnrollmentEnrollmentEnrollment
UserUserUserUser
Disengaged UserDisengaged UserDisengaged UserDisengaged User
CancelCancelCancelCancel
Freemium churn
Engaged UserEngaged UserEngaged UserEngaged User
Free user disengagement
ReactivateReactivateReactivateReactivate
CancelCancelCancelCancel
Trial abandonment rate
Invite OthersInvite OthersInvite OthersInvite Others
Paying CustomerPaying CustomerPaying CustomerPaying Customer
Reactivation rate
Paid conversion
FORMER USERSFORMER USERSFORMER USERSFORMER USERS
User Lifetime Value
ReactivateReactivateReactivateReactivate
FORMER CUSTOMERSFORMER CUSTOMERSFORMER CUSTOMERSFORMER CUSTOMERS
Customer Lifetime Value
Viral coefficientViral rate
ResolutionResolutionResolutionResolution
Support data
Account CancelledAccount CancelledAccount CancelledAccount Cancelled Billing Info Exp.Billing Info Exp.Billing Info Exp.Billing Info Exp.
Paid Churn Rate
Tiering
Capacity LimitCapacity LimitCapacity LimitCapacity Limit
Upselling rate UpsellingUpsellingUpsellingUpselling
DisengagedDisengagedDisengagedDisengaged DissatisfiedDissatisfiedDissatisfiedDissatisfiedTrial OverTrial OverTrial OverTrial Over
One MetricThat Matters.One Metric
That Matters.
How It All Comes Together
The business youThe business you’’re inre inThe business youThe business you’’re inre in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
ScaleThe
sta
ge y
ouT
he s
tage
you
’’ re a
tre
at
The
sta
ge y
ouT
he s
tage
you
’’ re a
tre
at
• Stage: Scale
• Model: SaaS
• SEO toolkit (product suite)
• Reduced KPIs to focus on Net Adds
Moz reduces the KPIs it tracks
If Net Adds:If Net Adds:If Net Adds:If Net Adds: Why & Next Steps:Why & Next Steps:Why & Next Steps:Why & Next Steps:
Net Adds = “business health” indicator
• Was a marketing campaign successful?• Were customer complaints lowered?• Was a product upgrade valuable?
• Can we acquire more valuable customers?• What product features can increase engagement?• Can we improve customer support?
• Are the new customers not the right segment?• Did a marketing campaign fail?• Did a product upgrade fail somehow?• Is customer support falling apart?
What’s your OMTM?
Empathy
Stickiness
Virality
Revenue
Scale
E-commerce
SaaS MediaMobile
appUser-gencontent
2-sidedmarket
Interviews; qualitative results; quantitative scoring; surveysInterviews; qualitative results; quantitative scoring; surveys
Loyalty, conversionLoyalty, conversion
CAC, shares, reactivation
CAC, shares, reactivation
Transaction, CLVTransaction, CLV
Affiliates, white-labelAffiliates, white-label
Engagement, churnEngagement, churn
Inherent virality, CACInherent virality, CAC
Upselling, CAC, CLVUpselling, CAC, CLV
API, magic #, mktplaceAPI, magic #, mktplace
Content, spamContent, spam
Invites, sharingInvites, sharing
Ads, donationsAds, donations
Analytics, user dataAnalytics, user data
Inventory, listingsInventory, listings
SEM, sharingSEM, sharing
Transactions, commission
Transactions, commission
Other verticalsOther verticals
(Money from transactions)
Downloads, churn, virality
Downloads, churn, virality
WoM, app ratings, CACWoM, app ratings, CAC
CLV, ARPDAUCLV, ARPDAU
Spinoffs, publishersSpinoffs, publishers
(Money from active users)
Traffic, visits, returns
Traffic, visits, returns
Content virality, SEMContent virality, SEM
CPE, affiliate %, eyeballs
CPE, affiliate %, eyeballs
Syndication, licensesSyndication, licenses
(Money from ad clicks)
3The Lean Analytics Cycle
Follow the Lean model, and it becomes increasingly hard to lie,
especially to yourself.
Instincts are experiments. Data is proof.
Identify a key business problem, pick the OMTM, and get started.
Pivot orgive up
Try again
Success!
Measure the results
Make changes in production
Design a test
Hypothesis
With data:find a
commonality
Without data: make
a good guess
Find a potential
improvement
Draw a linein the sand
Pick the OMTM
The Lean Analytics Cycle
Draw a new line
Did we move the needle?
Some interesting benchmarks
Growth5% / week (revenue or active users)
Churn2% / month
Engaged visitors30% monthly users10% daily users
Time on site17 minutes
Page load time< 5 seconds
CLV:CAC3:1
Mobile file size< 50MB
Free to paid2% of free users
@byosko
ORDER!
follow me.
instigatorblog.com
leananalyticsbook.com
subscribe.
email me.
Thank you!
39
Have an idea?
Have a suggestion on the speaker/topics?
Want to know about product events?
LinkedIn Group “Adaptive Marketing”