Lean Analytics for Startups and Enterprises

Download Lean Analytics for Startups and Enterprises

Post on 21-Apr-2017

3.874 views

Category:

Business

1 download

Embed Size (px)

TRANSCRIPT

  • Using Lean Analytics for Startups and

    Enterprises

    Ben Yoskovitz | @byosko

  • Introduction

    @byosko

    I am a

    product guy entrepreneur author angel investor

  • Find me online

    Blog: http://instigatorblog.com

    Slideshare: http://slideshare.net/LeanAnalytics

    Book: http://leananalyticsbook.com

    Email: byosko@gmail.com

    @byosko

    http://instigatorblog.comhttp://slideshare.net/LeanAnalyticshttp://leananalyticsbook.commailto:byosko@gmail.com

  • CORPORATE PARTNERS VENTURE-BACKABLE FOUNDERS PRE-SEED FUNDING

    BETTER STARTUPS

    + +=

    Highline BETA is a startup co-creation company that launches new ventures with leading corporations.

    http://highlinebeta.com @byosko

    http://highlinebeta.com

  • Metrics:The Fundamentals

  • Metrics: The fundamentals

    How data fits in

    What makes a good metric

    Types of metrics

    Analytical superpowers

    @byosko

  • How to get things built properly (in theory)

  • Everyone has great ideas, right?

    People love this part (but thats not always a good thing!)

    This is where things start to fall apart.

    No data, no learning.

    Build Measure Learn seems so easy!

  • INTELLECTUALLY HONESTY

    Follow the Lean model and it becomes

    increasingly hard to lie, especially to yourself.

  • FOCUS

    Dont chase shiny objects. You might

    succeed without focus, but itll be by accident.

  • BETTER DECISION MAKING

    Everyone has data. The key is figuring

    out what pieces will improve your learning

    and decision making.

  • USE YOUR GUT PROPERLY

    Instincts are experiments.

    Data is proof.

  • So what makes a good metric?

  • Question: What are the metrics youre tracking?

    Take 2 minutes to write down the key metrics youre tracking (or your business is tracking) right now.

    These could be at a business level or project level.

    At the end of this section we can re-evaluate if the metrics youre tracking are still the right ones.

    @byosko

  • WHAT IS ANALYTICS?

    Analytics is the measurement of movementtowards business goals.

  • A good metric is:

    Understandable

    If youre busy explaining the data, you wont be busy acting on it.

    Comparative

    Active Users vs. Active Users/month

    Ratio / Rate

    % Monthly Active Users

    Behavior Changing

    Youll know how youll change your business based on what the metric tells you.

    @byosko

  • If a metric wont change how you behave, its a

    bad metric.

    THE GOLDEN RULE OF METRICS

    http://www.flickr.com/photos/circasassy/7858155676/

    http://www.flickr.com/photos/circasassy/7858155676/

  • Acquisition1-15% Low cost of acquisition, high checkout

    Customers that buy >1x in 90d

    Then you are in this mode

    Your customers will buy from you

    You are just like Focus on

    15-30%

    >30%

    Hybrid

    Loyalty

    Once

    2-2.5

    >2.5

    per year

    per year

    70%

    20%

    10%

    of retailers

    of retailers

    of retailers

    Increasing return rate, market share

    Loyalty, selection, inventory size

    (Thanks to Kevin Hillstrom for this.)

    Metrics help you know yourself:

  • Types of Metrics

  • Vanity vs. Actionable metrics

    Vanity ActionableMakes you feel good but doesnt change how youll act.

    Helps you pick a direction and change your behavior.

    Up and to the right. These are good.

    @byosko

  • Beware of vanity metrics:

    Users

    Follows / friends / likes

    Logins

    This tells you nothing about what they did, why they stuck around, or why they left.

    Count actions instead. Count how many followers will do your bidding.

    What are they actually doing when they login? Logins dont tell you about actions and value.

    DownloadsSure, people need to download your app in order to use it, but so what?

    @byosko

  • The best (worst!) vanity metric of all time

    # of Features

    @byosko

    https://www.flickr.com/photos/pinoyed/5009440499

  • Qualitative vs. Quantitative metrics

    Qualitative QuantitativeUnstructured, anecdotal, revealing, hard to aggregate.

    Numbers and stats; hard facts, but less insights.

    Warm and fuzzy. Cold and hard.

    @byosko

  • Discover qualitatively.

    Prove quantitatively.

  • Do Airbnb hosts get more business if their property is professionally photographed?

  • Gut instinct (hypothesis)Professional photography helps Airbnbs business

    Concierge MVPSent 20 photographers out into the field

    Measure the resultsCompared photographed listings to control group

    Make a decisionLaunched photography as a new feature to all hosts

    CASE STUDY

    Do professional photos make a difference?

  • Exploratory vs. Reporting metrics

    Exploratory ReportingSpeculative. Tries to find unexpected or interesting insights. Source of unfair advantages.

    Predictable. Keeps you abreast of normal, day-to-day operations. Can be managed by exception.

    Cool. Necessary.

    @byosko

  • Started as Circle of Friends Leveraged Facebook early Grew to 10M users fast

    ENGAGEMENT SUCKED!

    CASE STUDY

    Finding insights in the data

  • ENGAGEMENT SOLVED.

    CASE STUDY

    Moms are crazy! (in a good way) Messages to one another were ~50% longer

    115% more likely to attach a picture to a post

    110% more likely to engage in a threaded conversation

    Invited friends were 50% more likely to become engaged users

    60% more likely to accept invitations to the app

  • Lagging vs. Leading metrics

    Lagging LeadingHistorical metric that shows you how youre doing: reports the news.

    Number today that shows a metric tomorrow: makes the news.

    Start here. Try and get here.

    @byosko

  • Examples of leading metrics

    A Facebook user reaching 7 friends within 10 days of signing up. (Chamath Palihapitiya)

    A Dropbox user who puts at least 1 file in 1 folder on 1 device. (ChenLi Wang)

    A Twitter user who follows a certain number of people, and a certain percentage of those people follow the user back. (Josh Elman)

    A LinkedIn user getting to X connections in Y days. (Elliot Schmukler)

    @byosko

  • 1. People who install the Chrome extension 2. People who connect more than 1 social account 3. People who share 15 pieces of content in 7 days

    CASE STUDY

    Buffer discovered 3 leading metrics

  • Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec

    Correlation vs. causation

  • Correlated vs. Causal metrics

    Correlated CausalTwo variables that are related (but may be dependent on something else.)

    An independent variable that directly impacts a dependent one.

    Ice cream and drowning.

    Summertime and drowning / Summertime and eating ice cream

    @byosko

  • A leading, causal metricis a superpower.

  • Causality is a superpower because it lets you change the future.

    Correlation lets you predict the future

    Causality lets you change the future

    I will have 420 engaged users and 75 paying customers next month.

    If I can make more first time visitors stay for 17 minutes I will increase sales in 90 days.

    Pick a metric to change

    Find correlation

    Test for causality

    Optimize the causal factor

    @byosko

  • Cohort analysis

    https://blog.kissmetrics.com/cohort-and-multi-touch-attribution/

    @byosko

    https://blog.kissmetrics.com/cohort-and-multi-touch-attribution/https://blog.kissmetrics.com/cohort-and-multi-touch-attribution/

  • Ricky (product manager) has some ideas for improving the Proposal Send Screen (based on qualitative feedback & his gut), but before prioritizing this work, he digs into the data.

    http://proposify.biz

    Putting basic data to use

    http://proposify.biz

  • 50% of people send proposals through Proposify (50% dont) (quantitative)

    Is this good or bad?

    Putting basic data to usehttp://proposify.biz

    http://proposify.biz

  • Ricky isnt sure. So hes going to need to look at additional data (exploratory):

    Churn Proposal won rate Any correlations here?

    Putting basic data to usehttp://proposify.biz

    http://proposify.biz

  • @byosko

    Also needs to do more direct customer development to learn more (qualitative)

    All of this work might lead to additional, meaningful product dev (actionable)

    Putting basic data to usehttp://proposify.biz

    http://proposify.biz

  • Look back at the metrics youre tracking

    Remember the metrics you wrote down earlier? How do they stack up now? Are they good metrics?

    What might you change about the metrics youre tracking as a business and/or on a project/feature level?

    @byosko

  • Quick summary on the basics of analytics

    Analytics is about measuring movement towards business goals

    Analytics is about simplifying not complicating

    Analytics is about helping you focus on what really matters

    Remember the Golden Rule: A good metric has to change your behaviour

    @byosko

  • Measuring Success:An introduction to

    Lean Analytics

  • Lean Analytics Framework

    The five stages of business & product development

    Mapping business models

    The One Metric That Matters (KPIs)

    The Lean Analytics Cycle

    @byosko

  • Two keys: the Business youre in & the Stage youre at

    What business are you in?

    What stage are you at?

    E-Commerce SaaS Free Mobile App 2-Sided Marketplace Media User-Generated Content

    Empathy Stickiness Virality Revenue Scale

    @byosko

  • Big companies need one more thing.An understanding of what type

    of innovation theyre doing.

  • Core Adjacent TransformativeDo the same thing

    better.Nearby product, market,

    or method.Start something

    entirely new.

    Regionaloptimizations.

    Innovation, go-to-market strategies.

    Reinvent the business model.

    Get there faster Smaller batches Solution, then testing Increased accountability

    Customer development Test similar cases Parallel deployment Analytics & cycle time

    Fail fast Skunkworks/R&D Focus on the search Ignore the current model &

    margins

    Many models for enterprise innovation

  • Know the problem (customers tell you it)

    Know the solution (customers/regulations/

    norms dictate it.)

    Know the problem (market analysis)

    Dont know the solution (non-obvious innovation

    confers competitive advantage.)

    Dont know the problem (just an emerging need/change)

    Dont know the solution.

    Waterfall:Execution

    matters

    Agile/scrum:Iteration matters

    Lean Startup: Discovery

    matters

    Another way to look at it

    Core Adjacent Transformative

  • Currentstate

    Business optimization

    Product,market,method

    innovation

    Business model

    innovation

    You can convince executives of this

    because some of it is familiar.

    This terrifies them because it eats the current business.

    A three-maxima model for enterprise innovation

  • Improvement Adjacency RemodellingDo the same,only better.

    Explore whatsnearby quickly

    Try out new business models

    Lean approaches apply, but the metrics vary widely.

    Sustain / core

    Innovate / adjacent

    Disrupt / transformative

  • Sustaining Adjacent DisruptiveNext years car Electric car,

    same dealerOn-demand, app-based

    car service

  • So the metrics that matter to a big

    company are dependent on the type of innovation being done.

  • Stages of business & product development

  • Erics three engines of growth

    Stickiness Virality Price

    Approach

    Math that matters

    Keep people coming back.

    Get customers faster than you

    lose them.

    Make people invite friends.

    How many they tell, how fast they

    tell them.

    Spend money to get customers.

    Customers are worth more than

    they cost.

    @byosko

  • Dave McClures Pirate Metrics

  • Dave McClures Pirate Metrics

    Acquisition

    Activation

    Retention

    Referral

    Revenue

    How do your users become aware of you?

    Do drive-by visitors subscribe, use, etc.?

    Does a one time user become engaged?

    Do users promote your product?

    Do you make money from user activity?

  • The Lean Analytics Stages

    Empathy Youve found a real, poorly-met need that a reachable market faces.Youve figured out how to solve the problem in a way that users will adopt, keep using and pay for.

    Your users and features fuel growth organically and artificially.

    Youve found a sustainable, scalable business with the right margins in a healthy ecosystem.

    STAGE GATE

    Stickiness

    Virality

    Revenue

    Scale

  • The Lean Analytics Stages

    Empathy Youve found a real, poorly-met need that a reachable market faces.Youve figured out how to solve the problem in a way that users will adopt, keep using and pay for.

    Your users and features fuel growth organically and artificially.

    Youve found a sustainable, scalable business with the right margins in a healthy ecosystem.

    STAGE GATE

    Stickiness

    Virality

    Revenue

    Scale

    Most products (and startups) fail at this point.

  • CASE STUDY

    Stage: Empathy/Stickiness Model: E-Commerce Originally tied to Instagram with

    an Insta-Order feature

    Jumping the gun on product development

  • Optimize for 1st time purchases or repeat orders?

    WITH INSTA-ORDER

    Click checkout

    Confirmation page

    Confirm order

    Success page

    Sign in to PayPal

    Back to PayPal

    Authorized pre-approved payments

    WITHOUT INSTA-ORDER

    Click checkout

    Sign in to PayPal

    Confirmation page

    Confirm order

    Success page

    2x transactions Lower bounce rate Sign-in goals increased

  • THERE ARE NO SHORTCUTS TO ANY PLACE WORTH GOING. - Beverly Sills

  • Mapping business models

  • Does recurring revenue work for everyone?

    CASE STUDY

    @byosko

  • The leader in predictive analytics for people. Clearfit helps thousands of companies build better teams. As featured in:

    CASE STUDY

    10x revenue increase off of 3x in sales volumePeople dont do subscriptions for haircuts, hamburgers or hiring. You have to understand your customer, who they are, how and why they buy, and how they value your product or service. - Ben Baldwin

  • The goal is to understand the customerslifecycle / journey through every

    touchpoint with your product.

  • Paid Direct WOM Search Inherent virality

    Customer Acquisition Cost

    VISITOR

    User

    FORMER USERS

    Engaged user

    Reactivate Trial over

    Invite others

    Paying customer

    Disengaged

    Account cancelled

    Freemium / trial offer

    Enrollment

    Disengaged user

    Cancel Cancel

    Reactivate

    FORMER CUSTOMERS

    Billing info exp.

    Resolution

    Dissatisfied

    Capacity Limit

    UpsellingSignup conversion

    rate

    Free user disengagement

    Freemium churnReactivation

    rate

    User lifetime value Customer lifetime value

    Trial abandonment rate

    DAU/WAU/MAUPaid

    conversion

    Viral coefficient Viral rate

    Paid churn rate

    Support data

    Tiering

    Upselling rate

    SaaS Customer Lifecycle

  • Returning Paid Direct Search Viral

    Customer Acquisition Cost

    VISITOR

    E-Commerce Customer Lifecycle

    Navigation Search Reco Engine

    1-time buyer

    Cart

    Additions

    Conversion

    Logistics, delays

    Delivery

    Enrollment

    Call to Action

    Sharing

    Unsocial buyer

    Sharing rate

    Returning rate

    Customer Lifetime Value

    Open rate, engagement

    Transaction size

    Emphasis on maximizing cart value, minimizing acquisition

    costs

    Bounced

    Not interested

    Abandoned

    Bounce rate

    Unsatisfied

    Ratings, delivery issues

    Feature usage, product discovery

  • CASE STUDY

    A

    A/B testing what really matters

    B

  • CASE STUDY

    B

    41% increase in revenue per customer! (People bought a lot more product.)

    Conversion also went up, but was secondary in importance.

  • All business models have issuesCAC vs. LTV -- margins are usually very small. A $10M e-commerce business is small.

    Freemium requires tens of millions of free users. They can be expensive to support. Will enough convert?

    The average # of apps downloaded by North Americans per month is now 0. Monetizing is incredibly hard. Popularity is fleeting.

    Chicken & egg problem. Supply and demand. How do you build up both enough?

    Real monetization requires hundreds of millions of engaged visitors. Peoples attention is hard to capture and keep.

    Content creation. Will it be good enough? Will enough people do it? Why?

    E-Commerce

    SaaS

    Mobile Apps

    2-sided Marketplace

    Media

    UCG

    @byosko

  • You know what busi...