how do i analytics? a practical guide for pragmatic startups

Andy Young // @andyy // [email protected]

How do I analytics?a practical guide forpragmatic startups


Hi, I’m Andy!

@andyy


Wait what, why?

Analytics?


What do we measure, and why?

Vanity metricsRevenue metrics

Conversion rate metricsPirate metrics..

We need to know how we’re doing.


If you're not keeping score there's no point playing the game -

you'll never know if you'rewinning or not

- @distrodom


Today’s tools make it super-easy to track

thingsGoogle Analytics

MixpanelKissMetricsLocalytics

Branch Metrics..


Today’s tools make it super-easy to track

thingsBUT they also make it really easy to

- become overwhelmed with data- focus on the wrong things


Typical analytics challenges/pitfalls

Drowning in too much data

Failure to select + focus on the top metrics that matter

Not tracking the data you need to answer key questions


Why analytics?

1. How are we doing? - are KPIs on the right track?

2. What are the results of our experiments?- so we can learn

3. What’s happening right now?- did something great or terrible just happen?


A. How are we doing?


How’re we doing?

1. Identify top-level KPI


Identify top-level KPI

it’s hard.



if you pick the wrong KPIs, you're screwed.

If you pick KPIs and then ignore them, you're screwed.

If you pick and monitor KPIs diligently, but don't assess everything you and your whole team does on the basis of whether your tasks are the most effective way to grow your KPIs, you're screwed.



How?



Keep it simple!

The good news: there’s probably apre-determined answer

forwhat drives your business



There’s probably a pre-determined answer for what drives your business

Spoiler: ultimately it’s $$$


How’re we doing?

There’s probably a pre-determined answer for what drives your business

Commerce: # salesSubscription / SaaS: # subscribers

Marketplace: # transactions


How’re we doing?

1. Identify top-level KPI


How’re we doing?

1. Identify top-level KPI2. Next, add nuance


How’re we doing?

Nuance behind your top level KPI

E.g. for commerce: # salesNuance: average sale $; # customers


How’re we doing?

1. Identify top-level KPI2. Add nuance3. Add drivers


How’re we doing?

Drivers behind your top level KPI

E.g. for marketplaces: # transactionsDrivers: # suppliers, # customers


How’re we doing?


4. Add funnel for these drivers


How’re we doing?



End up with AARRR


How’re we doing?



Put in a spreadsheet(Template Google Sheet: http://bit.ly/kpi-sheet)

http://bit.ly/kpi-sheet


How’re we doing?





How’re we doing?

Put in a spreadsheet

- key KPI at the top, nuance and drivers below, finally the detailed funnel below for reference

- columns for weekly numbers, w/w growth

- review weekly- share with whole team


Weekly/Monthly reporting

% week-on-week or month-on-month growthin your one metric that matters


Tracking events


Tracking events

Events vs. Properties vs. People

Events: something happenedProperties: something about what just happened

People: connect events to particular users- people can also have properties


Track events from where?Client/app vs. server

Tracking events


Tracking events

Tip #1: Choose easy-to-read and meaningful event names

Short!Pick a convention; stick to it

Omit superfluous words

“user_viewed_homepage”“Viewed homepage”


Tip #2: Track each user based on a distinct ID

Don’t use email address -use autogenerated user_id from your own DB

Use aliasing to connect up events tracked pre/post signup

Tracking events


Tip #3: Annotate your users with source data

referrer; utm tags; install tracking via AppsFlyer1. Track a signup event

2. Add as user properties3. Potentially also as properties to key events

Tracking events


Tip #4: (Mixpanel specific) - People vs. Events

Mixpanel won’t let you query for userswho did particular events

So, our options:- Do this using your own DB

- Annotate your users (People) with properties for each key event

Tracking events


Tip #5: Ecommerce/revenue tracking

Mixpanel/AppBoy etc havenative support for tracking revenue

Annotate your Purchase events with revenue data using the relevant properties for each platform

Tracking events


Tip #6: Use a development project for testing

Tracking events


Tracking the funnel

Start with the pirate metrics AARRR

Top of funnel: acquisition; signups/installsMid funnel: post-install events; engagement;

retentionBottom of funnel: purchase / monetisation.


Tracking the funnel

Looking at each stage (AARRR) in aggregate is a good start

but it will only get you so far

the “truth” is much more nuanced


Tracking the funnel

Users acquired via different channelswill have different behaviours

Different cohorts will have different experiences of your product

Different users will have been exposed to different A/B tests


Tracking the funnel

Key: these are all properties of your users

UTM tags: source, medium, campaign, termsLanding pageSignup time

A/B test bucketsReferrer

Viral source


Tracking the funnel

Annotate your users in your database/analytics system with these attributes

UTM tags: source, medium, campaign, termsLanding pageSignup time

A/B test bucketsReferrer

Viral source


Other key metricsCAC, LTV, churn


Other key metrics

Customer Acquisition Cost (CAC)how much you spend (on average) to acquire a

customer

Lifetime Value (LTV)How much revenue $$ an average customer

brings you in all time


If yourLTV

is greater than yourCAC

then you’re in business


If yourLTV

is greater than 3x yourCAC

then you’re in a good business


CAC & LTV: nuances

Payback period: time to recoup CAC

Magnitude of your numberse.g. enterprise vs. social


Calculating CAC

Simple approach: total spend / total signups

“50% of the money I spend on advertisingis wasted - the problem is I don't know which half”

- John Wanamaker

Eventual goal: calculate CAC per channel


Calculating LTV

Problem!You don’t have a lifetime of data

We don't measure LTV - we estimate it

Extrapolate revenue curve over time


Analysing your data


How not to do Metrics

Outdated information

Just 1 view of your data

Manual calculations

Bad metrics lead you astray


Doing metrics right

Graphs

Automated

Realtime


Cohort analysis?


Problems with Cohort Analysis

Time consuming

Delays to get the latest data

Inflexible


Rolling Cohorts


How?


Use your existing database

Users

Learn SQL! It's not hard

Just need a slave database for analytics- “read replica” - i.e. a live copy


Use your existing data

Users

SELECT COUNT(*) FROM users



Users


WHERE created > ‘2013-07-01’

AND created < ‘2013-08-01’




LEFT JOIN sales USING (user_id)

WHERE users.created > ‘2013-07-01’

AND users.created < ‘2013-08-01’

AND sales.date < DATE_ADD(users.created, 1 MONTH)


1. Automate running queries (every hour!)

2. Store the results in a simple database

3. Create a page to graph the results (HighCharts..)

Roll your own


Visitor numbersUsage / engagement

Revenue Conversion rates

Pirate metrics


Analytics = Knowledge


Knowledge = power confidence

sanity


Good luck!

how do i analytics? a practical guide for pragmatic startups

Data & Analytics