The Evolution of TC’sBusiness Intelligence

OUR COMPANY

• Mobile F2P games• Android, iOS and Amazon• Casual & Midcore titles

OVERVIEW

• F2P market has no barriers to entry• Every user matters• So how do you study users? Their

behavior? How do you make your game perform the way you want?

• Analytics

DATA COLLECTION SYSTEM

What information is collectedHow does this relate to your KPIs?

Analytics systems are not perfectWhat are the biases in your system?

Where is this explained?

DATA COLLECTION EXAMPLE• TinyCo: Data is a 99.5% biased sample

• If you cut our data down far enough, you will find outliers that influence your metric

• Solution: • Use medians and quartiles to

understand dist.• Understand common failure

points

DATA COLLECTION (Cont.)

DEFINE YOUR TERMS

• ARPU, ARPDAU, ARPPU, ARPPDAU, % Payer, Retention

• Where are they standardized? When do you tell PMs the definition?

DEFINE TERMS (Cont.)

Don’t spend meetings arguing if D7 ARPU includes the 7th day or ends the night of the 6th day!

AVOID COLLECTING USELESS INFO• Now you have a system – don’t

collect useless information!

• Have a data warehouse, but treat it like a tiny closet.

DON’T IGNORE SAMPLE SIZE BIAS• Now you have a system – don’t

collect useless information!

• Have a data warehouse, but treat it like a tiny closet.

SAMPLE SIZE BIAS (Cont.)

Even though spoilage looks great, don’t do it!

SAMPLE SIZE BIAS (Cont.)

Just a small sample size after all…

POST HOC ERGO PROPTER HOC• Because something occurred after

another event those events are related

• Pretty obvious, one example

DON’T IGNORE CULTURE• Every fact, every email, every

statement needs a source. • A data driven culture will only occur if

all parties are held to a high standard.

• “Query or it didn’t happen”

DON’T GET ANGRY WHEN THINGS FAIL• Assertions are easy, Facts are hard• Things will break, mistakes will be

made• Getting angry doesn’t help• Find the cause, correct and move on

WHY IS THIS SO HARD?!To Quote one S. Holmes:

“Most people, if you describe a train of events to them, will tell you what the result would be. They can put those events together in their minds, and argue from them that something will come to pass. There are few people, however, who, if told them a result, would be able to evolve from their own inner consciousness what the steps were which led up to that result. This power is what I mean when I talk of reasoning backward, or analytically.”

THANK YOU!