measurement - data

Measurement and Digital Toolbox

@danielsiders

Types of Information

Reported

Understanding Questions

Finishing Forms

Self Consciousness

Don’t Know

Asking the wrong questions

Questions that shape possible answers

Setting expectations

Have you stopped beating your wife?

Would you vote for a woman for president if she were qualified in every other way?

Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened? (34% negative)

Are you certain the Holocaust happened? (1% negative)

Digital is dataEverything is recorded Just pay attention

Observed/Behavioral

Data never tells the whole truth

But most of the time it’s people who screw up

Correlation vs Causation

1 2 3 4 5 6

100

0

20

40

60

80

Month

Sales

Potential customerscost $2.68

What is a viewer worth?

Qualitative vs QuantitativeDescriptiveObservedDisagreement possibleSpeaks for itself

NumericalMeasuredUnitsRequires interpretation

Counting is not measurement

Great! I got up early, the

weather was great, people on the subway were nice, lunch was great and I felt good!

I awoke at 0600. A passenger held a train

door for me. The median temperature

was 72ºF, and I consumed 1700

Tell me about your day?

What is the source?

Who collected it?

Who analyzed it?

Why was it commissioned?

What is the perspective/agenda?

Data from Another Source

Binaries don’t exist

More than one thing can be true

Don’t put people in boxes

Life is analog

0

5

10

15

20

0 3.75 7.5 11.25 15

Assumptions can be dangerous

Data should raise questions

One dot equals 25 people. White Black Hispanic Asian

http://www.flickr.com/photos/walkingsf/

http://blog.okcupid.com/

1.51 Million Active Users

43,397 Tests

http://blog.okcupid.com/

Messages a man gets, by Age and Income

10th 50th 100th

Messages received per week by percentile

<20K

1920

21222324 25 26 27282940

41424344 4546 4748 4950

18

30K40K50K60K70K80K

100K>100K

30313233 34 3536 37 3839

(Observed)

http://blog.okcupid.com/

Lowest High School Graduation rate nationwide

http://blog.okcupid.com/

(Self-reported)

Would you rather lose the right to bear arms or vote?

Would you consider role-playing a rape fantasy

with a partner who asked you to?

http://blog.okcupid.com/

4:1 male to female suicides2:1 female to male attempts

http://blog.okcupid.com/

afsp.org

http://blog.okcupid.com/

http://blog.okcupid.com/

http://blog.okcupid.com/

http://blog.okcupid.com/

http://blog.okcupid.com/

http://blog.okcupid.com/

http://blog.okcupid.com/

http://blog.okcupid.com/

http://blog.okcupid.com/

More gay curious Less gay curious

http://blog.okcupid.com/

last

.fm

last

.fm

What questions does this raise?

How can I test my assumptions?

What problem do I want to solve?

What’s the Goal?

http://www.squaretrade.com/

18

2

4

6

8

10

12

14

16

X Axis

Failu

re R

ate

(%)

iPhone 3GS Blackberry Motorolla HTC iPhone 4 Other

Malfunction

Accident

Google.com/adplanner

Spyfu.com

SEMRush.com

Backtype.com

Scoutlabs.com

Webnumbr.com

#himc