Download - data science history / data science @ NYT

data science @ The New York Timesand how a 164-year old content company became data-driven

[email protected]@nytimes.com@chrishwiggins

references: bit.ly/icerm

“data science” jobs, jobs, jobs


data science: mindset & toolset

drew conway, 2010


modern history:2009


“data science” blogs, blogs, blogs



The first time I heard "data science" was in 2007 while reading a proposal that my adviser had passed along, outlining an academic program similar to what we think of as data science.

The first time I heard "data science" was in 2007 while reading a proposal that my adviser had passed along, outlining an academic program similar to what we think of as data science.


“data science” ancient history: 2001


data science context


home schooled


PhD in topology


“By the end of late 1945, I was a statistician rather than a topologist”


invented: “bit”


invented: “software”


invented: “FFT”


“the progenitor of data science.” - @mshron


“The Future of Data Analysis,” 1962John W. Tukey


introduces: “Exploratory data anlaysis”


Tukey 1965, via John Chambers


TUKEY BEGAT S WHICH BEGAT R


Tukey 1972


? 1972


Jerome H. Friedman


Tukey 1975

In 1975, while at Princeton, Tufte was asked to teach a statistics course to a group of journalists who were visiting the school to study economics. He developed a set of readings and lectures on statistical graphics, which he further developed in joint seminars he subsequently taught with renowned statistician John Tukey (a pioneer in the field of information design). These course materials became the foundation for his first book on information design, The Visual Display of Quantitative Information


TUKEY BEGAT VDQI


Tukey 1977


TUKEY BEGAT EDA


fast forward -> 2001


“The primary agents for change should be university departments themselves.”



histories

1. in academia -> Bell: as heretical statistics (see also Breiman)

2. in industry: as job description

historical rant: bit.ly/data-rant

biology: 1892 vs. 1995

biology changed for good.


genetics: 1837 vs. 2012

ML toolset; data science mindset



ML toolset; data science mindset

arxiv.org/abs/1105.5821 ; github.com/rajanil/mkboost

data science: mindset & toolset


1851


news: 20th century

church state


church


news: 20th century

church state


news: 21st century

church state

engineering


1851 1996

newspapering: 1851 vs. 1996


example:

millions of views per hour2015

data science: the web



is your “online presence”



is a microscope



is an experimental tool



is an optimization tool


1851 1996

newspapering: 1851 vs. 1996 vs. 2008

2008


“a startup is a temporary organization in search of a repeatable and scalable business model” —Steve Blank


every publisher is now a startup


news: 21st century

church state

engineering


learnings


learnings

- supervised learning- unsupervised learning- reinforcement learning


learnings

- supervised learning- unsupervised learning- reinforcement learning

cf. modelingsocialdata.org


stats.stackexchange.com


from “are you a bayesian or a frequentist” —michael jordan

L =NX

i=1

' (yif(xi;�)) + �||�||

supervised learning, e.g.,


supervised learning, e.g.,

“the funnel”


interpretable supervised learning

supe

r co

ol s

tuff


interpretable supervised learning

supe

r co

ol s

tuff


arxiv.org/abs/q-bio/0701021

optimization & learning, e.g.,

“How The New York Times Works “popular mechanics, 2015

recommendation as supervised learning

unsupervised learning, e.g,

cf. daeilkim.com ; import bnpy

modeling your audiencebit.ly/Hughes-Kim-Sudderth-AISTATS15

modeling your audience(optimization, ultimately)

also allows recommendation as inferencemodeling your audience

Reporting

Learning

Testaka “A/B testing”;

business as usual

(esp. supervised)

Some of the most recognizable personalization in our service is the collection of “genre” rows. …Members connect with these rows so

well that we measure an increase in member retention by placing the most tailored rows higher on the page instead of lower.


reinforcement learning: from A/B to….

real-time A/B -> “bandits”

GOOG blog:


Reporting

Learning

Test

Optimizing

Exploreunsupervised:

supervised:

reinforcement:

common requirements in data science:

common requirements in data science:

1.people2.ideas3.things

cf. USAF

things:what does DS team deliver?

things:what does DS team deliver?

- build data prototypes- build APIs- impact roadmaps

- build data prototypes

- build data prototypes

cf. daeilkim.com

- in puppet, w/python2.7- collaboration w/pers. team

- build APIs

- impact roadmaps

flickr/McJex

data science: ideas

data skills

- data engineering- data science- data visualization- data product- data multiliteracies- data embeds

cf. “data scientists at work”, ch 1

data science: people

- new mindset > new toolset

summary:pay attention to:

1.people2.ideas3.things

cf. USAF

thanks to the data science team!

Download - data science history / data science @ NYT

Top Related