june 2013 irmac slides
TRANSCRIPT
SOLVEforINTERESTINGOTHERWISE LIFE IS DULL.
BIG DATAAND THE QUANTIFIED SOCIETYIRMAC 2013 ANNUAL LUNCH
Denver International Airport
“Managing information and data as a key enterprise asset.”
It’s not about data.
(This is called foreshadowing.)
First:What’s Big Data?
Breadcrumb trail
http
://w
ww.
flick
r.com
/pho
tos/
jenn
y-pi
cs/3
2396
3849
4/si
zes/
l/
Big Data:It’s people.
A technology shift.
The duck definition.
Large amountsof information
Publicand private
Easily linkedand collected
Stored just because we can
Analyzed by humans and
machinesIn near real time
Applied to business
Usable by everyone
Fed backinto the system
Volume(the “big”
part)
Velocity(the “fast”
part)
Variety(the
“anything” part)
Pickanytwo
BIG * FAST * VARIED = K
None of this is new.
What’s new is that the constant dropped to
nearly zero.
Big, fast, and varied got cheap.
Uses 75% less coal.
WTF?
Shouldn’t more efficiency mean less consumption?
more consumption.
Efficiency means
lower costs means
new uses means
more demand means
Big Data is about abundance.
(Abundance also means a billion bad
data managers.)
Second:Good or bad?
Relational
Statistical
BIG
“All your friends are poor” is an awkward conversation.
A force for good.
Ward off disease.Pinpoint disasters.Reveal corruption.Make cities smarter.Improve how we teach.
Big healthcare
Big philanthropy
Big commuting
A force for bad.
Erode our privacy.Justify prejudices.Polarize groups.Leak private truths.
Big prejudice
Audience participation time!
How amusing.
“…nobody notices offers they do not get. And if these absent opportunities start following certain social patterns (for example not offering them to certain races, genders or sexual preferences) they can have a deep civil rights effect.”
Anders Sandberg, Oxford University
Personalization looks a lot like prejudice.
http://www.flickr.com/photos/thisisbossi/3361608344/
Big nanny state
Big radio
0
15
30
2007 2012
Times a song in “heavy rotation”is played each day
Every 4h
Every 55m
Don’t feel bad.
Don’t feel bad.
Even Einstein
had a therapist.
Third:Two main camps
Semantic idealists.
“It has never been spread out, yet,” said Mein Herr. “The farmers objected: they said it would cover the whole country, and shut out the sunlight! So we now use the country itself, as its own map, and I assure you it does nearly as well.”
Lewis Carroll’s Sylvie and Bruno Concluded (1893)
Lewis Carroll’s Sylvie and Bruno Concluded (1893)
Chaotic nihilists.
Urban Dictionary
Chaotic nihilist
Semantic idealist
Chaotic nihilist
Semantic idealist
Fourth:We’re bad at data
Broken windows
http://www.flickr.com/photos/shoes_on_wires/3348377521/sizes/l/
Roe V. Wade
Leaded gasoline
http
://w
ww.
flick
r.com
/pho
tos/
stev
ensn
odgr
ass/
4385
0456
39/
Winter tires
http://www.flickr.com/photos/highwaysagency/6256644401/sizes/l/
Ice cream and drowning
http://www.flickr.com/photos/wtlphotos/1086968783
“At least one of my kidsis a girl.”
What are the chances the other is a boy?
BB BG
GB GG
GB GG BG
2 of 3 (66%) are boys.
“A subjective degree of belief should rationally change to
account for evidence.”
(AKA Bayes’ Theorem.)
“Anyone who values truth should stop worshipping reason.”
(AKA the real world.)
We prefer false positives.
Wooly mammoth
http://www.flickr.com/photos/pong/172438102/sizes/o/
Sun temple
http://www.flickr.com/photos/30787002@N02/3298693694/sizes/l/
Polarizing through tone
Pew and political polarization
We’re bad at this
Mistake correlation for causalitySeek truthiness rather than factFind patterns where they don’t existEasily swayed by toneSide with our tribesDig in and ignore new evidence
Athenian swimming pools
Kenya
28,400,000
America
204,000,000
Obama born in America: 87.8%
Fifth:Sloth
24 months ago, the average person was still afraid of IT.
Today, the average person is terrified of being without it.
These are quickly becoming our prosthetic brains
Don’t worry, there’s more.
“Where’s a good sushi restaurant?”
Nearby
That friends like
Within price range
Novel Known
Within diet restrictions
No allergies
Location
Socialgraph
Personalfinances
Personality
Weight,activity
Medicalhistory
A good brainknows when to interrupt.
“Excuse me, but OMG best sushi
joint!”
Nearby
That friends like
Within price range
Novel Known
Within diet restrictions
No allergies
Location
Socialgraph
Personalfinances
Personality
Weight,activity
Medicalhistory
“Excuse me, but OMG best
sushi joint!”Visits, pays, & reviews.
Ignores.
Sixth:What it’s really about
525,000 state & local officersUnder 25 officers per precinct130 million incident reports200,000 uses of force31% keep computer files
Evidence.com
Hard drive
This is about analysis on a level playing field.
High
Low
RewardLow High
Risk
A better prediction of the future means a lower risk at
the present.
This fundamentally changes the net present value of
investments.
This is about dramatically changing economic
fundamentals.
John Boydcan shootyou down.
The OODA loop.
This is about reducing organizational cycle time.
“The right information in the right place just changes your life.”
- Stewart Brand
“Design is the human capacity to plan and produce desired outcomes.”
- Bruce Mau
This is about designing interruptions and
producing outcomes.
Assets that don’t produce results are called liabilities.
The Big Data revolution is your chance to go beyond
asset management.
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES HAD TAKEN
BATHS BEFORE.
Once, a leader convinced others in the absence of data.
Now, a leader knows what questions to ask.
SOLVEforINTERESTINGOTHERWISE LIFE IS DULL.
Alistair [email protected]@solveforinteresting.comTHANKS!
More:The long game
Live prototypesPlanned for 1024x768 minimum resolution and 32 MB memory
Last compiled in 1998: Works with Explorer 4.0 and Netscape 4.5
Every interface works both ways.
http://www.flickr.com/photos/thisisbossi/3361608344/
