web2 0-summit-jonas-distrib(2)
TRANSCRIPT
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 1/41
1
Cops & RobbersLas Vegas Style
Jeff Jonas, Chief Scientist, IBM Entity Analytics
Blogging at www.JeffJonas.TypePad.com
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 2/41
2
Background
Founded Systems Research & Development(SRD) in 1983
Moved to Las Vegas in early 90’s
Assisted gaming industry in betterunderstanding who they were doing businesswith (e.g., the MIT Team)
Acquired by IBM January 2005Now Chief Scientist of IBM Entity Analytics
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 3/41
3
My Living Room – December 31, 2005
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 4/41
4
Cheating Las Vegas
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 5/41
5
The “Cold” Deck : $250,000 Gone in 15 Minutes
[Video Redacted]
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 6/41
6
More AboutCorporate Amnesia
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 7/417
ProspectDatabase
EmployeeDatabase
HumanResourcesDepartment
CorporateSecurity
Department
InvestigationsDatabase
Marketing department is mailing offers to a
person currently in jail for stealing from you!
MarketingDepartment
Perception Isolation … Produces Corporate Amnesia
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 8/418
Enterprise Intelligence
RequiresPersistent Context
The Brain!
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 9/419
For Example
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 10/41
10
Marc R Smith123 Main St713 730 5769
M. Randal SmithDOB: 06/07/74
713 731 5577
Mark Randy Smith123 Main Street713 731 5577
ProspectDatabase
FraudDatabase
Record #A-701The Query
Record #B-9103
SensorsObservations
Consider the Query Against these Observations
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 11/41
11
Marc R Smith123 Main St713 730 5769
M. Randal SmithDOB: 06/07/74
713 731 5577
Mark Randy Smith123 Main Street713 731 5577
The Query Record #A-701
Record #B-9103
ProspectDatabase
FraudDatabase
SensorsObservations
Some Observations are Discoverable
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 12/41
12
Marc R Smith123 Main St713 730 5769
M. Randal SmithDOB: 06/07/74
713 731 5577
Mark Randy Smith123 Main Street713 731 5577
The Query Record #A-701
Record #B-9103
ProspectDatabase
FraudDatabase
SensorsObservations
Other Observables … are Undiscoverable
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 13/41
13
M. Randal SmithDOB: 06/07/74
713 731 5577
Mark Randy Smith123 Main Street713 731 5577
FEATURES:Mark Randy Smith, M. Randal Smith
123 Main Street, 713 731 5577DOB 06/07/74
EVENTS:
Internet InquiryArrest
ReconstructedIdentities
If You First Construct Context (Features and Events)
SensorsObservations
Record #A-701
Record #B-9103
ProspectDatabase
FraudDatabase
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 14/41
14
PersistentContext
Mark
FEATURES:Mark Randy Smith, M. Randal Smith
123 Main Street713 731 5577
DOB 06/07/74
Accumulating and Persisting this Context
M. Randal SmithDOB: 06/07/74
713 731 5577
Mark Randy Smith123 Main Street713 731 5577
SensorsObservations
Record #A-701
Record #B-9103
ProspectDatabase
FraudDatabase
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 15/41
15
Marc R Smith123 Main St713 730 5769
Mark Randy Smith123 Main Street713 731 5577
Record #A-701
M. Randal SmithDOB: 06/07/74
713 731 5577
Record #B-9103
Queries
Now the Un-discoverable …
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 16/41
16
Mark Randy Smith123 Main Street713 731 5577
Record #A-701
M. Randal SmithDOB: 06/07/74
713 731 5577
Record #B-9103
FEATURES:Mark Randy Smith, Randal Smith123 Main Street
713 731 5577DOB 06/07/74
PersistentContext Observations
… After Accumulating and Persisting Context …
Marc R Smith123 Main St713 730 5769
Queries
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 17/41
17
Marc R Smith123 Main St713 730 5769
FEATURES:Mark Randy Smith, Randal Smith123 Main Street
713 731 5577DOB 06/07/74
Mark Randy Smith123 Main Street713 731 5577
Record #A-701
M. Randal SmithDOB: 06/07/74
713 731 5577
Record #B-9103
Queries PersistentContext Observations
Enables Enterprise Discovery
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 18/41
18
Marc R Smith123 Main St713 730 5769
FEATURES:Mark Randy Smith, Randal Smith123 Main Street
713 731 5577DOB 06/07/74
Mark Randy Smith123 Main Street713 731 5577
Record #A-701
M. Randal SmithDOB: 06/07/74
713 731 5577
Record #B-9103
Queries PersistentContext Observations
Enables Enterprise Discovery
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 19/41
19
Marc R Smith123 Main St713 730 5769
The query could be:- A user with a question
Or, also could be data:- An account opening- A new watch list entry- A background check- An address change- A vendor application- A customer inquiry
Queries
EXCEPT: Always Treat Data as a Query
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 20/41
20
1st principle
If you do not process every newpiece of key data (perception)
first like a query … then you willnot know if it matters … until
someone asks.
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 21/41
21
Prospect
Database
EmployeeDatabase
HumanResourcesDepartment
CorporateSecurity
Department
Investigations
Database
“The Data is a Query” Beats “Boil the Ocean”
Midnight Batch Analytics?
MarketingDepartment
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 22/41
22
Emile SwelterSan Francisco12/03/72
Mark Randy Smith123 Main Street713 731 5577
Record #A-701
M. Randal Smith
DOB: 06/07/74713 731 5577
Record #B-9103
?
Queries PersistentContext
Observations
And … Any Query can be Treated as Data …
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 23/41
23
Mark Randy Smith123 Main Street713 731 5577
Record #A-701
M. Randal Smith
DOB: 06/07/74713 731 5577
Record #B-9103
PersistentContext
ObservationsQueries
Emile SwelterSan Francisco12/03/72
… In Which Case the Query can Stick (Persist)
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 24/41
24
PersistentContext
Notable, Stick in the Same Data Space
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 25/41
25
Question answeredwhen it becomes true!
Emilee Swelter
321 Ovington PlaceSan Francisco03/12/72
New ObservationPersistentContext
Emile SwelterSan Francisco12/03/72
Queries
Now, New Observations Answer Persistent Queries
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 26/41
26
2nd principle
Treat queries like data to avoidasking every question every day.
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 27/41
27
M. Randal SmithDOB: 06/07/74
713 731 5577
Mark Randy Smith123 Main Street713 731 5577
ProspectDatabase
FraudDatabase
Record #A-701
Record #B-9103
ObservationsPersistentContext
Mark
FEATURES:Mark Randy Smith, Randal Smith
123 Main Street713 731 5577
DOB 06/07/74
Sensors
This is Context Construction (A Librarian Function)
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 28/41
28
M. Randal SmithDOB: 06/07/74
713 731 5577
Mark Randy Smith123 Main Street713 731 5577
ProspectDatabase
FraudDatabase
Record #A-701
Record #B-9103
ObservationsPersistentContext
Mark
FEATURES:Mark Randy Smith, Randal Smith
123 Main Street713 731 5577
DOB 06/07/74
Sensors
!
The Ideal Moment for Enterprise Awareness
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 29/41
29
3rd principle
Enterprise awareness iscomputationally most efficient
when performed at the momentthe observation is perceived.
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 30/41
30
The “data finds the data” …
and “relevance finds the user.”
Towards Enterprise Intelligence
Introducing Perpetual Analytics
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 31/41
31
Real Technology
Scalable to >3Bhistorical observations
while handling >2,000real-time perceptions a
second
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 32/41
32
Privacy and Civil Liberties – Policy Think
What perceptions can or shouldbe placed into context (in onebrain)?
What if someone steals the
brain?
What if the librarian is corrupt?
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 33/41
33 The Brain!
EmployeeDatabase
HumanResourcesDepartment
Analytics in the Anonymized Data Space
Mark Randy Smith
Cd5dced41028cb …
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 34/41
34
The Main Think – Towards Enterprise Intelligence
Without persistent context you have nobrain
Treat data and queries with equal rights
More intelligence possible when thinking onstreaming perceptions
More or less perceptions, that is thequestion
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 35/41
35
Battling Corporate Amnesia is Broadly Useful
National security
Financial services
Health care
Heavily focused on threat and fraud
intelligence
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 36/41
36
Cops & RobbersLas Vegas Style
Jeff Jonas, Chief Scientist, IBM Entity Analytics
Blogging at www.JeffJonas.TypePad.com
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 37/41
37
Bonus Section!
If 6%
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 38/41
38
If a .6%difference
matters thismuch…
… no wondertraditionalinformation
systems lack somuchintelligence!
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 39/41
39
FEATURES:Mark Randy Smith, Randal Smith123 Main Street
713 731 5577DOB 06/07/74
FEATURES:Mark Randy Smith, Randal Smith, Randy Smith123 Main Street, Flat 6 20 Lennox Gardens
713 731 5577, 796 064 03 04DOB 06/07/74, Passport: 001003429002
2 Observations 6 Observations
More ObservationsMore
More Observations = Better Context
8/14/2019 Web2 0-Summit-Jonas-distrib(2)
http://slidepdf.com/reader/full/web2-0-summit-jonas-distrib2 40/41
40
Stable(e.g., Analytics withSequence Neutrality)
Data Loading Over Time
P e r c e n t
o f
E r r o r Reload #11 Reload #12
Unstable(e.g., data warehousingwhich requires periodic
reloads to handle data drift)
Drift
Sequence Neutrality is Critical for Context Stability