From  Jeopardy! to  the  Future

Eric  Brown,  PhD  Director,  Watson  AlgorithmsIBM  Watson  Health

Rob  High,  Jr.IBM  Fellow,  Vice  President,  CTOIBM  Watson

Please  note IBM’s statements  regarding  its  plans,  directions,  and  intent  are  subject  to change  orwithdrawal without  notice  and  at  IBM’s sole  discretion.

Information  regarding potential  future  products  is  intended  to  outline  our  general  productdirection  and  it  should  not  be  relied  on  in  making  a  purchasing  decision.

The  information  mentioned regarding  potential  future  products  is  not  a  commitment,  promise,  or legal  obligation  to  deliver  any  material,  code  or functionality.  Information  about  potential  futureproducts  may not  be  incorporated  into  any  contract.

The development,  release,  and  timing  of  any  future  features  or functionality  described  for  our  products  remains  at  our  sole  discretion.

Performance  is  based  on measurements  and  projections  using  standard  IBM  benchmarks  in  a controlled  environment.  The  actual  throughput  or  performance  that  any user  will  experience  will vary  depending  upon  many  factors,  including  considerations such  as  the  amount  of  multiprogramming  in  the  user’s  job  stream,  the I/O  configuration,  the  storage  configuration,  and  theworkload processed.  Therefore,  no  assurance  can  be  given  that  an  individual user  will  achieve  results  similar  to  those  stated  here.

The History of Watson

Eric  Brown,  PhD  Director,  Watson  AlgorithmsIBM  Watson  Health

A  Brief  History  of  AI

1945

Memex:VannevarBush

1970 1980 1990 2000 2010

Natural  Language  ProcessingInformation  RetrievalMachine  Learning

Knowledge  Representation  and  ReasoningQuestion  Answering

Dartmouth  Conference  of  

1956

State  of  the  Art  in  Question  Answering

1st AI  Winter

Reasoning  as  SearchNatural  Language  Understanding

LogicNeural  Networks

2nd AI  Winter

Expert  SystemsKnowledge  and  Reasoning

Text  REtrievalConference

Big  DataComputational  Power

Real  Language  is  Real  Hard

Chess– A  finite,  mathematically  well-­defined  search  space– Limited  number  of  moves  and  states– Grounded  in  explicit,  unambiguousmathematical  rules

Human  Language– Ambiguous,  contextual  and  implicit– Grounded  only  in  human  cognition– Seemingly  infinite number  of  ways  to  express  the  same  meaning

$200The  juice  of  this  bog  fruit  is  sometimes  used  to  treat  urinary  tract  infections

$400

An  early  name  for  Google  was  this  type  of  massage

The  Jeopardy!  Challenge  

Broad/Open  Domain

Complex  Language

High  Precision

Accurate  Confidence

HighSpeed

$600In  cell  division,  mitosis  splits  the  nucleus  &  cytokinesis  splits  this  liquid  cushioning

the  nucleus

$800Grace  Murray  Hopper  is  credited  with  applying  this  3-­letter  term  to  a  mysterious  computer  problem

$1000

IBM  is  known  informally  as  Big  this

A  palpable,  compelling  and  notable  way  to  drive  the  technology  of  Question  Answering  along  Key  Dimensions

What  It  Takes  to  compete  against  Top  Human  Jeopardy!    PlayersOur  Analysis  Reveals  the  Winner’s  Cloud

Winning  Human  Performance

2007  QA  Computer  System

Grand  Champion  Human  Performance

Top  human  players  are  remarkably  good.

Each  dot  – actual  historical  human  Jeopardy!  games

More  Confident Less  Confident

In  2007,  we  committed  to  making  a  Huge  Leap!

Computers?Not  So  Good.

Large  Hand-­Crafted  Models  won’t  cut  it– Too  Slow,  Too  Narrow,  Too  brittle,  Too  Biased– Need   to  acquire  and  analyze  information  from  As-­Is  Knowledge  sources

Intelligence  from  the  combination  of  many– Consider  Many  Hypotheses.  Reduce  early  biases.  – Consider  Many  diverse  algorithms.  No  single  one   is  perfect  or  complete.– Analyze  evidence  form  different  perspectives– Balanced  combination  is  continually  learned,   tested  and  refined

Architecture  to  Support  Integration  and  Learning– Facilitate  integration  of  many  components– Automatically  learn  how  to  combine– Enable  scale-­out  and  parallel  deployment

Drive  Performance  with  Measurement  and  Analysis– Define  appropriate  metrics– Measure,  analyze,  improve,  repeat

Early  Philosophical   Commitments

DeepQA:  The  architecture  underlying  Watson

Generates  many  hypotheses,  collects  a  wide  range  of  evidence  and  balances  the  combined  confidences  of  over  100  different    analytics  that  analyze  the  evidence  from  different  dimensions

Answer  Scoring

Models

Answer  &  Confidence

Question

Evidence  Sources

Models

Models

Models

Models

ModelsPrimarySearch

CandidateAnswer

Generation

HypothesisGeneration

Hypothesis  and  Evidence    Scoring

Final  Confidence  Merging  &  RankingSynthesis

Answer  Sources

Question  &  Topic  Analysis

EvidenceRetrieval

Deep  Evidence  Scoring

Learned  Modelshelp  combine  and  weigh  the  Evidence

HypothesisGeneration

Hypothesis  and  Evidence  Scoring

QuestionDecomposition

11

Goal-­Oriented  Metrics  and  Incremental  Investments– Identify  a  Target  and  Technical  Approach– Headroom  Analysis:  Estimate  idea’s  potential   impact  on  key  metrics– Balance   long-­term  &  short-­term   investments.  Have  the  next  priority  ready.  Be  Agile.

Extreme  Collaboration– Implemented   “One  Room”  to  optimize   team  work  and  communication– Immediate   access  to  the  right  “expert”,  spontaneous  discussions,  no  good   idea   lost

Disciplined  Engineering  and  Evaluation  (Regular  Blind  Data  Experiments)– Bi-­weekly  End-­to-­End   Integration   Runs  &  Evaluations   (Large  Compute  Resources)– >10  GBs  of  error  analysis  output  made  accessible  via  Web-­Based  Tool– Positive  impact  on  last  run  required   to  get  into   the  next  bi-­weekly  run

Rapid  Innovation  Methodology  Emerged

>8000  Documented  experiments  performed  in  4  years

DeepQA:  Incremental  Progress  in  Answering  Precision  on  the  Jeopardy  Challenge:  6/2007-­11/2010  

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Precision

%  Answered

Baseline  12/06

v0.1    12/07

v0.3  08/08

v0.5    05/09v0.6    10/09

v0.8    11/10

v0.4    12/08

v0.2    05/08

IBM  WatsonPlaying  in  the  Winners  Cloud

V0.7    04/10

With  Precision,  Accurate  Confidence  and  Speed,  the  rest  was  History

From  battling  humans  at  Jeopardy!  to  transforming  how  business  thinks,  acts,  and  operates

Contact Center

Healthcare Financial Services

Government

Diagnostic/treatment assistance, evidenced-based insights, collaborative medicine

Investment and retirement planning, institutional trading and decision support

Call center and tech support, enterprise knowledge management, consumer insight

Public safety, improved information sharing, security

The Era of Cognitive Computing

Rob  High,  Jr.IBM  Fellow,  Vice  PresidentChief  Technology  OfficerIBM  Watson

Cognitive  systems  amplify  human  cognition.

Cognitive  systems  understand human  expressions  – textual,  verbal,  visual

By  reasoning about   the  actual  intention  or  problem  being  addressed  

They   learn how  to  recognize  patterns  of  meaning   through  examples  and  feedback

And  they  interact with  humans  on  their  own  terms,  and  in  a  way  that  inspires  people.

What  is  Cognitive  Computing?

…  and  do  so  at  enormous  scale.  

Watson  Reference  Model

Developer Tooling

Platforms as a ServiceWatson Services

Data as a ServiceWatson Content

Application Tooling

Maker Tooling

Content Tooling

Cloud Infrastructure

Public Private Crowd Sourced

Knowledge  Organization  

Skills

Foundational  Cognitive  Skills

Higher  Reasoning  Skills

IBM

Wat

son

Mar

ketp

lace

Skills as a Service

Watson Built Apps

& Bots

Watson Built Skills

IBM Built Apps & Bots

IBM Built Skills

3rd Party Apps & Bots

3rd Party Skills

Software as a ServiceWatson Apps & Bots

Hybrid Client PlatformWatson Explorer

Watson  is  available  as  a  set  of  services  delivered  as  APIs  in  the  Cloud    

Higher Reasoning Skills• Conversation

Higher  Reasoning  SkillsIbm.com/bluemix

Can be combined with the 100s of other available services on Bluemix

• Add  a  natural  language  interface  to  your  application  to  automate  interactions  with  your  end  users.  

• Common  applications  include  virtual  agents  and  chat  bots  that  can  integrate  and  communicate  on  any  messaging  platform.

• Model  is  trained  on  user-­defined  intents,  entities  and  dialogs

• Expanded  to  recognize  the  emotion  of  the  user  and  to  respond  accordingly

Conversation

• Intents  classify  the  kinds  of  actions  or  questions  that  the  conversation  will  respond  to

• You  can  begin  with  as  few  as  just  one  intent,  and  then  expand  as  new  functionality  is  added  to  the  virtual  agent

• It  only  takes  a  few  examples  to  train  Watson  to  recognize  a  wide  variety  of  ways  to  express  an  intent

• The  tooling  allows  you  to  test  as  you  train

Conversational  Intents

• Conversation  can  detect  entities  in  an  utterance  and  identify  them  to  clarify  the  intent

• Typically  used  with  Dialogs   to  condition  the  response

Conversational  Entities

• Dialogs  can  be  created  to  control  the  flow  of  the  conversation  around  specific  intents

Conversational  Dialogs

Watson  is  available  as  a  set  of  services  delivered  as  APIs  in  the  Cloud  Watson  Foundational  Skills  are  grouped  into  four  categories

Ibm.com/bluemix

Foundational  Cognitive  Skills

Language• Author• Concepts• Dates• Entities

Speech• Speech to Text• Telephony Speech to Text• Keyword Spotting

Vision• Image Classification• Face Detection and Attribution• Celebrity Recognition

Empathy• Personality Insights• Tone Analyzer• Emotion Analysis

• Text to Speech• Expressive Text to Speech

• Relations• Typed Relations• Sentiment• Taxonomy• Text Extraction

• Feeds• Keywords• Language• Microformats• Publication Date

• Visual Text Recognition• Similarity Searching

Can be combined with the 100s of other available services on Bluemix

Tone  Analyzer  understands  and  helps  fine  tune  your  messageUses  psycholinguistics,  emotion  analysis  and  language  analysis  to  assess  Tone

Online   Dating  Profile

I'm  a  hard  working  adventurous,  very  talented  man  who's  been  caring  and  helpful  throughout  my  life,  I  like  to  travel,  play  my  guitar,  dance,  and  cook,  I  love  the  beach,  sailing  my  boat,  and  the  outdoors.  

I  raised  two  great  kids  and  now  I'm  starting  a  new  chapter  in  my  life.  

Thanks.

What  I’m  doing  with  my  lifeWorking  toward  a  new  goal,  keeping  fit,  helping  others,  and  traveling  whenever  i get  a  chance.

I’m  really  good  at  Listening,  enjoying  the  moment,  and  many  other  things.

The  six  things  I  could  never  do  without

Family,  the  ocean,  intimacy,  friends,  adventure,  music,  love.

On  a  typical  Friday  night  I  am

Meeting  with  friends,  listening  to  a  band  or  playing  my  guitar,  dancing  or  just  staying  home  with  someone  special  and  enjoying  each  other.

You  should  message  me  if

You're  looking  for  a  relationship  with  someone  that  likes  to  sail  his  boat,  ride  bicycles,  travel,  swim,  go  to  the  beach,  listen  to  music  and  enjoy  everyday  pleasures  together.

Emotional  Analysis  helps  build  empathetic  systemsUses  state-­of-­the-­art  machine  learning  models  and  feature  engineering  techniques  to  predict  emotion  labels

Personality Insights

Watson  Visual  Recognition  Service

Watson  Visual  Recognition  leveraged  to  enhance  the  game  of  Pokemon Go

Enables  speech  that  reflects  intended  tone– Expressive  SSML– Voice  Transformation  SSML

Expressive  Text  to  Speech

Watson  is  available  as  a  set  of  services  delivered  as  APIs  in  the  Cloud    

Knowledge Organization Skills• Watson Knowledge Studio• Document Conversion• Retrieve and Rank

Knowledge  Organization  

SkillsIbm.com/bluemix

Can be combined with the 100s of other available services on Bluemix

Enable  subject  matter  experts  and  developers  to  teach  Watson  the  linguistic  nuances  of  industries  and  knowledge  domains

Watson  Knowledge  Studio

Uses  machine  learning  to  improve  search  across  documents  using  natural  language  questions

Retrieve  and  Rank

Amplifying  Human  Cognition  Requires  that  we  affect  people  in  a  very  human  way

Human  can  be  influenced  by  cognitive  experiences– Something  that  creates  a  sense  of  presencewith  people– Something  that  inspires  people– Something  that  embodies  intellectual  cognition  with  behavioral  cognition

Project  Intu  is  a  Platform  for  creating  great  cognitive  experiences

avatars In-the-walls

devices

spaces wearables

robotics

Go  to  the  Project  Intu  workshop  to  get  access  to  the  experimental  platform  and  to  learn  how  to  program  Intu  to  create  great  cognitive  experiences

• Amplify  human  creativity- Inspiring  us  to  new  alternatives  to  decision  options- Bringing  the  breadth  of  all  human  knowledge  to  the  tip  of  our  tongue

• Learn  their  behavior  through  formal  and  informal  training  processes

• Interact  with  humans  on  our  terms  – in  the  language  of  humans

• Demonstrate  their  expertise  through  trust  and  depth  of  character

• Evolve  strategies  of  success  – adapting  to  ever  changing  knowledge  and  understanding

• Establish  transformative  relationships  between  humans  and  machines    

In  5  years,  cognitive  systems  will  be  to  computing  what  transaction  processing  is  today

ibmwatson.com        facebook.com/ibmwatson        @ibmwatson