“I'm sorry, Dave, I'm afraid I can't do that”: Can computers really

Download “I'm sorry, Dave, I'm afraid I can't do that”: Can computers really

Post on 13-Feb-2017

217 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Im sorry, Dave, Im afraid I cant do that:

    Can computers really understand what we say?

    Cornell summer evening lecture, July 2012

    Lillian Lee

    Professor, Computer Science

    http://www.cs.cornell.edu/home/llee

  • the dream

  • Why is this man smiling?

    http

    ://w

    ww

    .nat

    ure.

    com

    /nat

    ure/

    jour

    nal/v

    482/

    n738

    6/fu

    ll/48

    2440

    a.ht

    ml

  • The Turing test: Intelligence human-level language use

    http

    ://bi

    tter

    swee

    tsag

    e.bl

    ogsp

    ot.c

    om/2

    010/

    01/c

    omic

    -con

    vers

    e-tu

    ring

    -tes

    t.htm

    l

    Turing predicted wed be close in about 50 years.

    ]http://ghostradio.files.wordpress.com

    /2011/03/blade_runner_fondo.jpg

  • http

    ://w

    ww

    .blo

    gcdn

    .com

    /ww

    w.tu

    aw.c

    om/m

    edia

    /201

    0/04

    /jarv

    ism

    ac.jp

    g

    http

    ://w

    ww

    .nav

    tone

    s.co

    m/m

    edia

    /imag

    e/ca

    ched

    _kni

    ght_

    ride

    r_ki

    tt.jp

    g

    http

    ://up

    load

    .wik

    imed

    ia.o

    rg/w

    ikip

    edia

    /en/

    0/09

    /Dat

    aTN

    G.jp

    g

    Do authors dream of electric speech?

    Jarvis, the A.I. system in Iron Man

  • Why is this man not smiling?

    http

    ://w

    ww

    .net

    braw

    l.com

    /mat

    chup

    .php

    ?mid

    =11

    131&

    brac

    ketid

    =49

    7

    http

    ://4.

    bp.b

    logs

    pot.c

    om/_

    Qm

    9Cek

    v5Jj4

    /S8I

    q3em

    ehgI

    /AA

    AA

    AA

    AA

    ARU

    /oBZ

    6Ih5

    J4fI/

    s200

    /200

    1-a-

    spac

    e-od

    ysse

    y.jpg

    Open the pod bay doors, Hal.

    Im sorry, Dave, Im afraid I cant do that.

  • from sci-fi to science and engineering

  • Goal: create systems that use human language as input/output

    speech-based interfaces

    information retrieval / question answering

    automatic summarization of news, emails, postings, etc.

    automatic translation

    and much more!

    Interdisciplinary: computer science; linguistics, psychology, communication; probability & statistics, information theory

    Natural-language processing (NLP)

  • Recently deployed (in beta): Siri

    http://www.apple.com/iphone/features/siri.html

  • State of the art: Watson

    C

    redi

    t: A

    P Ph

    oto/

    Jeop

    ardy

    Pro

    duct

    ions

    Inc.

    The Watson system beat human Jeopardy! champions (and didnt have internet access; it learned by reading before the match)

  • Why is this man smiling?

    Charles (Bud) Jermy, Jr., host of this lecture series

  • But were not all the way there yet

  • Real-life error (1)

    Hey bunch of grapes

    isto

    ck |

    blan

    kabo

    skov

    A bunch of grapes.

    ht

    tp://

    rand

    omha

    ndpr

    ints

    .blo

    gspo

    t.com

    /201

    1_01

    _01_

    arch

    ive.

    htm

    l

  • Real-life error (2)

    We can email you when you're fat.

    We can email you when we're back.

    http

    ://ca

    tand

    girl

    .com

    /?p=

    2678

    isto

    ck |

    blan

    kabo

    skov

  • Real-life error (3)

    [This U.S. citys] largest airport

    What is Toronto???

    http

    ://je

    opar

    dy.e

    dogo

    .com

    /wp-

    cont

    ent/

    uplo

    ads/

    2009

    /01/

    prog

    ram

    -jeop

    ardy

    1.jp

    g

  • why is understanding language so hard?

  • List all flights on Tuesday

    Challenge: ambiguity

    List all flights on Tuesday = List all the flights leaving on Tuesday.

    List all flights on Tuesday = Wait til Tuesday, then list all flights.

  • Retrieve all the local patient files

    More realistic example

  • Baroque example

    I saw her duck with a telescope.

    [http://www.supercoloring.com/pages/duck-outline/]

    [http://casablancapa.blogspot.com/2010/05/fore.htm]l

  • Baroque example

    I saw her duck with a telescope.

    [http://www.supercoloring.com/pages/duck-outline/]

    http

    ://w

    ww

    .clip

    artm

    ojo.

    com

    /plu

    gins

    /Clip

    art/

    Clip

    artS

    tock

    1/st

    ar%

    20ga

    zing

    .png

    http

    ://w

    ww

    .geo

    citie

    s.w

    s/lo

    oney

    ebay

    /del

    l/bb0

    40.jp

    g

    http://pokerfoldingtable.com/wp-content/uploads/2009/02/three-men-gambling-sitting-at-poker-table-playing-cards-betting-party-pen-ink-drawing-300x234.png

  • Conversation complications

    [Grishman 1986]

    Q: Do you know when the train to Boston leaves?

    A: Yes.

    Q: I want to know when the train to Boston leaves.

    A: I understand.

    Images: http://3.bp.blogspot.com/_o4kq5TNL0Z4/TUx0j6E5BLI/AAAAAAAAA5k/J7xjhvrcNlU/s1600/Trillian-hitchhikers-guide-to-the-galaxy-the-2005.jpg, http://www.tvacres.com/images/robots_androids_marvin_movie.jpg

  • [http

    ://br

    owse

    .dev

    iant

    art.c

    om/?

    qh=

    &se

    ctio

    n=&

    glob

    al=

    1&q=

    mus

    cled

    uck#

    /d14

    nst5

    ]

    Im sorry, Dave, Im afraid I cant do that.

    Im afraid you might be right.

    [htt

    p://s

    elco

    uth.

    com

    /201

    1/03

    ]

  • Meeting these challenges: a brief history

  • 1940s 50s: From language to probability

    The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point ...

    [The] semantic aspects of communication are irrelevant to the engineering problem.

    The significant aspect is that the actual message is one selected from a set of possible messages.

    --C. Shannon, 1948

  • Language, statistics, cryptography

    WWII: Turing helps break the German Enigma code

    (An original Enigma machine for encrypting messages is on display now in the Kroch Library in Olin.)

  • Why is this man smiling?

    ht

    tp://

    arto

    frev

    olut

    ion.

    co.u

    k/ne

    w/in

    dex.

    php?

    mai

    n_pa

    ge=

    prod

    uct_

    info

    &cP

    ath=

    1_3&

    prod

    ucts

    _id=

    239&

    zeni

    d=vi

    d94s

    pfpa

    9vtr

    18sb

    tgug

    1h64

    I can see Alaska from my house!

    Encryption process

    [W. Weaver memo on translation, 1949]

  • Two probabilities to infer

    I can see Alaska from my house!

    Encryption process

    [Russian]

    Prob. of generating this original message?

    Prob. of doing this encryption of the original?

  • Another use of message probs: speech recognition

    (1) Its hard to recognize speech

    (2) Its hard to wreck a nice beach

    Both messages have almost the same acoustics, but different likelihoods.

  • 1950s-1980s: Breaking with statistics

    (a) Colorless green ideas sleep furiously

    (b) Furiously sleep ideas green colorless

    N. Chomsky (1957):

    The argument: Neither sentence has ever occurred in the history of English. So any statistical model would given them the same probability (zero).

    The field moved to sophisticated non-probabilistic models of language.

  • 1990s: The empiricists strike back

    Huge amounts of data start coming online

    Advances in algorithms and computational power

    Every time I fire a linguist, my [systems] performance goes up -- F. Jelinek (apocryphal)

  • 2000s and beyond: integrating language insights and

    statistical techniques

    [All 8 results were from March 2011 or earlier]

    Is Snooki on stork watch?

    (wondered in March 2012)

  • Integrating lang and stats (cont)

    Snooki and fianc Jionni LaValle are expecting their first child together

    Angie Harmon on Stork Watch By Marcus Errico

    Angie Harmon's going from assistant district attorneying to diaper duty. The former Law & Order legal dish is expecting her first child with football stud hubby Jason Sehorn, her publicist confirmed Tuesday.

    Bowie & Iman On Stork Watch BY GEORGE RUSH DAILY NEWS COLUMNIST Monday, February 14, 2000 Rock legend David Bowie and supermodel Iman said yesterday they're expecting their first child

    Is Snooki on stork watch?

    Snooki?!!

  • the game-changers:

    data-driven approaches

    models of language

  • computers can also (help us) understand us

  • Why is this man smiling?

    C. Danescu-Niculescu-Mizil et al. ACL 2012

    Beyond situational effects, phrasing also affects memorability:

    memorable movie quotes (in aggregate) are unusual word choices built on a scaffolding of common part-of-speech patterns

    shown via language models

    carries over to ad slogans

    http

    ://w

    ww

    .sch

    wim

    mer

    lega

    l.com

    /200

    6/11

    /evi

    denc

    e-of

    -sec

    onda

    ry-m

    eani

    ng-in

    -tv-

    c