Cognitive neuroscience approach to telemedicine - ?· Cognitive neuroscience approach to telemedicine…

Download Cognitive neuroscience approach to telemedicine - ?· Cognitive neuroscience approach to telemedicine…

Post on 18-Aug-2018

213 views

Category:

Documents

0 download

TRANSCRIPT

  • Tokyo, Japan, 4-5 February 2013

    Cognitive neuroscience approach to telemedicine

    Makio KASHINO, Ph.D. Senior Distinguished Scientist, Executive Manager

    NTT Communication Science Laboratories Kashino.makio@lab.ntt.co.jp

    ITU Workshop on E-health services in low-resource settings:

    Requirements and ITU role

    (Tokyo, Japan, 4-5 February 2013)

  • Tokyo, Japan, 4-5 February 2013 2

    Human Information Lab, NTT CSL

    01

    23453#678479:

    25:;73453#678479:

    7:

    !7?#7@A7347BC7

    01

    23453#678479:

    25:;73453#678479:

    7:

    !7?#7@A7347BC7

    Informatics

    Neuroscience

    Psychophysics

    Man-machine interface

    Perception, Emotion, Motor control

  • Tokyo, Japan, 4-5 February 2013 3

    Challenges in telemedicine

    ! Transmission delay and asynchrony

    ! Guiding a remote collaborator

    ! Face-to-face-like communication

  • Compensation for transmission delay

    Tokyo, Japan, 4-5 February 2013 4

    +

    "

    Takamuku, Gomi (2011) ECVP2011

    No delay With delay Delay+flowDelay Delay

    Heavy

    SmoothSmoothAwkward

  • Compensation for transmission delay

    Tokyo, Japan, 4-5 February 2013 5 Takamuku, Gomi (2011) ECVP2011

  • Compensation for transmission delay

    Tokyo, Japan, 4-5 February 2013 6 Takamuku, Gomi (2011) ECVP2011

  • Involuntary arm movement induced by vision

    Tokyo, Japan, 4-5 February 2013 7

    1717

    Movement duration: 1.0 sassisted by beeping soundsFixation marker

    0.05 [m]

    XY 0.0

    5 [m

    ]

    RLC

    110 ms

    85 ms

    91 ms

    norm

    aliz

    ed

    Saijo et al. 2005 J.Nneurosci.

    MFR (Manual Following Response)

    1717

    Movement duration: 1.0 sassisted by beeping soundsFixation marker

    0.05 [m]

    XY 0.0

    5 [m

    ]

    RLC

    110 ms

    85 ms

    91 msno

    rmal

    ized

    Saijo et al. 2005 J.Nneurosci.

    MFR (Manual Following Response)

    1717

    Movement duration: 1.0 sassisted by beeping soundsFixation marker

    0.05 [m]

    XY 0.0

    5 [m

    ]

    RLC

    110 ms

    85 ms

    91 ms

    norm

    aliz

    ed

    Saijo et al. 2005 J.Nneurosci.

    MFR (Manual Following Response)

    Saijo, Murakami, Nishida, Gomi (2005) The Journal of Neuroscience

    Manual Following Response (MFR)

    Involuntary Fast

  • Different visual processing systems for perception and motor control

    Tokyo, Japan, 4-5 February 2013 8 Gomi, Abekawa, Nishida (2006) The Journal of Neuroscience

    Fast motion of a coarse pattern is not salient perceptually, but effective to induce MFR

  • Audiovisual simultaneity

    Tokyo, Japan, 4-5 February 2013 9

    How to judge A-V timing in the presence of significant external and internal asynchrony?

  • Adaptation to A-V asynchrony induces

    Tokyo, Japan, 4-5 February 2013 10 Fujisaki, Shimojo, Kashino, Nishida (2004) Nature Neuroscience

  • Recalibration of A-V simultaneity

    Tokyo, Japan, 4-5 February 2013 11

    Sim

    ult

    aneo

    us

    res

    ponse

    A first V first

    N=7

    Test AV lag (ms)

    adaptation (0ms)

    adaptation (-235ms)

    adaptation (+235ms)

    Fujisaki, Shimojo, Kashino, Nishida (2004) Nature Neuroscience

  • Tokyo, Japan, 4-5 February 2013 12

    Challenges in telemedicine

    ! Transmission delay and asynchrony

    ! Guiding a remote collaborator

    ! Face-to-face-like communication

  • Perceptual attraction force: Buru-Navi

    Tokyo, Japan, 4-5 February 2013 13

    Amemiya, Gomi (2012) IEEE Transactions on Haptics

    Amemiya, Ando, Maeda (2007) Laval Virtual 2007

    How to generate pull or push sensation in the mobile devices having no external fulcrum?

  • Navigation by Buru-Navi

    Tokyo, Japan, 4-5 February 2013 14 Amemiya, Sugiyama (2009) ASSETS

    Successfully navigated ~90% (21/23) of visually-impaired people in a maze

  • Tokyo, Japan, 4-5 February 2013 15

    Challenges in telemedicine

    ! Transmission delay and asynchrony

    ! Guiding a remote collaborator

    ! Face-to-face-like communication

  • Implicit InterPersonal Information

    Tokyo, Japan, 4-5 February 2013 16

    Supported by JST CREST (PI: Makio Kashino [NTT], Collaborators: Sinsuke Shimojo [Caltech], Katsumi Watanabe [U Tokyo])

    !"#

    !$#

    !%#

    !

    !'#

    !(#

    )#

    (#

    '#

    %#

    $#

    "#

    *#

    +#

    ,#

    ()#

    )# (# '# %# $# "# *# +#

    -./#

    01#

    )#

    ()#

    ')#

    &)#

    %)#

    $)#

    ")#

    )# (# '# %# $# "# *# +#

    -./#

    01#

    !"#

    !$#

    !%#

    !

    !'#

    !(#

    )#

    (#

    '#

    %#

    $#

    "#

    )# (# '# %# $# "# *# +#

    !"#$%&'()*+),#!(-'.)

    01#

    !"#

    !$#

    !%#

    !

    !'#

    !(#

    )#

    (#

    '#

    %#

    $#

    "#

    )# (# '# %# $# "# *# +#

    ,#!(-'.)*+),#!(-'.)

    01#

    Gamma, 40-60 Hz (650ms) Theta, 5-8 Hz (500ms)

    ERPs after Face1 presented (200-400ms)

    Wavelet-based time-freq. plot, after Face2 presented

    All before decision! (Lindsen, et al., NeuroImage, '10)

    Two face presented sequentially.

    22

    Caltech Caltech

    IIPI (interpersonal interaction of involuntary and subconscious body movements and physiological responses) may play critical roles in communication Decoding and controlling IIPI for better communication systems

    Interpersonal interaction of brain and ANS activities

    Interpersonal interaction of body movements

  • Acoustic IIPI for talker presence

    Tokyo, Japan, 4-5 February 2013 17

    Kobayashi, Ooishi, Enomoto, Kitagawa, Ueno, Ise, Kashino (in preparation)

    Spontaneous utterances recorded and reproduced by the sound field sharing system BoSC (CREST Ise team)

    Subjective rating, ANS activity, hormone concentration

    Dynamic Static

  • Acoustic IIPI for talker presence

    Tokyo, Japan, 4-5 February 2013 18

    Kobayashi, Ooishi, Enomoto, Kitagawa, Ueno, Ise, Kashino (in preparation)

    n Significant differences also found for sympathetic nerve activity and salivary hormone concentration

    n Subjective rating 1. Artificial sound? 2. Feel talker presence? 3. Like the talker? 4. Interested in the content?

  • Concluding remark

    Tokyo, Japan, 4-5 February 2013 19

    ! Challenges in telemedicine may be overcome by taking advantage of brain mechanisms underlying human perception, emotion, motor control, and communication.

Recommended

View more >