development virtual environments for fmri socially to interact with a virtual avatar ; a pilot study...
TRANSCRIPT
![Page 1: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/1.jpg)
Development Virtual Environments for fMRI Development Virtual Environments for fMRI Socially to Interact with a virtual avatar Socially to Interact with a virtual avatar ; a pilot study; a pilot study
Hyeongrae, Lee
Dept. of Biomedical Engineering, Hanyang University,
Seoul, Korea
![Page 2: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/2.jpg)
IntroductionIntroduction
![Page 3: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/3.jpg)
3
Introduction
• The social virtual environments (VEs)The social virtual environments (VEs)
VEsVEs Virtualavatar
Virtualavatar Social VEsSocial VEs
![Page 4: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/4.jpg)
4
Introduction
Schizophrenia
• The research and therapy using the social VEs The research and therapy using the social VEs
AlcoholismSocial phobia
Social VEsSocial VEsNeuroscienceNeuroscience
![Page 5: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/5.jpg)
5
Displaydevice
Trackingdevice
Tactiledevice
Conventional device
…
Introduction
• The functional MRI environments
MR room
High magnetic and electronic fields
For social VEs, the other method is needed!!!
![Page 6: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/6.jpg)
6
Introduction
•In this study, we purpose that In this study, we purpose that
1.1.development the social VEs for social cognitive neuroscience development the social VEs for social cognitive neuroscience
study using MR compatible devices.study using MR compatible devices.
2.2.a pilot study performed on the developed social VEs.a pilot study performed on the developed social VEs.
![Page 7: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/7.jpg)
MethodsMethods
![Page 8: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/8.jpg)
8
Outside of magnet roomInside of magnet room
Real-timevideo image
Marker detectionon the hand
Computer
Filter panel
Calculatemaker position
Apply hand positionto virtual environments
MR compatible HMD
MR compatible camera
marker
development the social VEs
• fMRI environment for social interactionfMRI environment for social interaction
![Page 9: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/9.jpg)
9
development the social VEs
• Contents for the social VEsContents for the social VEs
(b) Acceptance
(a) Rejection
![Page 10: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/10.jpg)
10
development the social VEs
![Page 11: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/11.jpg)
11
A pilot study A pilot study
Normal groupNormal group
( n = 12 )( n = 12 )
AgeAge 20.92 (SD: 1.78)20.92 (SD: 1.78)
Sex (M:F)Sex (M:F) 5 : 75 : 7
• ParticipantsParticipants
![Page 12: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/12.jpg)
12
A pilot study
• fMRI experiment design
![Page 13: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/13.jpg)
13
A pilot study
• Questionnaires of social realism about the other avatar
(1) How real the other avatar’s action was?(2) Did you feel that you really shook hands with (or was
rejected by) the virtual other ?
• Questionnaires of social cognition about the other avatar
(1) Did you feel positive feeling about the other avatar’s response?
(2) Did you feel negative feeling about the other avatar’s response?
(3) Did you think that other avatar’s response was suitable in the situation?
(4) Did you feel that the other avatar rejected your offer?(5) Did you feel that the other avatar hated you or had
negative emotion about you?(6) Did you feel that the other avatar took threatening
attitude to you?(7) Did you feel that the virtual other ignored or excluded
you?(8) Did you feel that the virtual other was aware of you?
• Questionnaire (5 point response scale )Questionnaire (5 point response scale )
![Page 14: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/14.jpg)
14
A pilot study
• fMRI data acquisitionfMRI data acquisition - The functional MRI conducted with 1.5T machine (GE Medical System).
• BOLD (Blood Oxygenate Level Dependent)
•EPI sequence, Gradient Echo•64x64x30 matrix with 3.75x3.75x5-mm spatial resolution, FOV: 240mm,•Slice thickness: 5mm, TR: 2s, TE: 14.3, FA=90, # of slices: 30
• High resolution anatomical images
•Fast Spoiled Gradient Echo sequence•256x256x116 matrix with 0.94x0.94x1.50-mm spatial resolution, FOV: 240mm,•Slice Thickness: 1.5mm, TR: 8.5s, TE: 1.8s, FA: 12, # of slices: 116).
![Page 15: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/15.jpg)
15
CoregistrationCoregistration
Temporal NormalizationTemporal Normalization
3D construction3D construction
Preprocessing
Preprocessing
Temporal Smoothing(3-point Low pass filter )
Temporal Smoothing(3-point Low pass filter )
Spatial Normalization(MNI N27 templete,
2x2x2mm3)
Spatial Normalization(MNI N27 templete,
2x2x2mm3)
Spatial Smoothing(Gaussian filter, 8 mm
FWHM)
Spatial Smoothing(Gaussian filter, 8 mm
FWHM)
- Analysis of Functional NeuroImages (AFNI, Ver. 2006_06_30_1332)- subjects whose haed movements are more than 1mm are excluded
A pilot study
Statistical Analysis
Statistical Analysis
General Linear ModelAnalysis in each subjectGeneral Linear Model
Analysis in each subject
Group Analysis
Group Analysis
Paired t-test within group(uncorrected P < 0.001)
Paired t-test within group(uncorrected P < 0.001)
• fMRI analysisfMRI analysis
![Page 16: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/16.jpg)
Results & Results & DiscussionsDiscussions
![Page 17: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/17.jpg)
17
- Social realism about other avatars
Results & Discussions
(1) How real the other avatar’s
action
(2) really shake hands with (or
was rejected by) the virtual
other ?
• The questionnaire results.The questionnaire results.
- 1 point : not at all 5 point : very much
![Page 18: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/18.jpg)
18
- social cognition about other avatars
Results & Discussions
• The questionnaire results.The questionnaire results.
(1) positive feeling(2) negative feeling(3) other avatar’s response was
suitable(4) the other avatar rejected
participant offer(5) hated participant or had
negative emotion(6) threatening attitude(7) ignored or excluded participant(8) awareness of participant
- 1 point : not at all 5 point : very much
![Page 19: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/19.jpg)
19
Results & Discussions
uncorrected P <0.001, cluster size > 80 microliter
• Acceptance > Rejection
L thalamus (-11, -17, 8)R insula lobe (37, -3, 0)
• Rejection > Acceptance
L prefrontal gyrus (-17, 53, 20) R rolandic operculum (37, -3, 20)
L superior temporal sulcus (-37, -31, 6)L cerebelum (VI) (-25, -45, -22)
![Page 20: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/20.jpg)
20
Results & Discussions
• Acceptance > Rejection
L thalamus
R insula lobe
• mirroring other’s emotion related to empathy
• Social cognition related friendship and teamwork
![Page 21: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/21.jpg)
21
Results & Discussions
• Rejection > Acceptance
L prefrontal gyrus
L superior temporal sulcus
• understanding other’s mental states• Social emotion (ex. embarrassment)
• Social emotion (ex. embarrassment)
![Page 22: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/22.jpg)
22
In this study, we develop the VEs for social cognitive In this study, we develop the VEs for social cognitive neuroscience study that it is possible to interact with the neuroscience study that it is possible to interact with the virtual avatars. virtual avatars.
This social VEs would provide social environment to This social VEs would provide social environment to patients with deficit in social cognition, such as patients patients with deficit in social cognition, such as patients with schizophrenia, social phobia and etc., even though just with schizophrenia, social phobia and etc., even though just only handshake with other.only handshake with other.
The social interaction methods using maker would be The social interaction methods using maker would be practicable in various social cognitive neuroscience studies.practicable in various social cognitive neuroscience studies.
Summary
![Page 23: Development Virtual Environments for fMRI Socially to Interact with a virtual avatar ; a pilot study Hyeongrae, Lee Dept. of Biomedical Engineering, Hanyang](https://reader035.vdocuments.site/reader035/viewer/2022062802/56649e995503460f94b9bee7/html5/thumbnails/23.jpg)
23
Prof.Prof.
Ph.D Ph.D candidatecandidate
M.S candidateM.S candidate
Hyeongrae Lee Neuro-mechanism On Virtual Environment Lab.
Dept of Biomedical Engineering In Hanyang University
E-mail: [email protected]: http://bme.hanyang.ac.kr
FAX: +82-2-2296-5943 Tel: +82-2-2290-0691