what functional brain imaging reveals about the neuroarchitecture of object knowledge kai-min kevin...
TRANSCRIPT
What functional brain imaging reveals about the neuroarchitecture of object
knowledge Kai-min Kevin Chang, Vicente Malave, Svetlana Shinkareva, and Marcel Adam Just
Center for Cognitive Brain Imaging, Carnegie Mellon University
Categories Exemplars
body parts leg, arm, eye, foot, hand
furniture chair, table, bed, desk, dresser
vehicles car, airplane, train, truck, bicycle
animals horse, dog, bear, cow, cat
kitchen utensils glass, knife, bottle, cup, spoon
tools chisel, hammer, screwdriver, pliers, saw
buildings apartment, barn, house, church, igloo
part of building window, door, chimney, closet, arch
clothing coat, dress, shirt, skirt, pants
insects fly, ant, bee, butterfly, beetle
vegetables lettuce, tomato, carrot, corn, celery
man made objects refrigerator, key, telephone, watch, bell
refrigerator
celery
antX
X
3s
7s
time
OverviewWe used functional Magnetic Resonance Imaging to study the cortical systems that underpin semantic representation of object knowledge.
MethodParticipant was presented with black and white line drawings of 60 objects from a range of categories (see below) and were instructed to think of the same properties consistently during each presentation.
fMRI procedure:•Cerebral activation was measured using BOLD contrast (Kwong et al., 1992; Ogawa et al., 1990).•Functional images were acquired on a Siemens Allegra 3.0T scanner at the Brain Imaging Research Center.•EPI acquisition sequence, 17 oblique axial slices, TR=1000ms, TE=30ms, 64x64 acquisition matrix, 5 mm thickness, 1 mm gap, flip angle 60°.•Image were corrected for slice acquisition timing, and motion-corrected with Statistical Parametric Mapping software (SPM99, Wellcome Department of Cognitive Neurology, London, UK).•Data were normalized to the Montreal Neurological Institute (MNI) template, and resampled to 3x3x6 mm voxels for Machine Learning analysis.•Analyses of a single brain region at a time used region definitions derived from the Anatomical Automatic Labeling (AAL) system (Tzourio-Mazoyer et al., 2002).
Machine Learning
EvaluationClassification results were evaluated using k-fold cross validation, where one example per class was left out for each fold. Both raw and rank accuracies are reported.
4CAPS ModelingWe implemented a set of neural processing regions in the 4CAPS neural architecture (Just & Varma, in press), along with their corresponding specialization to process each feature. The 4CAPS model was used to simulate human performing the picture-naming task.
TheoreticalEach object is associated with a set of distinguishing sensory/functional features (e.g. visual-motion, visual-parts, function, etc.) according to Cree & McRae (2003)'s semantic feature norming studies. The sensory/functional features are thought to be processed by a set of neural processing regions that are mostly agreed by the current state of the literature. Correlation = 0.5265
Data-drivenCan we instead learn the set of activated neural processing centers from our data? Yes, but we need to check if it is generailzable, otherwise we may overfit to the data.
EvaluationThe activation pattern of the neural processing regions in 4CAPS is correlated to human's cortical activation pattern reading the same word.
Mean PSC matrixwords x voxels
Preprocessed fMRI datatime x voxels
Construct mean Percent Signal Change (PSC) matrix per stimulus event averaged over 4 successive images
Full training setwords training x voxels Full test set
words test x voxels
Training setwords training x voxels select
Test setwords test x voxels select
Partition data into training and test sets
Select Voxels (features)
Evaluate classification
Subset of voxels selected from Training set
Train classifier to identify categories based on
selected features
Get classification accuracy
Alternative Feature (voxel)
Selection Methods
Select individually most discriminating voxels:
Threshold Number of Misclassifications
select voxels with the minimum number of misclassifications made by the best threshold chosen for each voxel (Ben-Dor et al., 2000)
Wilcoxon select voxels with highest statistic value
Select conjointly most discriminating voxels:
Logistic regression select voxels with highest absolute value of the regression weights
Select most stable voxels :
Stability of differential response for wordsselect voxels with the largest average
pairwise correlations across the word presentation
2 Alternative Classifiers:
Gaussian Naïve Bayes
Uses Bayes rule to estimate the probability distribution from the training set. Makes conditional independence assumption of the features.
Logistic Regression
Uses parametric form to directly fit the distribution P(Y|X).
Classification performance is evaluated using k-fold cross validation
ConclusionWe have shown that high classification accuracies can be obtained for distinguishing the 12 categories of objects and even the 60 exemplars. Furthermore, the neural processing regions associated with the sensory/functional features are modeled in 4CAPS. Simulation of 4CAPS shows comparable activation pattern to human.
Experiment Correlations
Across trials 0.7838
Across words 0.6658
Across studies0.0225 on all categories
0.3039 on tools and dwellings
Across subjects N/A
Experiment Chance Raw Rank
Category(12-way) 8.3% 52% 88%
Exemplar(60-way) 1.6% 37% 91%
Hammer
A_tool
Has_a_hand
Has_a_handle
Has_a_metal_head
Has_a_clefted_head/claw
Found_in_tool_boxes
Has_a_wooden_handle
Is_loudIs_heavyMade_of_metal
Made_of_word
Used_for_carpentry
Used_for_construction
Used_for_pounding
Used_for_pounding_
nails
Used_for_pulling_nails
encyclopedic function smell soundtactile taste taxonomicVisual-color
Visual_form_and_
surface_properties
Visual-motion
tool dwelling
Bilateral fusiform
gyrus
Left sensory motor cortex
Piriform cortex
Right lateral orbital
frontal area
Left ventralpre-motor cortex
Superior temporal sulcus
Left middle
temporal gyrus
Ventro occipito temporal
cortex
Left parahippocampal
gyrus
Right parahippocampal
gyrus
Left inferior parietal lobule
Left postcentral
gyrus
Left precentral
gyrus
Left cuneus
PRECENT POSTCENTLSTANT LSTMID LSTPOS LIES LPARAHIP
RPARAHIP
LFUSIFORM LIPL
RFUSIFORM
LSES
Word
Feature
Knowledge Type
Neural Processing
Regions
ROI
Machine Learning Flowchart Correlation Between Neural and 4CAPS Activation Vector
4CAPS Modeling Hierarchy