a similarity retrieval system for multimodal functional brain images rosalia f. tungaraza advisor:...
Post on 19-Dec-2015
227 views
TRANSCRIPT
![Page 1: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/1.jpg)
A Similarity Retrieval System for Multimodal Functional Brain
ImagesRosalia F. Tungaraza
Advisor: Prof. Linda G. Shapiro
Ph.D. Defense
Computer Science & EngineeringUniversity of Washington
1
![Page 2: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/2.jpg)
Functional Brain Imaging
Study how the brain works
Imaging while subject performs a task
Image represents some aspect of the brain e.g.
fMRI: brain blood oxygen level
ERP: scalp electric activity
2
![Page 3: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/3.jpg)
Motivation
3
Given a database of functional brain images from various subjects, cognitive tasks, and image modality.
Database users need to retrieve similar images
A system that can automatically perform this retrieval will reduce amount of time and effort users spend during this task
3
![Page 4: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/4.jpg)
Content-Based Image Retrieval
4
• Given a query image and an image database, retrieve the images that are most similar to the query in order of similarity.
• Example system for photographic images: Andy Berman’s FIDS system; Yi Li’s Demo
http://www.cs.washington.edu/research/imagedatabase/demo
![Page 5: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/5.jpg)
Image Features / Distance Measures
Image Database
Query Image
Distance Measure
Retrieved Images
Image Feature
Extraction
User
Feature SpaceImages 5
![Page 6: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/6.jpg)
Contributions1. Created a similarity retrieval system for multimodal
brain images
I. fMRI, ERP, and combined fMRI-ERP
II. User interface
2. Developed feature extraction methods for fMRI and ERP data
3. Developed pair-wise similarity metrics
4. Simulated human expert similarity scores
6
![Page 7: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/7.jpg)
Outline Background
fMRI ERP Existing Similarity Retrieval Systems for
these modalities Feature Extraction Process Similarity Metric User Interface Retrieval Performance Simulate Human Expert
77
![Page 8: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/8.jpg)
Functional Magnetic Resonance Imaging (fMRI)
8
A non-invasive brain imaging technique
Records blood oxygen level in brain
While imaging, subject performs a task8
![Page 9: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/9.jpg)
fMRI Statistical ImagesfMRI Statistical Images
Statistical Analysis
Voxel Thresholding
9
![Page 10: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/10.jpg)
Event-Related Potentials (ERP)
10
A non-invasive brain imaging technique
Records electric activity along scalp
While imaging, subject performs a task
@ 2004 by Nucleus Communications, Inc.
10
![Page 11: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/11.jpg)
ERP Source Localization
11
Researchers want to identify the electric activity and its source for each electrode
But, multiple sources for each electrode
LORETA approximates anatomic locations of sources
11
![Page 12: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/12.jpg)
Comparison of fMRI and ERP Data
12
fMRI ERP
Spatial resolution Good (in mm) undefined/poor
Temporal resolution Poor (in sec) Excellent (in msec)
12
![Page 13: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/13.jpg)
Similarity Retrieval Systems for fMRI Images
1313
![Page 14: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/14.jpg)
Similarity Retrieval Systems for ERP Images
14
No relevant literature found
14
![Page 15: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/15.jpg)
Similarity Retrieval Systems for Combined fMRI-ERP Images
15
No relevant literature found
15
![Page 16: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/16.jpg)
Outline Background
Feature Extraction Process fMRI features ERP features
Similarity Metric User Interface Retrieval Performance Simulate Human Expert
1616
![Page 17: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/17.jpg)
Threshold
Perform connected component analysis
Perform clustering
Approximate cluster centroids
Compute region vectors
Original database
fMRI Feature Extraction
k=3
k=2
k=1
1. Centroid 2. Avg Activation Value 3. Var Activation Value4. Volume5. Avg Distance to
Centroid6. Var of those Distances
17
![Page 18: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/18.jpg)
Select time-segment
Compute voxel-wise statistically significant difference between means
Threshold
ERP Feature Extraction
Compute feature(X,Y,Z)
positions of retained voxels
18
Signals at each point incorporateinformation from that voxel and neighbors.
The retained voxels have significant activationmeaning activities A and B are very different.
![Page 19: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/19.jpg)
Outline Background Feature Extraction Process
Similarity Metric Summed Minimum Distance Similarity Score for Combined fMRI-ERP
Images
User Interface Retrieval Performance Simulate Human Expert
1919
![Page 20: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/20.jpg)
Summed Minimum Distance (SMD) for fMRI and ERP Images
20
Subject Q Subject T
Q2T =
SMD = (Q2T+T2Q) / 220
Euclideandistancebetweenfeaturevectors*
*We also used normalized Euclidean distance.
![Page 21: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/21.jpg)
Sample SMD Scores
21
SMD
21
![Page 22: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/22.jpg)
Similarity Score for Combined fMRI-ERP Images
SIM(i,j) = αSMDfMRI(i,j) + (1-α)SMDERP(i,j)
22
![Page 23: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/23.jpg)
Outline
Background Feature Extraction Process Similarity Metric User Interface
Retrieval Performance Simulate Human Expert
2323
![Page 24: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/24.jpg)
GUI: Front Page
24
![Page 25: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/25.jpg)
GUI: Retrievals with SMD Scores
25
SMD
25
![Page 26: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/26.jpg)
GUI: Query-Target Activations (fMRI)
26
![Page 27: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/27.jpg)
GUI: Query-Target Activations (ERP)
27
![Page 28: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/28.jpg)
Outline Background Feature Extraction Process Similarity Metric User Interface
Retrieval Performance Data Sets fMRI Retrieval Performance ERP Retrieval Performance Combined fMRI-ERP Retrieval Performance
Simulate Human Expert 2828
![Page 29: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/29.jpg)
Data Sets for fMRI Retrievals
29
Central-Cross -- 24 subjects (Face Recognition)
AOD -- 15 subjects (Sound Recognition)
SB -- 15 subjects (Memorization)
Checkerboard -- 12 subjects (Face Recognition)
d g f p g n
29
![Page 30: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/30.jpg)
Data Set for ERP Retrievals
30
View Human Faces (Face Up) -- 15 subjects
View Houses (House Up) -- 15 subjects
… …
30
![Page 31: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/31.jpg)
Data Set for Combined fMRI-ERP Retrievals
31
ERP: same data set as used in ERP retrieval
fMRI: • Task: Face recognition using a house up
background
• Same subjects and images as data set for ERP retrieval
31
![Page 32: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/32.jpg)
fMRI Retrieval Performance
32
1. RFX Retrievals
2. Individual Brain Retrieval
3. Testing Group Homogeneity
4. Feature Selection
32
Random effects models are very conservativeaverage activation models from a group, whichcontain only activated voxels present in all members.
![Page 33: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/33.jpg)
fMRI Retrieval Score
33
Perfect score : Retrieval Score = 0
Random score: Retrieval Score ~ 0.5
Worst score: Retrieval Score = 1
33
![Page 34: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/34.jpg)
Example Scores
34
• Let N = 100 and Nrel = 3
• Sample Case1 Ri = i, i = 1 to 3 1 + 2 + 3 – 6 = 0/300
• Sample Case 2 R1 = 3, R2 = 2, R3 = 1 3 + 2 + 1 – 6 = 0/300
• Sample Case 3: R1 = 10, R2 = 20, R3 = 30 10 + 20 + 30 – 6 = 54/300
![Page 35: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/35.jpg)
fMRI Individual Brain Retrievals
35
Use individual brain as query
Mean Retrieval Scores (Top 6% activated voxels)
Checkerboard 0.09
SB 0.16
Central-Cross 0.21
AOD 0.26
35
![Page 36: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/36.jpg)
Testing Group Homogeneity for fMRI
36
CB
AOD
Central-Cross
SB
36
MDS1 and MDS2 are 2 projections of themultidimensional feature data.
All groups except AOD had tight clusters.
![Page 37: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/37.jpg)
ERP Retrieval Performance
3737
![Page 38: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/38.jpg)
Subject #8 RetrievalsTo
p Re
trie
vals
Bott
om
Retr
ieva
ls
38
![Page 39: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/39.jpg)
Combined fMRI-ERP Retrieval
SIM(i,j) = αSMDfMRI(i,j) + (1-α)SMDERP(i,j)
α = 0.0, ERP only
α = 1.0, fMRI onlyα = 0.6
α = 0.3
39
![Page 40: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/40.jpg)
Outline Background Feature Extraction Process Similarity Metric User Interface Retrieval Performance
Simulate Human Expert Simulation Method Data Set Testing Function Performance
4040
![Page 41: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/41.jpg)
Simulate Human Expert Current retrieval system requires some expert
knowledge
Estimate a function to generate similarity scores with high correlation to expert scores
41
Dr. JeffOjemann
![Page 42: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/42.jpg)
Simulation Method1. Uniform feature representation: create
codebook and encode each subject
2. Concatenate the codebook features for each pair of subjects
3. Create eigenfeatures
4. Estimate a function
5. Test function performance42
![Page 43: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/43.jpg)
The Codebook
43
• Out of all the clusters found in all N brains, create a single brain that has a representation of each unique cluster. This is the codebook.
• Then for each of the N brains use the codebook to create a subject-specific vector representing each of those clusters.
• In the case where the codebook has a given cluster, but that particular subject misses it, that whole portion of this subject's codebook will be empty.
• Otherwise, the other parts of this subject's codebook will be filled with the properties of this subject's clusters.
![Page 44: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/44.jpg)
S1 S2 S3
S1 S2 S3
Create the reference brain (codebook)
Encode Images
1. Uniform Feature Representation
44
![Page 45: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/45.jpg)
2. Concatenate Codebook Features
XYZ Centroid A Avg Activation Value VA Var Activation ValueS Size (Volume)D Avg Distance to CentroidVD Var of those Distances
45
Pairs of Subjects Expert Score
![Page 46: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/46.jpg)
3. Create Eigenfeatures
46
originalfeaturespace
eigenfeaturespace
![Page 47: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/47.jpg)
4. Estimate a Function
Linear function using linear regression
Non-linear function using generalized regression neural networks (GRNN)
S1,S1
S1,S2
S1,S3
S3,S3
… … … … … … … …
0.00
0.30
0.90
0.00
…
47
We want to estimate a function that takes a pair of region vectorsfrom two subjects and computes their similarity score.
![Page 48: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/48.jpg)
5. Test Function Performance The Pearson Correlation Coefficient (CC)
The Average Absolute Error (A-ABSE)
The Root Mean Square Error (RMSE)
48
![Page 49: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/49.jpg)
Data Set
fMRI data (Central-Cross)
-- 23 subjects
-- Face Recognition task
+Human Expert Generated Pair-wise Similarity Matrix
49
![Page 50: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/50.jpg)
Overall Function Performance
50
overfitting!
![Page 51: A Similarity Retrieval System for Multimodal Functional Brain Images Rosalia F. Tungaraza Advisor: Prof. Linda G. Shapiro Ph.D. Defense Computer Science](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d2c5503460f94a027fb/html5/thumbnails/51.jpg)
Contributions1. Created a similarity retrieval system for multimodal
brain images
I. fMRI, ERP, and combined fMRI-ERP
II. User interface
2. Developed feature extraction methods for fMRI and ERP data
3. Developed pair-wise similarity metrics
4. Simulated human expert similarity scores
51