scanlab meta-analysis of neuroimaging data what, why, and how tor d. wager columbia university
Post on 14-Jan-2016
217 Views
Preview:
TRANSCRIPT
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analysis of neuroimaging Meta-analysis of neuroimaging datadata
What, Why, and HowWhat, Why, and How
Tor D. WagerTor D. WagerColumbia UniversityColumbia University
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Uses of meta-analysis in neuroimaging
• Meta-analysis is an essential tool for summarizing the vast and growing neuroimaging literature
Wager, Lindquist, & Hernandez, in press
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Uses of meta-analysis in neuroimaging
Wager, Lindquist, & Kapan, 2007
• Assess consistency of activation across laboratories and task variants
• Compare across many types of tasks and evaluate the specificity of activated regions for particular psychological conditions
• Identify and define boundaries of functional regions
• Co-activation: Develop models of functional systems and pathways
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Functional networks in meta-analysis
• Use regions or distributed networks in a priori tests in future studies
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analyses of cognitive control
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analyses of emotion & motivation
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analyses of disorders
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analyses of language
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analyses of other stuff
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Using meta-analysis to evaluate consistency:
Why?
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Locating emotion-responsive regions
164 PET/fMRI studies, 437 activation maps, 2478 coordinates
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Why identify consistent areas?
• Making statistic maps in neuroimaging studies involves many tests (~100,000 per brain map)
• Many studies use uncorrected or improperly corrected p-values
Long-term Memory
P-value thresholds used
Corr.
# of
Map
s
Uncorrected
How many false positives?A rough estimate: 663 peaks, 17% of reported activations
Wager, Lindquist, & Kaplan, 2007
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Consistency
Reported peaks163 studies
ConsistentlyActivatedregions
Emotion: 163 studies
SCANLab
http://www.columbia.edu/cu/psychology/tor/
mTC
pOFC
vmPFC
BF
aINSsTC
latOFClFG TC
pgACCrdACC
dmPFCPCC
OCC
sgACC
vmPFC
CM, MD
Deep nuclei
Gyrus rectusCentral sulcus
dmPFC Pre SMA
Fig 4: MKDA Results
Ventral surfaceLateral surface (R)Medial surface (L)
Kober et al., in press, NI
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Using meta-analysis to evaluate specificity:
Why?
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Disgust responses: Specificity in insula?
Insula
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Disgust responses: Specificity in insula?
Feldman-Barrett & Wager, 2005; Phan, Wager, Taylor, & Liberzon, 2002;Phan, Wager, Liberzon & Taylor, 2004
Search Area: Insula
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analysis plays a unique role in answering…
• Is it reliable?– Would each activated region replicate in future studies?– Would activation be insensitive to minor variations in task
design?
• Is it task-specific? – Predictive of a particular psychological state or task type?– Diagnostic value?
The Neural Correlates of Task X
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Using meta-analysis to evaluate consistency:
How?
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Monte Carlo:Expected maximum proportionUnder the null hypothesis
Apply threshold
Weightedaverage
E
Damasio, 2000 Liberzon, 2000 Wicker, 2003
Peak coordinate locations (437 maps)
…
Kernel convolution
Comparison indicator maps
…
Proportion of activated Comparisons map
(from 437 comparisons)
Significant regions
Meta-analysis: Multilevel kernel density estimate (MKDE)
Wager, Lindquist, & Kaplan, 2007; Etkin & Wager, in press
Permute blobs within study maps
Permute blobs within study maps
SCANLab
http://www.columbia.edu/cu/psychology/tor/
MKDA: Key points
• Statistic reflects consistency across studies. Study comparison map is treated as a random effect. Peaks from one study cannot dominate.
• Studies are weighted by quality (see additional info on handouts for rationale)
• Spatial covariance is preserved in Monte Carlo. Less sensitive to arbitrary standards for how many peaks to report.
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Whether and how to weight studies/peaks
MKDA analysis weights by sqrt(sample size) and study quality (including fixed/random effects)
€
P = CIMc
δ c N c
δ c N cc
∑
⎛
⎝
⎜ ⎜ ⎜
⎞
⎠
⎟ ⎟ ⎟c
∑
€
δc =1
€
δc = 0.75Fixed effectsRandom effects
Activation indicator (1 or 0) for map c
Study quality weightSample size for map c
Weighted proportion of activating studiesWeightedaverage
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Monte Carlo Simulation
• Simulation vs. theory (e.g. Poisson process)
• Simulation allows:– Non-stationary spatial distribution of peaks
(clumps) under null hypothesis; randomize blob locations
– Family-wise error rate control with irregular (brain-shaped) search volume
– Cluster size inference, given primary threshold
Monte Carlo:E(max(P|H0))
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Compare with Activation Likelihood Estimate (ALE), Kernel Density
Analysis (KDA)
Peak coordinatesCombined across studies
€
⊗
Kernel convolutionDensity kernel
ALE kernelOR
€
=
Peak density orALE map
Apply significance threshold
Significant results
Density kernel: Chein, 1998; Phan et al., 2002; Wager et al., 2003, 2004, 2007, in press
Gaussian density kernel + ALE: Turkeltaub et al., 2002; Laird et al., 2005; others
Ignores the fact that some studies report more peaks than others!
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Comparison with other methods
• Statistic reflects consistency across studies. Study comparison map is treated as a random effect. Peaks from one study cannot dominate.
• Studies are weighted by quality
• Spatial covariance is preserved in Monte Carlo. Less sensitive to arbitrary standards for how many peaks to report.
• Peaks are lumped together, study is fixed effect. Peaks from one study can dominate, studies that report more peaks dominate.
• No weighting, or z-score weighting (problematic)
• Spatial covariance is not preserved in Monte Carlo. Effects of reporting standards large.
MKDA KDA/ALE
See handouts for more comparison points
SCANLab
http://www.columbia.edu/cu/psychology/tor/
ALE approach
• Treats points as if they were Gaussian probability distributions.
• Summarize the union of probabilities at each voxel: probability of any peak “truly” lying in that voxel
€
P(X1 ∪ X2...∪ Xn ) =1− P(∪X) =1− P(X1) * P(X2) * ...P(Xn )
€
P(X i ) is the probability that peak Xi lies in a given voxelThe bar indicates the complement operator
Null hypothesis: No peaks lie in voxel
€
P(∪X) = 0Alt hypothesis: At least one peak lies in voxel
€
P(∪X) ≠ 0
SCANLab
http://www.columbia.edu/cu/psychology/tor/
ALE meta-analysis
• Analyst chooses smoothing kernel• ALE analysis with zero smoothing:
– Every voxel reported in any study is significant in the meta-analysis
• Test case: 3-peak meta analysis, one peak activates in voxel:
€
P(X1) =1,P(X2) = 0,P(X3) = 0
€
1− Pr(∪X ) = 1− (0)* (1)* (1) = 1
ALE statistic:Highest possible value!
• In practice: 10 – 15 mm FWHM kernel
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Comparison across methods: Inference
Property KDA ALE Multilevel KDA Kernel Spherical Gaussian SphericalInterpretation of statistic
Num nearby peaks Prob. that at least one peak nearby
Num. study maps activating nearby
Null hypothesis Peaks are not spatially consistent
No peaks truly activate
Study maps are not spatially consistent
Interpretation of significant result
More peaks lie near voxel than expected by chance
One or more peaks lies at this voxel
A higher proportion of studies activate near voxel than expected by chance
Assumptions 1. Study is fixed effect (homogenous sample of studies)2. Peaks are spatially independent under the null hypothesis
1. Study is fixed effect (homogenous sample of studies)2. Peaks are spatially independent under the null hypothesis
Activation ‘blobs’ are spatially independent under the null hypothesis
Generalize to New peaks from same studies
New peaks from same studies
New study maps
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Comparison: Correction and Weighting
Property KDA ALE Multilevel KDA
Multiple comparisons
FWER FDR FWER (recommended) or FDR
Weighting None, or weight peaks by z-score
None Weight studies by sample size, fixed/random effects, quality
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Density analysis: SummaryWorking memoryExecutive WMLong-term memory
Inhibition Task switching
Memory
Response selection
Wager et al., 2004; Nee, Wager, & Jonides, 2007; Wager et al., in press;
Van Snellenberg & Wager, in press
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Using meta-analysis to evaluate specificity:
How?
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Specificity
• Task-related differences in relative activation frequency across the brain: – MKDA difference maps (e.g., Wager et al.,
2008)
• Task-related differences in absolute activation frequency– Nonparametric chi-square maps (Wager,
Lindquist, & Kaplan, 2007)
• Classifier systems to predict task type from distributed patterns of peaks (e.g., Gilbert)
SCANLab
http://www.columbia.edu/cu/psychology/tor/
MKDA Difference maps: Emotion example
• Approach: – Calculate density maps for two conditions, subtract to get
difference maps– Monte Carlo: Randomize blob locations within each study,
re-calculate density difference maps and save max– Repeat for many (e.g., 10,000) iterations to get max
distribution– Threshold based on Monte Carlo simulation
Experienced
Perceived
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Emotion example: Selective regions
AmyTP OFC
Amy
aIns
Experience > Perception
aIns
OFC
vaIns
dmPFC
Hy vaIns
TP
PAG
PAG
Midb
mOFC
TP
OFC
OFC
Midb
TP
Hy
Hy
Perception > Experience
pgACC
Amy
CB
IFG
IFGAmy
CB
Wager et al., in press, Handbook of Emotion
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Task-brain activity associations in meta-analysis
Study contrast map
Region/Voxel 1
Task condition
Study 1 1 Disgust
Study 2 0 Fear
Study 3 1 Disgust
Study 4 1 Happiness
Study 5 0 Anger
… … …
Study N 0 Sadness
Measures of association:Chi-square• But requires high expected counts (> 5) in each cell. Not appropriate for map-wise testing over many voxelsFisher’s exact test (2 categories only)Multinomial exact test• Computationally impractical!Nonparametric chi-square• Approximation to exact test• OK for low expected counts
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Nonparametric chi-square: Details
Study contrast map
Region/Voxel 1
Task condition
Study 1 1 Disgust
Study 2 0 Fear
Study 3 1 Disgust
Study 4 1 Happiness
Study 5 0 Anger
… … …
Study N 0 Sadness
Idea of exact test: • Conditionalize on marginal counts for activation and task conditions. • Null hypothesis: no systematic association between activation and task• P-value is proportion of null-hypothesis possible arrangements that can produce distribution across task conditions as large as observed or larger.
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Nonparametric chi-square: Details
Study contrast map
Region/Voxel 1
Task condition
Study 1 0 Disgust
Study 2 1 Fear
Study 3 0 Disgust
Study 4 1 Happiness
Study 5 0 Anger
… … …
Study N 1 Sadness
Permutation test:• Permute activation indicator vector, creating null-hypothesis data (no systematic association)
• Marginal counts are preserved. • Test 5,000 or more samples and calculate P-value based on observed null-hypothesis distribution
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Density difference vs. Chi-square
• Relative vs. absolute differences
Voxels (one-dimensional brain)
ExperiencePerception
Chi-square
Density
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Can we predict the emotion from the pattern of brain activity?
• Approach: predict studies based on their pattern of reported peaks (e.g., Gilbert, 2006)
• Use naïve Bayesian classifier (see work by Laconte;
Tong; Norman; Haxby). Cross-validate: predict emotion type for new studies that are not part of training set.
Experienced
Perceived
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Classifying experienced emotion vs. perceived emotion: 80% accurate
Exp
eri
en
ceP
erc
ep
tion
PAG vs. Ant. thalamus
Deep cerebellar nuc. vs. Lat. cerebellum
DMPFC vs. Pre-SMA
EXP vs. PER
DMPFC
EXP
PAG
Deep cerebellar nuc.
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Outline: Why and How…
• Consistency: Replicability across studies– Consistency in single-region results: MKDA– Consistency in functional networks: MKDA + Co-
activation
• Specificity and “reverse inference”– Brain-activity – psychological category mappings
for individual brain regions: MKDA difference maps; Nonparametric Chi-square
– Brain-activity – psychological category mappings for distributed networksApplying classifier systems to meta-analytic data
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Extending meta-analysis to connectivity
Study contrast map
Region/Voxel 1
Region/Voxel 2
Study 1 1 0
Study 2 0 0
Study 3 1 1
Study 4 1 1
Study 5 0 0
… … …
Study N 0 1
Co-activation: If a study (contrast map) activates within k mm of voxel 1, is it more likely to also activate within k mm of voxel 2?
Measures of association:Kendall’s Tau-bFisher’s exact testNonparametric chi-square
Others…
N = 45 Region 1No
Region 1 Yes
Region 2 Yes
6 23
Region 2No
12 4
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Kendall’s Tau: Details• Ordinal “nonparametric” association between
two variables, x and y• Uses ranks; no assumption of linearity or
normal distribution (Kendall, 1938, Biometrika)• Values between [-1 to 1], like Pearson’s
correlation
€
τ =4 min(rank(x),rank(y)
i=1
N−1
∑ ) > i
N(N −1)
⎛
⎝
⎜ ⎜ ⎜ ⎜
⎞
⎠
⎟ ⎟ ⎟ ⎟
−1
Tau is proportion of concordant pairs of observations sign(x diff. between pairs)= sign(y diff. between pairs)Tau = (# concordant pairs - # discordant pairs) / total # pairs
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Meta-analysis functional networks: Examples
• Emotion: Kober et al. (in press), 437 maps
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Acknowledgements
Ed SmithEd Smith
Funding agencies: National Science FoundationNational Institute of Mental Health
Martin LindquistMartin Lindquist
Derek NeeDerek NeeJohn JonidesJohn JonidesEd SmithEd Smith
Tom NicholsTom Nichols
Lisa Feldman Lisa Feldman BarrettBarrett
Hedy KoberLauren KaplanJason BuhleJared Van Snellenberg
Luan PhanLuan PhanSteve TaylorSteve TaylorIsrael LiberzonIsrael Liberzon
Meta-analysis of emotion
StatisticsMeta-analysis of cognitive
function
Students
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Weighting
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Whether and how to weight studies/peaks
• Studies (and peaks) differ in sample size, methodology, analysis type, smoothness, etc.
• Advantageous to give more weight to more reliable studies/peaks
• Z-score weighting– Advantages: Weights nominally more
reliable peaks more heavily– Disadvantages: Small studies can produce
variable results. Reporting bias: High z-score peaks are high partially due to error; “capitalizing on chance”• Must convert to common Z-score metric across
different analysis types in different studies
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Whether and how to weight studies/peaks
• Alternative: Sample-size weighting– Advantages:
• Weights studies by the quality of information their peaks are likely to reflect
• Avoids overweighting peaks reported due to “capitalizing on chance”
– Disadvantages: Ignores relative reliability of various peaks within studies
SCANLab
http://www.columbia.edu/cu/psychology/tor/
MKDA vs. KDA vs. ALE:Comparison chart
SCANLab
http://www.columbia.edu/cu/psychology/tor/
More details on reverse inference
SCANLab
http://www.columbia.edu/cu/psychology/tor/
Is brain activity diagnostic of a particular psychological state?
‘Forward’ and ‘reverse’ inference are not the same!Reverse inference requires comparing across many psychological states!
Pleasure?
Punishing wrongdoersBrain activity
Given a psychological state
We observe brain activity
P(Brain | Psy)Forward inference
Can we infer psychological pleasure?
P(Psy | Brain)Given brain activity
Reverse inference
SCANLab
http://www.columbia.edu/cu/psychology/tor/
The predictive value problem: Worked example
For a brain region to be used as a marker of pleasure
– The brain region must respond consistently to pleasure
– The brain region must respond specifically to pleasure (not activated by other things)
Ventral caudatePleasure
P(Brain|Pleasure) = .9Forward inference; Sensitivity
Non-pleasure
P(Brain|no pleasure) = .41-Specificity
P(pleasure) = .1
Prior
Caculate reverse inference:
P(Pleasure|Brain) = .2
SCANLab
http://www.columbia.edu/cu/psychology/tor/
More details on connectivity
SCANLab
http://www.columbia.edu/cu/psychology/tor/
More details on MKDA difference maps and nonparametric chi-square
maps
top related