non-parametric statistical permutation tests for local shape analysis
DESCRIPTION
Non-Parametric Statistical Permutation Tests for Local Shape Analysis. Martin Styner, UNC Dimitrios Pantazis, Richard Leahy, USC LA Tom Nichols, University of Michigan Ann Arbor. TOC. Motivation local shape analysis Local shape difference/distance measures Statistical significance maps - PowerPoint PPT PresentationTRANSCRIPT
NA-MICNational Alliance for Medical Image Computing
Non-Parametric Statistical Permutation Tests for Local
Shape Analysis
Martin Styner, UNC Dimitrios Pantazis, Richard Leahy, USC LA
Tom Nichols, University of Michigan Ann Arbor
National Alliance for Medical Image Computing http://na-mic.org
TOC
• Motivation local shape analysis• Local shape difference/distance measures• Statistical significance maps• Problem: Multiple correlated comparisons• 1st approach: It’s a hack!• 2nd approach: Let’s do it right!• Template free - Hotelling T2 measures• Example Results• Conclusions & Outlook
National Alliance for Medical Image Computing http://na-mic.org
Motivation Shape Analysis
• Anatomical studies of brain structures– Changes between patient and healthy controls– Detection, Enhanced understanding, course of
disease, pathology– Normal neuro-development
• interest in diseases with brain changes – Schizophrenia, autism, fragile-X, Alzheimer's
• Information additional to volume• Both volumetric and shape analysis• Shape analysis: where and how?
National Alliance for Medical Image Computing http://na-mic.org
Shape Distances
• Shape description: – SPHARM-PDM– M-rep
• Normalization:– Rigid Procrustes, brain
size normalized• Local scalar distance
– Euclidean distance– “Radius” difference– Signed vs absolute
National Alliance for Medical Image Computing http://na-mic.org
Local Shape Analysis
• Distance to template• Distance between
subject pairs• Sets of distance-maps• Significance map
– Statistical test at each point
– Mean difference test• P-values• Significance threshold
National Alliance for Medical Image Computing http://na-mic.org
Multiple Comparisons
• Lots of correlated statistical tests → Overly optimistic– M-rep: 2x24 tests, SPHARM: 2252 tests– Same problem with other shape descriptions and other
difference analysis schemes
• Correction needed, overly optimistic– Test locally at given level (e.g. α = 0.05)– Globally incorrect false-positive rate
• Bonferroni correction, worst case, assumption: 0% correlation – Correct False-Positive rate at α/n = 0.05/4000 = 0.0000125– Correct False-Positive rate at 1-(1- α)1/n = 0.0000128
National Alliance for Medical Image Computing http://na-mic.org
1st Approach: SnPM
• Statistical non-Parametric Maps in SPM (SPIE 2004)
• Decomposition of distance map into separate images for processing in SnPM
• 75% overlap necessary due to distortions
• Each image is tested separately in SnPM
• ONE BIG HACK:– 6 correlated tests– Averaging in overlap
National Alliance for Medical Image Computing http://na-mic.org
2nd Approach: Permutations
• Non-parametric permutation test using spatially summarized statistics, ISBI 2004
• Correct false positive control (Type II)• Summary:
– Random permutations of the group labels– Metric for difference between populations – Spatial normalization for uniform spatial sensitivity– Summarize statistics across whole shape – Choose threshold in summary statistic
National Alliance for Medical Image Computing http://na-mic.org
Statistical Problem
• 2 groups: a & b, #member na, nb
• Each member: p-features (e.g. 4000)• Test: Is the mean of each feature in the 2
populations the same? – Null hypothesis: The mean of each feature is the
same– Permutations of group label leave distributions
unchanged under null hypothesis– M permutations
• Specific test– Correct false positive rate
National Alliance for Medical Image Computing http://na-mic.org
Non-parametric Permutation Tests
• Goal: significance for a vector with 4’000 correlated variables• 50’000 to 100’000 permutations• Extrema statistic: controls false-positive
diff normSummaryStatisticMin/Max
Histogram
diff norm
National Alliance for Medical Image Computing http://na-mic.org
Single Feature Example
• Feature fA,1-fA,n1 vs fB,1-fB,n1
• Compute difference: T0 =|A- B|
• Permute group label → A’i,B’I → Ti
• Make Histogram of Ti
• Histogram = pdf
• Sum histogram = cdf
• Cdf at 1-α = Threshold
α
National Alliance for Medical Image Computing http://na-mic.org
Multiple features
• Testing a single feature → no problem• Testing multiple features together as a
whole, NOT individually• Summary is necessary of all features
across the surface• For correct Type II, use an extrema
measurement– Right sided distance metrics → Maxima– Left sided distance metrics → Minima
National Alliance for Medical Image Computing http://na-mic.org
Spatial Normalization
• Extremal summary is most influenced by regions with higher variance
• Assume 2 regions with same difference, but one has larger variance– Region with larger variance contributes more
to extremal statistics and thus sensitivity in that region is higher
• Normalization of local statistical distributions is necessary for spatially uniform sensitivity
National Alliance for Medical Image Computing http://na-mic.org
Spatial Normalization
• A) local p-values, non-parametric– Minimum, (1-α) thresh
• B) standard deviation, parametric– Maximum, α thresh
• C) q-th quantile, non-parametric– q = 68% ~ if Gaussian– Maximum, α thresh
• Assumptions: A > C > B
• Uniform sensitivity: A > C ~ B
• Numerical pdf: C > B > A
• Use A– Many permutations– High computation + space costs
Extrema statistics
Shape difference metric
α 1-α
Norm Max-stat
Norm p-valueMin-stat
National Alliance for Medical Image Computing http://na-mic.org
Raw vs Corrected P-values• Raw significance map:
– 4000 elements, 5% → 200 will be significant at 5% by pure chance, if locations are uncorrelated.
• Corrected significance map – Correct control of false negative– Single location significant → whole
shape significant
• No assumption over local covariance – Overly pessimistic– There is room for improvement!
National Alliance for Medical Image Computing http://na-mic.org
Raw vs Corrected P-values
• Raw p-values are comparable• But visualization of raw p-value map is misleading
even without statement about significance– Too optimistic, often viewed using linear colormap– P-value correction is non-linear !
Correction factor:F = Raw-P / Corr-P
National Alliance for Medical Image Computing http://na-mic.org
Metric for Group difference
• Scalar Local difference:– Signed/Unsigned
Euclidean distance– Thickness difference– Pairs, Template
• Difference of mean metric → Statistical feature T = |A- B|
• Needed: Positive scalar + shape difference metric between populations
PDM: Mean difference of Euclidean distance at a selected pointGaussian, passed Lilliefors test 0.01
National Alliance for Medical Image Computing http://na-mic.org
Template Free Stats
• No need for a scalar value at each location for each subject• Positive scalar difference value between populations• SPHARM-PDM
– So far: Signed/absolute Euclidean distance at each location to template → Scalar field analysis
– New: Difference vectors to template → Vector field analysis– Better: Location vector at each location → Template free analysis
→ Length of difference vector between mean vectors of populations
→ Hotelling T2 distance between populations = Hotelling T2 is mean difference 2 vector weighted with the pooled Covariance matrix
T2 = (μa – μ b) Σa,b (μa –μb)
Σa,b = ( (na - 1) Σa + (nb -1) Σb ) / (na +nb - 2)
National Alliance for Medical Image Computing http://na-mic.org
Hotelling T2 histogram
Hotelling T2 distance of locations (template free)→ 2
National Alliance for Medical Image Computing http://na-mic.org
Results
• SnPM hack vs Correct permutation tests• Sample Hippocampus study: Stanley
study, resp/non-resp SZ (56) vs Cnt (26)– Both M-rep & PDM
• Other example tests
National Alliance for Medical Image Computing http://na-mic.org
SnPM-Hack vs Correct Stat
• SnPM too optimistic – relatively good agreement
L
0.001
0.05
R
SnPM
National Alliance for Medical Image Computing http://na-mic.org
Hippocampus SZ Study
Left Right
National Alliance for Medical Image Computing http://na-mic.org
M-rep Shape Analysis
Left Right
National Alliance for Medical Image Computing http://na-mic.org
Vector Field Analysis
T2 location
T2 templatedifference
Abs templatedistance (scalar)
0.0010.05
Raw Significance Maps Corr Significance Maps
National Alliance for Medical Image Computing http://na-mic.org
Conclusions of Methods
• Multiple comparison correction scheme for local shape analysis– Non-parametric, Permutation-based– Globally correct for false-positive across whole
object– Applicable to scalar, vectors, any Euclidean
space measures– Black box– Pessimistic estimate
National Alliance for Medical Image Computing http://na-mic.org
NAMIC kit
• StatNonParamTestPDM– Command line tool, Win/Linux/MacOSX – E.g. StatNonParamTestPDM <listfile> -out <basename> -surfList
-numPerms 50000 -signLevel 0.05 -signSteps 1000
• Output (for meshes)– P-value of global shape difference between the
populations (mean T2 across surface)– Mean difference map (effect size)– Hotelling T2 map using robust T2 formula– Raw significance map– Corrected significance map– Mean surfaces of the 2 groups
National Alliance for Medical Image Computing http://na-mic.org
StatNonParamTestPDM• Input: File with list of ITK mesh files• Generic features also supported using customizable text-file
input option• Currently in NAMIC-Sandbox (open)• Next: submission to Insight Journal• MeshVisu, combination of Mesh and maps
0.011 0.2324 0.123 …..
Map Txt
National Alliance for Medical Image Computing http://na-mic.org
That’s it folks…
• Questions
National Alliance for Medical Image Computing http://na-mic.org
Corrected Analysis – Spatial Normalization• Without normalization → incorrect, unless uniformity is assumed
– High variability → overestimation of significance– Low variability → underestimation of significance
-normalization ~ 68% normalization
No norm max stat
L
R