Comparison Study of Clinical 3D Comparison Study of Clinical 3D MRI Brain Segmentation EvaluationMRI Brain Segmentation Evaluation

Ting Song Ting Song 11, Elsa D. Angelini , Elsa D. Angelini 22,,Brett D. Mensh Brett D. Mensh 33, Andrew Laine , Andrew Laine 11

11Heffner Biomedical Imaging LaboratoryHeffner Biomedical Imaging LaboratoryDepartment of Biomedical Engineering, Department of Biomedical Engineering,

Columbia University, NY, USAColumbia University, NY, USA22Ecole Nationale SupEcole Nationale Supéérieure des Trieure des Téélléécommunicationscommunications

Paris, FranceParis, France33Department of Biological Psychiatry, Columbia University, Department of Biological Psychiatry, Columbia University,

College of Physicians and Surgeons, NY, USACollege of Physicians and Surgeons, NY, USA

2

OverviewOverview•• IntroductionIntroduction•• Segmentation MethodsSegmentation Methods

–– Histogram Thresholding.Histogram Thresholding.–– MultiMulti--phase Level Set.phase Level Set.–– Fuzzy Connectedness.Fuzzy Connectedness.–– Hidden Markov Random Field Model and Hidden Markov Random Field Model and

the Expectationthe Expectation--Maximization (HMRFMaximization (HMRF--EM).EM).•• Results & Comparison of MethodsResults & Comparison of Methods•• ConclusionsConclusions

3

IntroductionIntroduction

Gray Matter (GM)Gray Matter (GM) White Matter (WM)White Matter (WM)

CerebroCerebro--Spinal Fluid (CSF)Spinal Fluid (CSF)

4

MotivationMotivation• Segmentation of clinical brain MRI data is

critical for functional and anatomical studies of cortical structures.

• Little work has been done to evaluate and compare the performance of different segmentation methods on clinical data sets, especially for the CSF.

• The performance of four different methods was quantitatively assessed according to manually labeled data sets (“ground truth”).

5

MotivationMotivation

CSF

GM

WM

Homogeneity of cortical tissues on simulated MRI data. (source: BrainWeb simulated brain database, www.bic.mni.mcgill.ca/brainweb)

6

MotivationMotivation

CSF

GM

WM

Homogeneity of cortical tissues on clinical T1-weighted MRI data.

7

MethodologyMethodology

Methods evaluated:1. Histogram thresholding (Method A)

2. Multi-phase level set (Method B)

3. Fuzzy connectedness (Method C)

4. Hidden Markov Random Field Model and Expectation-Maximization (HMRF-EM) (Method D)

8

1. Histogram Thresholding1. Histogram Thresholding

CSF

GM WM

9

1. Histogram Thresholding1. Histogram Thresholding

•• CharacteristicsCharacteristics:– Initialization with two threshold values.– Simple set up & fast computation.

•• Set up for Set up for ““optimaloptimal”” performanceperformance:– Tuning of threshold values for maximization of the

Tanimoto index (TI) for the three tissues.– Manually labeled data used as the reference. – Simplex optimization for co-segmentation of the

three tissues.

1TPTI

FP=

+

10

2. Multi2. Multi--PhasePhase Level SetLevel Set‘Active Contours Without Edges’ [Chan-Vese IEEE TMI 2001]

• Method:– 3D deformable model based on Mumford-Shah

functional.– Homogeneity-based external forces.– Multiphase framework with 2 level set functions to

segment 4 homogeneous objects simultaneously.

TwoTwo φ φ functions functions => Four phases

One One φφ functionfunction=> Two phases => Four phases=> Two phases

11

2. Multi2. Multi--PhasePhase Level SetLevel Set

•• Characteristics:Characteristics:– Automatic initialization.– No a priori information required.

•• Set up:Set up:– Details provided in:

E. D. Angelini, T. Song, B. D. Mensh, A. Laine, "Multi-phase three-dimensional level set segmentation of brainMRI," International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Saint-Malo, France, September 2004.

12

3. Simple Fuzzy Connectedness3. Simple Fuzzy Connectedness

• Method:– Computation of a fuzzy connectedness map to

measure similarities between voxels.

‘Fuzzy Connectedness and Object Definition: Theory, Algorithms, and Applications in Image Segmentation’, J. Udupa et al., GMIP, 1996.

AffinityHigh affinity

Connectedness Fuzzy maps

Low Affinity

– Thresholding of each tissue fuzzy map to obtain a final segmentation.

13

3. Simple Fuzzy Connectedness3. Simple Fuzzy Connectedness• Characteristics:

– Initialization with seed points and prior statistics.– Implementation from the National Library of Medicine

Insight Segmentation and Registration Toolkit (ITK). (www.itk.org)

• Set up for “optimal” performance:– The threshold value for fuzzy maps was optimized using

the Simplex scheme to obtain the segmentation with best accuracy (from the computed fuzzy connectedness map).

14

4. HMRF4. HMRF--EMEM‘Segmentation of Brain MR Images Through a Hidden Markov Random

Field Model and the Expectation-Maximization Algorithm ’

[Y. Zhang, M. Brady, S. Smith, IEEE Transactions on Medical Imaging, 2001]

• Method– Statistical classification method based on Hidden

Markov random field models.– Class labels, tissue parameters and bias fields are

updated iteratively.• Characteristics:

– The method was implemented in the FSL-FMRIB Software Library (http://www.fmrib.ox.ac.uk/fsl).

15

ResultsResults

•• DataData–– Ten T1Ten T1--weighted MRI data sets from healthy weighted MRI data sets from healthy

young volunteers.young volunteers.

–– Data sets size = (256x256x73) with 3mm slice Data sets size = (256x256x73) with 3mm slice thickness and 0.86mm inthickness and 0.86mm in--plane resolution.plane resolution.

–– Manual labeling available (manual protocol Manual labeling available (manual protocol requiring 40 hours per brain).requiring 40 hours per brain).

16

ResultsResults

•• Evaluation protocolEvaluation protocol–– Measurements of organMeasurements of organ’’s volume.s volume.

–– True positive, false positive voxel fractions True positive, false positive voxel fractions and the and the TanimotoTanimoto index for the each tissue.index for the each tissue.

–– Analysis of variance (ANOVA) performed to Analysis of variance (ANOVA) performed to evaluate the differences between the four evaluate the differences between the four segmentation methods.segmentation methods.

17

ResultsResultsSegmentation of CSF

(c)(b) (d)(a) (e)

(a) Histogram thresholding, (b) Level set, (c) Fuzzy connectedne(a) Histogram thresholding, (b) Level set, (c) Fuzzy connectedness, ss, (d) (d) HMRFsHMRFs, (e) Manual labeling., (e) Manual labeling.

18

ResultsResults

1.5 2 2.5 3 3.5 4 4.5

x 105

1.5

2

2.5

3

3.5

4

4.5x 105

Ground Truth Volume (Voxel)

Seg

men

ted

Vol

ume

(Vox

el)

Equal LineMethod AMethod BMethod CMethod D

GM volume

19

ResultsResults

1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8

x 105

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8x 105

Ground Truth Volume (Voxel)

Seg

men

ted

Vol

ume

(Vox

el)

Equal LineMethod AMethod BMethod CMethod D

WM volume

20

ResultsResults

0 1 2 3 4 5 6

x 104

0

1

2

3

4

5

6x 104

Ground Truth Volume (Voxel)

Seg

men

ted

Vol

ume

(Vox

el)

Equal LineMethod AMethod BMethod CMethod D

CSF volume

21

ResultsResultsAccuracy Evaluation: True Positive

Gray Gray MatterMatter

0102030405060708090

100

1 2 3 4 5 6 7 8 9 10

Data Index

True

Posi

tive

(%)

White MatterWhite Matter

CSFCSF

Level SetHist. Thresh

Fuzzy Conn.HMRF-EM

0102030405060708090

100

1 2 3 4 5 6 7 8 9 10

True

Pos

itive

(%)

0102030405060708090

100

1 2 3 4 5 6 7 8 9 10

True

Posi

tive

(%)

Data Index

22

ResultsResultsAccuracy Evaluation: False Positives

Gray Gray MatterMatter

0102030405060708090

100

1 2 3 4 5 6 7 8 9 10

Data Index

Fals

ePo

sitiv

e (%

)

White White MatterMatter

0102030405060708090

100

1 2 3 4 5 6 7 8 9 10

Data Index

Fals

ePo

sitiv

e (%

)CSFCSF

0

100

200

300

400

500

600

700

800

1 2 3 4 5 6 7 8 9 10

Fals

ePo

sitiv

e (%

)

Level SetHist. Thresh.

Fuzzy Conn.HMRF-EM

23

ResultsResults• Analysis of variance: ANOVA

• Inter-method variance / Intra-method variance of the TI index.• Statistical difference between methods confirmed for p < 0.005.

24

DiscussionDiscussion•• Segmentation Segmentation ofof WM & GMWM & GM

– All methods reported high TI values.

– Superior performance of methods A and B.

•• Segmentation of CSFSegmentation of CSF– Superiority of methods B and C (cf. TI values).

– Highest variance for method C.

– Significant under segmentation of CSF (i.e. high FN errors) due to very low resolution at the ventricle borders.

– Difference between methods for sulcal CSF:• Different handling of partial volume effects

• Manual labeling eliminates sulcal CSF. Arbitrary choice and no ground truth available for these voxels.

– Manual labeling of the ventricles and sulcal CSF can vary up to 15% between experts as reported in the literature.

B. Level SetA. Hist. Thresh.

C. Fuzzy Conn.D. HMRF-EM

25

ConclusionsConclusions• Four different methods were compared using clinical data.

• Statistical difference of methods was assessed.

• Difference of performance focused on the extraction of CSF structures.

– Method A and B have strong correlations with manual tracing.

– Method C tends to over segment the GM structure in several cases.

– Method D tends to over segment the CSF structures.

• Combining all results, the level set three-dimensional deformable model

(Method B) provides the best performance for high accuracy and low

variance of performance index.

B. Level SetA. Hist. Thresh.

C. Fuzzy Conn.D. HMRF-EM

26

ReferencesReferences

1. E. D. Angelini, T. Song, B. D. Mensh, A. Laine, “Multi-phase three-dimensional level set segmentation of brain MRI,"MICCAI (Medical Image Computing and Computer-Assisted Intervention) International Conference 2004, Saint-Malo, France, September 26-30, 2004.

2. E. D. Angelini, T. Song, B. D. Mensh, A. Laine, “Segmentation and quantitative evaluation of brain MRI data with a multi-phase three-dimensional implicit deformable model,"SPIE International Symposium, Medical Imaging 2004, San Diego, CA USA, Vol. 5370, pp. 526-537, 2004.

3. Heffner Biomedical Imaging Labhttp://hbil.bme.columbia.edu