miccai grand challenge on neonatal brain segmentation (neobrains12) i. išgum 1, m.j.n.l. benders 2,...
TRANSCRIPT
MICCAI Grand Challenge on Neonatal Brain Segmentation
(NeoBrainS12)
I. Išgum1, M.J.N.L. Benders2, M.A. Viergever1
1Image Sciences Institute, University Medical Center Utrecht2Department of Neonatology, University Medical Center Utrecht
Aim
To compare the performance of (semi-)automatic algorithms for neonatal brain tissue segmentation
in T1- and T2-weighted MR scans
Data
• Three different image sets of pre-term born infants are available: 1. axial scans acquired at 40 weeks corrected
gestational age
2. coronal scans acquired at 30 weeks corrected gestational age
3. coronal scans acquired 40 weeks corrected gestational age
Sheet 3
Patient population
• No brain pathology was visible• 15 months follow-up normal according to Griffith's
assessment test• All scans acquired on a Philips 3T system
Sheet 4
Protocol: axial scans, 40 w
• 3DT1-weighted: TR=9.4 ms; TE=4.6 ms; scan time=3.44 min, FOV=180x180; reconstruction matrix=512x512; consecutive sections with thickness=2.0 mm; number of sections=50, in-plane resolution 0.35 mm x 0.35 mm;
• T2-weighted: TR=6293 ms; TE=120 ms; scan time=5.40 min; FOV=180x180; reconstruction matrix=512x512; consecutive sections with thickness=2.0 mm; number of sections=50, in-plane resolution 0.35 mm x 0.35 mm
Sheet 5
Protocol: coronal scans, 30 w
• 3DT1-weighted: TR=9.4 ms; TE=4.6 ms; scan time=4.44 min, FOV=130x100; reconstruction matrix=384x384; consecutive sections with thickness=2.0 mm; number of sections=50, in-plane resolution 0.34 mm x 0.34 mm;
• T2-weighted: TR=10085 ms; TE=120 ms; scan time=6.23 min; FOV=130x100; reconstruction matrix=384x384; consecutive sections with thickness=2.0 mm; number of sections=50, in-plane resolution 0.34 mm x 0.34 mm
Sheet 6
Protocol: coronal scans, 40 w
• 3DT1-weighted: TR=9.5 ms; TE=4.6 ms; scan time=7.02 min, FOV=200x200; reconstruction matrix=256x256; consecutive sections with thickness=1.2 mm; number of sections=110, in-plane resolution 0.78 mm x 0.78 mm;
• T2-weighted: TR=4847 ms; TE=150 ms; scan time=5.05 min; FOV=180x180; reconstruction matrix=512x512; consecutive sections with thickness=1.2 mm; number of sections=110, in-plane resolution 0.35 mm x 0.35 mm
Sheet 7
Task
• To segment 1. cortical grey matter (CoGM)
2. basal ganglia and thalami (BGT)
3. unmyelinated white matter (UWM)
4. myelinated white matter (MWM)
5. brainstem (BS)
6. cerebellum (CB)
7. ventricles (Vent)
8. cerebrospinal fluid in the extracerebral space (CSF)
Sheet 8
Reference standard
• Manual segmentations were performed by trained observers
• The segmentations were verified independently by three neonatologists
• In case of disagreement, the decision was made in a consensus meeting
• Protocol: http://neobrains12.isi.uu.nl
Sheet 9
Evaluation
• Dice coefficient (DC)
• Modified Hausdorff distance (MHD)– 95th-percentile Hausdorff distance
Sheet 10
Ranking
• Per set ranking– based on per tissue rankings
• Per tissue rankings – for the eight tissue classes – unmyelinated + myelinated white matter– cerebrospinal fluid in the extracerebral space +
ventricles
• Rankings based on– Dice coefficient– modified Hausdorff distance
Sheet 11
Overall rankings
• The overall per tissue ranking– average of per tissue rankings based on the DC and
based on the MHD
• The overall per set ranking – Average of overall per tissue rankings within the data
set
Sheet 12
Winners
• One winner per set will be announced
• To qualify – a team must be ranked in at least three overall per
tissue rankings for the given set– the segmentation method needs to be automatic– team had no other data from the segmented sets but
those provided by the challenge
Sheet 13
Pre-challenge results
Sheet 14
Set 1: Axial, 40w GA
Sheet 15
Set 1: Overall ranking
Sheet 16
Imperial Oxford UCL UPenn FER-UMCU UMCU1*0
1
2
3
4
5
6
3
3.25
4.75
5.2
1.35
3.15
*modified version of Anbeek et al, Pediatr Res. 2008
Set 1: Dice coefficient
Sheet 17CoGM UWM CSF CB0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.830000000000001
0.89
0.760000000000001
0.840000000000001
0.88
0.720000000000001
0.830000000000001
0.870000000000001
0.710000000000001
0.8
0.850000000000001
0.610000000000001
0.850000000000001
0.91
0.770000000000001
0.830000000000001
0.88
0.730000000000001
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 1: Dice coefficient
Sheet 18BS CB BGT0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.840000000000001
0.91 0.91
0.8
0.92
0.870000000000001
0.820000000000001
0.9 0.89
0.740000000000001
0.9
0.8
0.870000000000001
0.93 0.93
0.860000000000001
0.93 0.92
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 1: Dice coefficient
Sheet 19UWM MWM UWM+MWM0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.89 0.890.88
0.43
0.880.870000000000001
0.22
0.8700000000000010.850000000000001
0.320000000000001
0.840000000000001
0.91
0.55
0.910.88
0.55
0.870000000000001
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 1: Dice coefficient
Sheet 20CSF Vent CSF+vent0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.760000000000001
0.80.770000000000001
0.720000000000001
0.850000000000001
0.7400000000000010.710000000000001
0.81
0.730000000000001
0.610000000000001
0.860000000000001
0.640000000000001
0.770000000000001
0.830000000000001
0.78
0.730000000000001
0.81
0.750000000000001
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 1: MHD
Sheet 21CoGM UWM CSF CB0
0.5
1
1.5
2
2.5
3
0.7300000000000010.630000000000001
1.68
0.6100000000000010.700000000000001
2.03
0.730000000000001
0.99
2.18
1.01
1.68
2.64
0.52
0.4
1.53
0.660000000000001
1.03
2.14
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 1: MHD
Sheet 22BS CB BGT0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
1.04
0.7000000000000010.8
1.24
0.630000000000001
1.321.3
0.92
1.29
1.96
0.850000000000001
4.18
0.52
0.35 0.35
0.750000000000001
0.35 0.4
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 1: MHD
Sheet 23UWM MWM UWM+MWM0
5
10
15
20
25
30
35
0.630000000000001 0.7000000000000010.700000000000001
4.41
0.7500000000000010.99
29.38
1.031.68
17.07
1.79
0.4
4.36
0.41.03
8.9
1.07
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 1: MHD
Sheet 24CSF Vent CSF+vent0
2
4
6
8
10
12
14
1.68
11.74
1.552.03
1.18
1.812.18
1.53
2.06
2.64
0.720000000000001
2.46
1.53
2.86
1.49
2.14
1.36
2
ImperialOxfordUCLUpennFER-UMCUUMCU1
Set 2: Coronal, 30w GA
Sheet 25
Set 2: Overall ranking
Sheet 26
Imperial UCL Upenn UMCU20
0.5
1
1.5
2
2.5
3
3.5
1.892
3.1
1.8
*Chita et al, submitted to a conference
Set 2: Dice coefficient
Sheet 27CoGM UWM CSF CB0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.70
0.91
0.84
0.71
0.90
0.76
0.60
0.87
0.63
0.70
0.94
0.85
ImperialUCLUpennUMCU2
Set 2: Dice coefficient
Sheet 28BS CB BGT0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.75
0.87
0.82
0.73
0.87
0.84
0.65
0.85
0.74
ImperialUCLUpenn
Set 2: Dice coefficient
Sheet 29UWM MWM UWM+MWM0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.90563 0.9054210.90
0.16
0.900.87
0.69
0.87
0.938652 0.938443
ImperialUCLUpennUMCU2
Set 2: Dice coefficient
Sheet 30CSF Vent CSF+vent0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.840.86 0.85
0.76
0.86
0.78
0.63
0.87
0.66
0.846348
0.79
ImperialUCLUpennUMCU2
Set 2: MHD
Sheet 31CoGM UWM CSF CB0.00
0.50
1.00
1.50
2.00
2.50
1.00
0.50
1.32
0.91
0.75
1.15
1.90
0.83
2.11
1.11
0.34
0.94
ImperialUCLUpennUMCU2
Set 2: MHD
Sheet 32BS CB BGT0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
3.03
0.81
2.01
3.27
0.82
2.01
3.45
1.17
3.97
ImperialUCLUpenn
Set 2: MHD
Sheet 33UWM MWM UWM+MWM0.00
5.00
10.00
15.00
20.00
25.00
0.498195 0.4982130.75
22.76
0.750.83 0.61 0.830.338562000000001 0.338562333333334
ImperialUCLUpennUMCU2
Set 2: MHD
Sheet 34CSF Vent CSF+vent0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
1.321.43
0.81
1.15
0.90
1.03
2.11
1.28
2.05
0.94332
3.03
ImperialUCLUpennUMCU2
Set 3: Coronal, 40w GA
Sheet 35
Set 3: Overall ranking
Sheet 36
Imperial Geneva UCL UPenn0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
1.7
2.3
2.7
2.9
Set 3: Dice coefficient
Sheet 37CoGM UWM CSF CB0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.76
0.89
0.67
0.75
0.86
0.750.73
0.84
0.58
0.69
0.83
0.71
GenevaImperialUCLUpenn
Set 3: Dice coefficient
Sheet 38BS CB BGT0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.71
0.89
0.85
0.73
0.92
0.86
0.73
0.92
0.85
0.36
0.91
0.79
GenevaImperialUCLUpenn
Set 3: Dice coefficient
Sheet 39UWM MWM UWM+MWM0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0.88764 0.8794160.855914 0.8469300000000010.84
0.08
0.830.83
0.07
0.82
GenevaImperialUCLUpenn
Set 3: Dice coefficient
Sheet 40CSF Vent CSF+vent0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
0.669200000000002
0.710988
0.75
0.82
0.77
0.58
0.70
0.61
0.71
0.84
0.72
GenevaImperialUCLUpenn
Set 3: MHD
Sheet 41CoGM UWM CSF CB0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
0.83
0.57
3.61
0.860.95
2.24
1.01
1.23
2.96
1.17 1.12
2.72
GenevaImperialUCLUpenn
Set 3: MHD
Sheet 42BS CB BGT0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
1.83
0.73
1.16
1.65
0.52
1.15
1.37
0.45
1.35
3.91
0.70
3.51
GenevaImperialUCLUpenn
Set 3: MHD
Sheet 43UWM MWM UWM+MWM0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
0.565854 0.7308070.945071 1.122171.23
23.52
1.301.12
33.23
1.20
GenevaImperialUCLUpenn
Set 3: MHD
Sheet 44CSF Vent CSF+vent0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
3.61137
2.445212.24
12.22
1.81
2.96
1.88
2.692.72
0.99
2.40
GenevaImperialUCLUpenn
CoGM: Sulci segmentation (40w)
Sheet 45
T2w Manual segmentationAutomatic 1 Automatic 2 Automatic 3 Automatic 4
CoGM: Sulci segmentation (30w)
Sheet 46
T2wManual segmentation
Automatic 1 Automatic 2 Automatic 3 Automatic 4
CoGM: Oversegmentation in CB
Sheet 47
T2w Manual segmentation
Automatic 1
Automatic 2
Automatic 3
Automatic 4
UWM: False positives in CSF
Sheet 48
T2wManual segmentation
Automatic 1 Automatic 2 Automatic 3 Automatic 4
CSF: sulci
Sheet 49
T2wManual segmentation
Automatic 1 Automatic 2 Automatic 3 Automatic 4
T2w Manual segmentation
Ventricles
Sheet 50Automatic 1 Automatic 2 Automatic 3 Automatic 4
CB: oversegmentation
Sheet 51
T2w Manual segmentation
Automatic 1
Automatic 2
Automatic 3
Automatic 4
BGT-BS-MWM
Sheet 52Automatic 1 Automatic 2 Automatic 3 Automatic 4
T2wManual segmentation
Combination of methods
• Majority voting with all submitted methods
• Weighted majority voting of all methods– Based on a dice coefficient– Leave-one-out experiments
Sheet 53
Set 1: Combinations
Sheet 54CoGM UWM CSF CB0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.850000000000001
0.91
0.770000000000001
0.85
0.89
0.78
0.86
0.91
0.79
FER-UMCUImperialOxfordUCLUpennUMCU1MVWMV
Set 1: Combinations
Sheet 55BS CB BGT0.650000000000002
0.700000000000002
0.750000000000002
0.800000000000002
0.850000000000002
0.900000000000002
0.950000000000002
1
0.870000000000001
0.93 0.93
0.86
0.94
0.91
0.87
0.93
0.91
0.93
FER-UMCUImperialOxfordUCLUpennUMCU1MVWMV
Set 1: Combinations
Sheet 56UWM MWM UWM+MWM0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.91
0.55
0.910.89
0.38
0.890.91
0.46
0.90
0.55
FER-UMCUImperialOxfordUCLUpennUMCU1MVWMV
Set 1: Combinations
Sheet 57CSF Vent CSF+vent0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
0.7700000000000020.780.78
0.88
0.790.79
0.88
0.80
0.860000000000001
FER-UMCUImperialOxfordUCLUpennUMCU1MVWMV
• All NeoBrains12 teams
• Christian Mol• Karina Kersbergen• Linda de Vries• Floris Groenendal• Pim Moeskops
Acknowledgement
Sheet 58
Questions?
Sheet 59