fast methods for vascular segmentation based on ...1085403/fulltext01.pdf · "skeleton-based...
TRANSCRIPT
ACTAUNIVERSITATIS
UPSALIENSISUPPSALA
2017
Digital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1496
Fast Methods for VascularSegmentation Based onApproximate Skeleton Detection
KRISTÍNA LIDAYOVÁ
ISSN 1651-6214ISBN 978-91-554-9874-0urn:nbn:se:uu:diva-318796
Dissertation presented at Uppsala University to be publicly examined in ITC 2446,Lägerhyddsvägen 2, Uppsala, Monday, 22 May 2017 at 10:15 for the degree of Doctor ofPhilosophy. The examination will be conducted in English. Faculty examiner: ProfessorAlejandro F. Frangi (The University of Sheffield).
AbstractLidayová, K. 2017. Fast Methods for Vascular Segmentation Based on ApproximateSkeleton Detection. Digital Comprehensive Summaries of Uppsala Dissertations from theFaculty of Science and Technology 1496. 79 pp. Uppsala: Acta Universitatis Upsaliensis.ISBN 978-91-554-9874-0.
Modern medical imaging techniques have revolutionized health care over the last decades,providing clinicians with high-resolution 3D images of the inside of the patient's body withoutthe need for invasive procedures. Detailed images of the vascular anatomy can be capturedby angiography, providing a valuable source of information when deciding whether a vascularintervention is needed, for planning treatment, and for analyzing the success of therapy.However, increasing level of detail in the images, together with a wide availability of imagingdevices, lead to an urgent need for automated techniques for image segmentation and analysisin order to assist the clinicians in performing a fast and accurate examination.
To reduce the need for user interaction and increase the speed of vascular segmentation, wepropose a fast and fully automatic vascular skeleton extraction algorithm. This algorithm firstanalyzes the volume's intensity histogram in order to automatically adapt the internal parametersto each patient and then it produces an approximate skeleton of the patient's vasculature.The skeleton can serve as a seed region for subsequent surface extraction algorithms. Furtherimprovements of the skeleton extraction algorithm include the expansion to detect the skeletonof diseased arteries and the design of a convolutional neural network classifier that reduces falsepositive detections of vascular cross-sections. In addition to the complete skeleton extractionalgorithm, the thesis presents a segmentation algorithm based on modified onion-kernel regiongrowing. It initiates the growing from the previously extracted skeleton and provides a rapidbinary segmentation of tubular structures. To provide the possibility of extracting precisemeasurements from this segmentation we introduce a method for obtaining a segmentationwith subpixel precision out of the binary segmentation and the original image. This method isespecially suited for thin and elongated structures, such as vessels, since it does not shrink thelong protrusions. The method supports both 2D and 3D image data.
The methods were validated on real computed tomography datasets and are primarily intendedfor applications in vascular segmentation, however, they are robust enough to work with otheranatomical tree structures after adequate parameter adjustment, which was demonstrated on anairway-tree segmentation.
Keywords: medical image analysis, automatic skeleton extraction, vascular segmentation,coverage segmentation, convolutional neural network classifier, CT angiography
Kristína Lidayová, Department of Information Technology, Division of Visual Informationand Interaction, Box 337, Uppsala University, SE-751 05 Uppsala, Sweden. Department ofInformation Technology, Computerized Image Analysis and Human-Computer Interaction,Box 337, Uppsala University, SE-75105 Uppsala, Sweden.
© Kristína Lidayová 2017
ISSN 1651-6214ISBN 978-91-554-9874-0urn:nbn:se:uu:diva-318796 (http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-318796)
List of papers
This thesis is based on the following papers, which are referred to in the text
by their Roman numerals.
I K. Lidayová, H. Frimmel, C. Wang, E. Bengtsson and Ö. Smedby.
"Fast Vascular Skeleton Extraction Algorithm", Pattern RecognitionLetters, Vol. 76, pp. 67-75, 2016.
Lidayová developed the method, designed and performed the
experiments and wrote the paper
II K. Lidayová, H. Frimmel, E. Bengtsson and Ö. Smedby. "Improved
Centerline Tree Detection of Diseased Peripheral Arteries with a
Cascading Algorithm for Vascular Segmentation", Journal of MedicalImaging, accepted for publication, 2017
Lidayová developed the method, designed and performed the
experiments and wrote the paper
III K. Lidayová, D. A. Gómez Betancur, H. Frimmel, M. Hernández
Hoyos, M. Orkisz and Ö. Smedby. "Airway-Tree Segmentation in
Subjects with Acute Respiratory Distress Syndrome", in ScandinavianConference on Image Analysis (SCIA), accepted for publication, 2017.
Lidayová developed the method and wrote a major part of the paper
IV K. Lidayová, J. Lindblad, N. Sladoje and H. Frimmel. "Coverage
Segmentation of Thin Structures by Linear Unmixing and Local Centre
of Gravity Attraction", in 8th International Symposium on Image andSignal Processing and Analysis (ISPA), pp. 83-88, 2013.
Lidayová developed the method, designed and performed the
experiments and wrote the paper
V K. Lidayová, J. Lindblad, N. Sladoje, H. Frimmel, C. Wang and Ö.
Smedby. "Coverage Segmentation of 3D Thin Structures", in
International Conference on Image Processing Theory, Tools andApplications (IPTA), pp. 23-28, 2015.
Lidayová developed the method, designed and performed the
experiments and wrote the paper
VI K. Lidayová, A. Gupta, H. Frimmel, I.-M. Sintorn, E. Bengtsson and
Ö. Smedby. "Classification of Cross-sections for Vascular Skeleton
Extraction Using Convolutional Neural Networks", submitted for
conference publication, 2017.
Lidayová and Gupta contributed equally to the method development,
experiments design and performance and the paper writing
Reprints were made with permission from the publishers.
Related work
In addition to the papers included in this thesis, the author has also written or
contributed to the following publications:
1 K. Lidayová, H. Frimmel, Ch. Wang, E. Bengtsson, Ö Smedby.
"Skeleton-based fast, fully automated generation of vessel tree
structure for clinical evaluation of blood vessel systems",
Skeletonization and its Application, Academic Press and Newnes,
Elsevier, Oxford, pp. 345-382, book chapter in print
2 K. Lidayová, H. Frimmel, E. Bengtsson, Ö Smedby. "Fast Vessel
Centerline Tree Extraction Algorithm", in Proceedings of SSBA 2014,
Swedish Society for Automated Image Analysis, pp. 86-90, 2014
3 T. Majtner, K. Lidayová, S. Yildirim-Yayilgan, J. Y. Hardeberg.
"Improving Skin Lesion Segmentation in Dermoscopic Images by Thin
Artefacts Removal Methods", in 6th European Workshop on VisualInformation Processing (EUVIP), pp. 1-6, 2016
Contents
Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.1 Objectives of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.2 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 Medical digital imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1 Early medical imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Computed tomography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Digitization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3 Medical context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1 Anatomy of blood vessels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.1 Peripheral Artery Disease . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.2 Diagnosing PAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.1.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.1.4 CTA dataset of the lower limbs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Anatomy of the lungs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2.1 Acute Respiratory Distress Syndrome . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.2 Treatment of ARDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2.4 Dataset of thoracic CT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4 Vascular skeletons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1 Definition of a skeleton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2 Digital skeletons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3 Our approximate skeleton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.3.1 Description of the approximate skeleton . . . . . . . . . . . . . . . . . . . . . . . 29
4.3.2 Evaluation of the approximate skeleton . . . . . . . . . . . . . . . . . . . . . . . . . 29
5 Vascular segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2 Preprocessing: Vesselness filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.3 Segmentation methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.3.1 Thresholding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.3.2 Region-growing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.3.3 Centerline-based methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.4 Geometric deformable model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.5 Skeleton guided level set based vessel segmentation . . . . 40
6 Coverage segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.1 Fuzzy set theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2 Coverage representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.3 Coverage segmentation methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.3.1 Coverage segmentation based on double thresholding . 43
6.3.2 Coverage segmentation by local unmixing . . . . . . . . . . . . . . . . . . . . 43
6.3.3 Energy based coverage segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.4 Coverage segmentation applied to thin structures . . . . . . . . . . . . . . . . . . . . . . . 45
7 Deep neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
7.1 Deep learning in neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
7.2 Convolutional neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
7.2.1 Convolutional layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
7.2.2 Pooling layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
7.2.3 Rectified linear unit (ReLU) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
7.2.4 Batch normalization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
7.2.5 Fully-connected layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
7.3 Our CNN classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
8.1 Fast vascular skeleton extraction algorithm (Papers I and II) . . . . 51
8.2 Automatic adaptation to intensity variation between scans
(Paper I) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
8.3 Skeleton-guided onion-kernel based segmentation (Paper III) . . . 55
8.4 Coverage segmentation of thin structures (Papers IV and V) . . . . . 58
8.5 High resolution crisp reconstruction (Papers IV and V) . . . . . . . . . . . . . 60
8.6 Boundary thinning in 3D (Paper V) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8.7 Convolutional neural network classifier (Paper VI) . . . . . . . . . . . . . . . . . . . . . 61
9 Conclusion and future perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
9.1 Summary of contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
9.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Summary in Swedish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Errata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Abbreviations
2D Two-Dimensional
3D Three-Dimensional
ABI Ankle-Brachial Index
ARDS Acute Respiratory Distress Syndrome
CPR Curved Planar Reformation
CT Computed Tomography
CTA Computed Tomography Angiography
DVR Direct Volume Rendering
HU Hounsfield Units
MIP Maximum Intensity Projection
PACS Picture Archiving and Communication System
PAD Peripheral Artery Disease
PVE Partial Volume Effect
11
1. Introduction
Vascular diseases are among the leading causes of death in developed coun-
tries [54]. An early diagnosis and the right treatment are crucial to reduce
the number of fatal cases. Thanks to medical imaging techniques the state of
diseased vessels can be examined and accurately evaluated, without the need
for open surgery. This greatly reduces the risk of complications and improves
patient comfort. However, technological advances in non-invasive vascular
medical imaging have lead to ever-increasing amounts of large and complex
datasets that need to be visualized, analyzed, and interpreted. Processing of
such a large amount of data is a monotonous, error-prone and time-consuming
task. Hence, there is a large interest in developing automated image process-
ing techniques to reduce the workload and assist the radiologist in performing
a quick and accurate diagnosis.
Vascular segmentation plays a distinctive role among these image process-
ing techniques since it can aid in obtaining precise measurements for disease
severity assessment, therapy planning and monitoring. However, there are nu-
merous challenges related to utilizing vascular segmentation methods in clin-
ical routine. Segmentation algorithms need to be fast enough to run at speeds
compatible with interactive exploration of the volume. The algorithms should
require only minimal human interaction and support segmentation of vessels
of different sizes over the complete vascular tree. In addition, these algo-
rithms should be robust enough to accurately segment healthy vessels as well
as pathological vessels, which are of the greatest clinical interest. These goals
are currently not met by existing algorithms.
In this thesis, we focus on achieving these goals. The process of segmenting
tubular structures, such as vessels, can benefit from detecting an approximate
vascular skeleton prior to segmentation. We propose a fast and fully auto-
matic algorithm for vascular skeleton detection that provides an initial base
for subsequent surface extraction and thereby accelerates and facilitates the
segmentation process. The skeleton extraction algorithm is designed to detect
the complete vascular tree, including pathological vessels and vessels of dif-
ferent sizes. We also introduce a fast segmentation algorithm which leverages
this skeleton and produces a binary segmentation. We developed methods that,
provided with an initial binary segmentation, create a more accurate coverage
segmentation with subpixel precision. In addition, we propose a convolutional
neural network (CNN) classifier to further improve the skeleton extraction al-
gorithm.
These methods were designed having the main application of vascular seg-
mentation in mind, however, they can be successfully applied for segmenting
13
other anatomical tree structures, such as airway trees. The methods proposed
in this thesis were evaluated on challenging Computed Tomography Angiog-
raphy (CTA) datasets of the lower limbs taken from the clinical routine and on
thoracic Computed Tomography (CT) volumes of piglets with induced acute
respiratory distress syndrome (ARDS).
1.1 Objectives of the thesisThe main objective of this thesis was to develop fast and automatic meth-
ods for approximate skeleton extraction and precise segmentation of tubular
structures from volume CT datasets. The work is divided into two main appli-
cations: detection of arteries in CTA datasets of the lower limbs and an airway
tree segmentation from subjects with ARDS. My thesis studies concentrated
on several aspects of the main objective. Each aspect was addressed by one
publication. These aspects can be detailed as:
Specific aims:I To develop a fast and fully automatic method for vascular skeleton
extraction.
II To extend and improve the vascular skeleton extraction method for
pathological arteries with focus on calcifications and collateral ar-
teries.
III To suggest a method for airway tree segmentation with the main
objective of obtaining a large number of branches that can serve as
anatomical landmarks needed for further image processing.
IV To propose a method for coverage segmentation of thin 2D struc-
tures that will not shrink the long protrusions.
V To extend the coverage segmentation method of thin structures to
support segmentation of 3D elongated structures from volume im-
ages.
VI To develop a classifier based on CNN to reduce the false positives in
the vascular cross-sectional classification and to compare the clas-
sifier with a previously utilized set of knowledge-based filters.
1.2 Thesis outlineThe thesis is structured as follows: Chapter 2 gives an introduction to medical
digital imaging and to the digitization process. Chapter 3 provides a medi-
cal background for two diseases: peripheral artery disease (PAD) and ARDS.
Chapters 4 and 5 introduce methods for vascular skeleton extraction and vas-
cular segmentation, respectively. Chapter 6 focuses on coverage segmentation
techniques and Chapter 7 deals with deep neural networks. The main con-
tributions of the thesis are presented and summarized in Chapter 8. Finally,
Chapter 9 concludes the thesis and discusses possible future perspectives.
14
2. Medical digital imaging
This chapter provides a brief overview of the main milestones in radiographic
medical imaging, from analogue to digital techniques. We explain how digi-
tal volume images are acquired, how they are represented in a computer and
which imaging artifacts are common in CT.
2.1 Early medical imaging
Medical imaging began in 1895 after the German physicist Wilhelm Rönt-
gen discovered X-rays [22]. Röntgen quickly realized that this new radiation,
invisible to the human eye, could penetrate different materials and could be
recorded on photographic plates. He demonstrated it with the first X-ray image
of the human anatomy - the hand possibly belonging to his wife (Figure 2.1).
This important discovery laid the foundations of radiography.
Early radiography captured radiographic images using an X-ray source on
one side of the patient and an X-ray detector, usually a photographic film, on
the other side. The X-rays emitted from the source were transmitted through
the patient’s body where a fraction of the X-rays was absorbed by various
tissues. The attenuated X-rays were then recorded by the X-ray detector [8].
Figure 2.1. A radiograph of the hand taken by Röntgen in December 1895. Used with
permission of the Deutsches Röntgen Museum, Remscheid, Germany.
15
Figure 2.2. Orthogonal cross-sectional slices (axial, sagittal and coronal) of a 3D CT
volume representing the pelvic region. The image intensity corresponds to the tissue
type, dense tissue (e.g. bones) absorb more X-rays and appear as bright regions on a
CT scan compared to sparse tissue (e.g. soft tissue, air) which appear as gray or dark
regions.
2.2 Computed tomography
Computers were introduced in medical imaging in the early 1970s, when the
English engineer Godfrey Hounsfield invented X-ray Computed Tomography
(CT) [31]. Compared to the traditional radiography images, CT images are
produced by passing X-rays through the patient body at a large number of an-
gles. The X-ray source and the detector rotate around the patient and collect
multiple projection data. The data are processed by applying mathematical
reconstruction algorithms and synthesized by a computer into a set of cross-
sectional tomographic images. These images are stacked on top of each other
to create a 3D volume image of the anatomy of interest [8]. From the 3D
volume different orientations of 2D cross-sectional images (slices) can be vi-
sualized; three principal orientations given the position of a human are: axial,
sagittal and coronal (Figure 2.2).
2.3 Digitization
Early radiography images were captured on a photographic film and thereby
allowed a continuous representation of the human anatomy. The captured in-
formation represented a physical quantity, i.e., the darkness of the picture was
proportional to the number of X-rays that hit the film layer.
In X-ray CT the analogue information from the detectors is, from inception,
converted into digital form to process, store and transfer images. The process
of converting the analogue information into the digital form is digitization and
generally consists of two concurrent processes - sampling and quantization
[27]. Sampling limits the spatial resolution by dividing the continuous image
into an array of image elements. Quantization restricts brightness resolution
to a fixed set of discrete values for each image element. Figures 2.3 and 2.4
illustrate sampling and quantization, respectively.
16
(a) (b) (c)Figure 2.3. A coronal CT slice from Figure 2.2 displayed with different spatial resolu-
tions. The larger the number of pixels, the closer the spatial resolution of the digitized
image approximates the resolution of the original object. (a) 64 × 64 pixels, (b) 32 ×32 pixels, (c) 16 × 16 pixels. Anatomical structures clearly visible in image (a) are
due to partial volume effect blurred in image (c).
(a) (b) (c)Figure 2.4. A coronal CT slice from Figure 2.2 displayed at different image quantiza-
tion levels. The more levels of gray are used, the closer the brightness information in
the digitized image approximates the original brightness. (a) 16 levels of grayscale (4
bits), (b) 8 levels of grayscale (3 bits), (c) 4 levels of grayscale (2 bits).
The image element is called a pixel in the case of a two-dimensional image
and a voxel in the case of a three-dimensional volume image. A pixel size
is usually isotropic, whereas, in medical volume images it is common to have
anisotropic voxel size. Each voxel can be viewed as cube or rectangular prism,
having 6 faces, 12 edges and 8 corners. A set of voxels that share a common
face with a voxel p is known as the 6-neighborhood of p denoted by N6(p).Similarly, N18(p) and N26(p) are 18- and 26-neighborhoods of p and have
a common edge and corner with p, respectively. If another voxel q is m-
connected to p it means that q ∈ Nm(p).The brightness of the voxel is called intensity. In the case of CT images
the intensity of a voxel is related to the attenuation of X-ray radiation in the
tissue it represents. X-ray attenuation coefficients are normalized to water
17
Table 2.1. Interval values of Hounsfield units for selected tissue types.
Tissue Type HU Value Interval
Air -1000
Lung tissue -900 to -170
Fat tissue -220 to -30
Water 0
Muscle 10 to 40
Blood 40
Trabecular bone 300 to 500
Cortical bone 600 to 3000
as a reference material and are referred to as Hounsfield units (HU) or CT-
numbers [27]. In the Hounsfield scale, water represents a value of 0 HU and
air represents a value of -1000 HU. Muscle and blood are both around +40 HU.
In order to differentiate between blood vessels and surrounding muscle tissue,
a contrast agent needs to be injected into the bloodstream before the scan.
By carefully timing the acquisition, a CT scan can be obtained in the arterial
phase, when the contrast medium has not yet reached veins therefore only
arteries are visible, or in the venous phase, when both arteries and veins are
filled with the contrast agent. In this thesis, we work with CTA datasets that
were acquired in the arterial phase. Typical Hounsfield units of other tissue
types are presented in Table 2.1 [27, 59] but might differ slightly between
different CT machines.
Digitization leads, due to the limited spatial resolution, to an imaging ar-
tifact called partial volume effect (PVE). This artifact occurs when a voxel
contains more than one tissue type. The resulting HU value of such a voxel
is calculated as a weighted average of the HU values of individual compo-
nents. As a consequence, the intensities across tissue boundaries are blurred
and the visualization of small structures is limited. Other artifacts that are
often present in CT volume images are noise, motion blur caused by patient
movement during the scanning process, and beam hardening artifacts (a rela-
tive increase in X-ray energy after passing through a dense object (e.g. metallic
implant)).
In this thesis, we work mostly with CT angiography volume images depict-
ing the lower limbs and having the average size of 512×512×1711 voxels and
a bit depth of 16 bits (65 536 levels of gray). The voxel size is anisotropic, on
average having the length of 0.78×0.78×0.7 mm. This implies that the small-
est vessel that we can clearly view in our data should be at least 1.4 mm in
diameter. Smaller vessels (0.7 - 1.4 mm in diameter) can possibly be detected
if they are well aligned with the voxel grid and they cover one complete voxel.
Vessels smaller than 0.7 mm in diameter will be blurred and their intensity
averaged with the neighboring tissue intensity due to PVE.
18
3. Medical context
This chapter introduces the medical background needed to understand the
severity of the diseases which represent the main interest of this thesis, i.e.,
peripheral artery diseases and acute respiratory distress syndrome.
3.1 Anatomy of blood vessels
The blood vessels are part of the cardiovascular system. Together with the
heart they distribute the blood throughout the body, deliver materials such
as oxygen, nutrients, and hormones to all cells and carry away the wastes.
They contribute to homeostasis, stabilize temperature and pH, and fight dis-
eases [76].
There are five main types of blood vessels in the human body: arteries, ar-
terioles, capillaries, venules and veins. Together they form closed circulatory
routes for blood to travel from the heart to body organs and back again. In the
systemic circuit, arteries carry oxygenated blood away from the heart. They
divide into smaller arteries and arterioles until they reach a tissue or organ
when they branch into numerous microscopic capillaries. After the gas, nu-
trients and wastes exchange, capillaries merge together to form venules and
form bigger veins. Veins carry deoxygenated blood back to the heart.
There is also a second, pulmonary, circuit. In this circuit deoxygenated
blood is pumped from the heart to the lungs where gas exchange occurs and
oxygenated blood is carried back to the heart to be pumped out again in the
systemic circuit.
The largest artery in the body is the aorta with a diameter of two to four
centimeters [44]. The aorta starts at the heart, forms the aortic arch and
continues downwards until it splits at the aortic bifurcation into the left and
right common iliac arteries. Common iliac arteries soon split again into ex-
ternal and internal iliac arteries, which distribute the blood to the lower limbs
(through femoral and popliteal arteries and their branches) and the pelvis, re-
spectively [36]. An illustration of peripheral arteries is given in Figure 3.1.
3.1.1 Peripheral Artery Disease
Peripheral artery disease (PAD) is a common circulatory disease, which oc-
cludes the abdominal aorta, iliac, and lower-extremity arteries [53]. The most
19
Figure 3.1. Ilustration of the peripheral arteries. Used with permission of Sheri Amsel,
www.exploringnature.org; (adapted).
common cause of the occlusion is due to atherosclerosis, which is character-
ized by a combined process involving a long-term accumulation of a plaque
(e.g., cholesterol) on the arterial wall and inflammation in the wall [19]. The
plaque cumulation results in a reduction of the arterial lumen which impairs
the blood flow and limits the oxygen supply to the organs and to the lower
limbs [74]. Until the time when an artery gets completely occluded, minor
collateral arteries may grow and dilate to offer an alternative path around the
blocked artery. In late stages, the atherosclerosis is also commonly associated
with calcifications in the arterial wall.
3.1.2 Diagnosing PAD
A physician can usually diagnose peripheral artery disease based on the med-
ical and family histories, reported symptoms and physical exam. A common
test is ankle-brachial index (ABI), which compares the blood pressure in the
ankle to the blood pressure in the arm [29]. However, whenever drug and
life-style interventions are not adequate, and revascularization therapy is con-
sidered, a detailed description of the state of each major artery in the lower
limbs is needed. Such a description can be obtained using different medical
20
imaging techniques. A brief description of these techniques follows:
Catheter angiography is the oldest way of imaging the arteries and it in-
volves the introduction of a catheter into the arterial system. Once the catheter
is in place, an iodinated radiopaque contrast medium is injected through the
tube and images are acquired. The image acquisition is performed by digital
subtraction angiography, however, other imaging techniques are also applica-
ble. The advantages of catheter angiography are the possibility to supplement
the imaging with pressure measurements carried out through the catheter and
combining diagnosis and treatment in a single procedure. Factors that limit
the use of catheter angiography are potential medical complications caused by
the catheter and contrast medium [28].
Digital Subtraction Angiography (DSA) utilizes fluoroscopy X-ray-based
technique for image acquisition. A pre-contrast image is subtracted from later
post-contrast images, which removes nonvascular structures from the images
and enhances blood contrast. Nowadays, it is only used with intraarterial in-
jection, i.e., catheter angiography. DSA offers very high spatial resolution
but gives a two-dimensional projection of the three-dimensional anatomy. An
exception is rotational angiography which is mostly used for intracranial ves-
sels [78].
Ultrasonography utilizes high-frequency ultrasound waves to capture live
images. It is capable of identifying stenoses (vessel narrowing) as well as ar-
terial wall thickening [57]. With the use of Doppler flow measurements, the
blood flow disturbance can be identified and quantified [21]. Ultrasound ex-
aminations are commonly used for selecting candidates for revascularization
or for clinical follow-up after intervention.
Computed Tomography Angiography (CTA) employs iodine-based con-
trast media and X-rays just like in catheter angiography, but the injection is
made intravenously. This reduces patient discomfort and risk of complica-
tions relative to catheter angiography. The image acquisition is performed by
CT, resulting in a detailed 3D image containing information about vessels as
well as surrounding tissues [62]. CTA is well suited for studying calcifica-
tions, composition of the vessel wall and lumen. The main factors limiting the
use of CTA are the radiation dose and potential negative effects of the contrast
medium.
Magnetic Resonance Angiography (MRA) uses a powerful magnetic field,
pulses of radio wave energy and, most commonly, an intravenous contrast in-
jection to acquire a detailed 3D image [60]. The injected contrast medium is
gadolinium-based which is less likely to cause allergic reaction than iodine-
based contrast-agents. The patient is not exposed to any ionising radiation as
21
used in DSA and CTA, therefore it is possible to acquire a set of images taken
at several phases (arterial or venous). In patients sensitive to contrast agents
several alternative MRA techniques, not depending on a contrast agent, are
available [47].
3.1.3 Problem statement
Medical imaging techniques like CTA provide a clinician with a detailed 3D
image of the vascular anatomy. Hundreds of 2D images stacked on top of each
other are, however, hard to interpret and simple visualization techniques such
as maximum intensity projection (MIP) or direct volume rendering (DVR) are
often of great help. Both of these techniques are based on casting simulated
rays through the volume; in MIP the brightest value along the ray is displayed,
whereas DVR uses various optical concepts such as emittance, reflectance and
absorption to calculate the displayed intensity [13]. For displaying tubular
structures, curved planar reformation (CPR) may be a more suitable tech-
nique. This technique generates longitudinal cross-sections along a previously
generated vascular centerline to show the vascular lumen, wall and a few mil-
limeters of surrounding tissue in a curved plane. Obtaining vascular center-
lines needed for CPR by manual interaction is feasible, but very tedious and
time-consuming to fit into a tight clinical workflow. A fast vascular centerline
extraction algorithm would improve the usefulness of CPR.
In clinical cases when a clinician is deciding whether to perform an in-
tervention, precise information and measurements are needed and vascular
segmentation is therefore required. Numerous methods for vascular lumen or
wall segmentation have already been presented in the literature [35, 38, 75].
For widespread use in clinical routine, however, time efficiency is extremely
important. Regardless of how much interaction the algorithm requires, most
working radiologists are not likely to accept computation times above 1 or
2 minutes. State-of-the-art algorithms do not meet this requirement. A pos-
sible exception could be if the segmentation process is initiated completely
automatically once the image data arrive in the picture archiving and commu-nication system (PACS) and only limited editing work requires attention of the
user [80].
Vascular pathologies, which are of great clinical interest, pose another chal-
lenge for the segmentation algorithms. Typical for peripheral artery disease is
the presence of calcifications and occluded arteries which are often accom-
panied by dilated collateral vessels. These collateral vessels tend to have
a tortuous and irregular course and are typically difficult to handle for cur-
rent segmentation algorithms. Calcifications with high radiopacity, also cause
problems for segmentation algorithms that might be adapted to the much lower
intensities of the contrast-filled lumen. Other challenges include narrow and
22
irregular lumen and intensity variations along the arteries caused by stenosis
or incomplete contrast filling.
In this thesis, we contribute in solving the above-mentioned problems by
proposing a vascular skeleton extraction method that is designed to detect
healthy arteries of different sizes as well as to cope with challenges intro-
duced by PAD. The algorithm might be combined with other segmentation
algorithms and initiate the segmentation process automatically providing the
approximate skeleton as a seed region once the image data arrive in the PACS.
The algorithm is thought to be suited for the use in CPR if the current center-
line representation using polyline segments would be replaced by splines.
3.1.4 CTA dataset of the lower limbs
The dataset used in the studies presented in Papers I, II, V and VI was obtained
from the clinical routine of the Radiology department of the University Hos-
pital in Linköping, Sweden. The dataset contains CTA from the abdominal
aorta to the feet and was acquired with a clinical question of arterial stenosis
or occlusion. CTA was acquired on a Siemens Somatom Flash scanner, using
settings of 80 kV and 228 mAs. The protocol included automated injection
of 75-100 ml of iopromide (Ultravist) 370 mg J/ml at a rate of 3.7-4.0 ml/s,
followed by 70 ml of isotonic saline injected at the same rate, and images were
acquired in the arterial phase.
3.2 Anatomy of the lungsThe lungs are the essential organ of the respiratory system and are responsible
for providing oxygen to the body’s cells and removing carbon dioxide from
them. The air is brought from the external environment through the upper
and lower respiratory tracts to the lung alveoli in a process called inspiration,
the reverse process being expiration. The lower respiratory tract consists of
the trachea, which branches into two primary bronchi that enter the lungs.
Inside the lungs the primary bronchi are further divided into smaller bronchi,
bronchioles and terminal bronchioles, until they reach the last bifurcation -
respiratory bronchioles. Respiratory bronchioles consist of alveolar ducts and
alveolar sacs which are covered with alveoli. Alveoli are the main sites of gas
exchange with the blood. [76].
All structures that form a passageway for air (from the upper respiratory
track until the terminal bronchioles) are called airways and are usually not
considered to be part of the lungs. Respiratory bronchioles and their branches
make up the primary lobules which are the anatomical units of the lungs. Sev-
eral primary lobules create a secondary lobule and those are the main com-
ponents of the parenchyma. The parenchyma is surrounded by the sub-serous
areolar tissue and together with the pleura, which is a thin sack covering and
23
Figure 3.2. Ilustration of the lung anatomy. Source: https://patient.info/health/the-lungs-and-respiratory-tract; (adapted).
protecting the lungs, form the main parts of the lungs [25]. An illustration of
the anatomy of the lungs is given in Figure 3.2.
3.2.1 Acute Respiratory Distress Syndrome
Acute Respiratory Distress Syndrome (ARDS) is a life-threatening respiratory
condition that is characterized by an inflammation in the lungs which leads to
increased pulmonary vascular permeability and collapse of alveoli [14]. As
a consequence, alveoli are not able to fill with air, and oxygen can not be
provided in a sufficient amount into the bloodstream. The lack of oxygen
may lead to a multiple organ dysfunction syndrome and, in the worst case,
to death. The explicit cause of this symptom is not known, however, some
clinical conditions, such as trauma, pneumonia or sepsis, are associated with
development of ARDS [15].
3.2.2 Treatment of ARDS
The treatment of ARDS requires the use of mechanical ventilation in inten-
sive care units. Mechanical ventilation assists the patient with breathing and
can be controled by two main parameters: tidal volume (Vt) and positive end-expiratory pressure (PEEP). Tidal volume is the amount of air that is pushed
by the ventilator to the patient lungs and PEEP is the positive airway pressure
maintained until the end of expiration. Different ventilation strategies have
different objectives in focus, e.g., to minimize oxygen toxicity, to re-open col-
lapsed alveoli, or to prevent atelectasis [2]. Lung-protective strategies usually
use low Vt values compared to hypoxemia-reducing strategies, where higher
24
(a) (b) (c)Figure 3.3. Example slices of thoracic CT acquired at low PEEP value (2mmH2O)
showing lungs with ARDS induced, (a) axial slice with red and blue lines showing the
location of the coronal slices framed with the same color, (b) coronal slice showing
reduced contrast between lung and surrounding structures, (c) coronal slice showing
higher lung boundary contrast.
Vt and PEEP are utilized. Deciding the right strategy is challenging, since the
response of the lungs to the treatment is very hard to predict.
3.2.3 Problem statement
Analysis of lung aeration is essential in order to understand and to improve
ventilation strategies and to reduce the mortality rate in ARDS [61]. Lung aer-
ation refers to the amount of air and its distribution inside the lung parenchyma.
In order to obtain this information, the lung parenchyma needs to be seg-
mented, which implies lung segmentation and non-parenchymal structures
(vessels, bronchi and nerves) removal. However, many traditional lung seg-
mentation algorithms do not perform well in 3D images of lungs acquired in
the presence of ARDS. These images are often completely or partly lacking
contrast at the outer boundary of the lungs, especially in low PEEP conditions.
Figure 3.3 shows an example of such a case.
ARDS studies performed on animal models make it possible to obtain a
sequence of volume images for one subject acquired at different ventilation
conditions. The problem of missing contrast in low PEEP condition might
then be addressed by gradually warping the lung segmentation obtained from
the most-contrasted volume in the sequence towards the least contrasted one
by means of intensity registration. This approach was successfully presented
in [48].
Human ARDS studies, compared to animal studies, can not afford a large
number of acquisitions per subject due to minimization of radiation doses in
patients. Having a sequence of very few images (commonly two or three) in-
troduces two new challenges: (1) large deformations of lung structures and
(2) considerable intensity changes inside the lung between the two images
25
in the sequence. Traditional gray-level based registration is not powerful
enough to overcome these large differences and additional information, e.g.,
in a form of landmark correspondences between the volumes, should be pro-
vided. The anatomical landmarks might be obtained by utilizing an airway-
tree segmentation and extracting the points where airway branches bifurcate.
Once the successful lung segmentation is reached and the regions containing
non-parenchymal structures are removed, the lung aeration can be obtained.
The amount of air in each parenchyma voxel is easily calculated from the
thoracic CT scan using the voxel’s HU value. The percentage of air A(pi, j,l)in a voxel p at position (i, j, l) is defined as
A(pi, j,l) =I(pi, j,l)−HUtissue
HUair −HUtissue, (3.1)
where I(pi, j,l) is the intensity of the voxel pi, j,l , HUair represent the Hounsfield
values for air and HUtissue is HU of tissue. Tissue is mainly composed of water
and is considered to have value 0 HU in this application.
Our contribution in solving this problem was the development of fast airway
tree segmentation, that handles the intensity heterogeneity of ARDS pathology
and segments a large number of branches. Each branch bifurcation provides an
anatomic landmark needed for improving the lung registration and subsequent
delineation. The airway-tree segmentation also serves for airway removal.
3.2.4 Dataset of thoracic CT
The dataset utilized for the development and evaluation of the airway-tree seg-
mentation algorithm presented in Paper III was obtained from the Hôpital de la
Croix-Rousse, Lyon, France, where the research team of Réanimation Médi-
cale is performing a research project based on an animal model with induced
ARDS. The use of piglets for this study was approved by the institutional
review board for the care of animal subjects. The dataset contains 3D tho-
racic CT volumes of piglets obtained at different ventilation conditions. In our
work we utilized volumes acquired with a constant tidal volume Vt (5ml/kg)
and PEEP of different values ranging from 2 to 20 cmH2O. A pair of volumes
was available for each of the ventilation parameter settings; one volume was
acquired at the end of inspiration and another one at the end of expiration. The
animal model is used instead of images from humans since there is a lack of
availability of such image sequences. However, the sequence of few images
is simulated by selecting very different ventilation conditions (extreme and
intermediate) and testing the lung registration on this sparse sequence.
26
4. Vascular skeletons
This chapter defines the morphological skeleton in the continuous as well as
the discrete cases and explains what the approximate skeleton used in our
algorithms represents and how this skeleton is evaluated. For a more detailed
discussion on different skeletonization principles and approaches please refer
to [63, 69].
4.1 Definition of a skeleton
A skeleton is a thinned representation of an object that preserves the topologic
and geometric properties of the object and reduces its dimensionality. The pro-
cess of obtaining a skeleton is called skeletonization. Skeletonization reduces
a 2D object to a curve-skeleton (centerline) consisting of only 1D structures.
A 3D object may be simplified either to a surface-skeleton consisting of 1D
and 2D structures or reduced to the curve-skeleton either directly from the
object or via the surface-skeleton.
The notion of the skeleton was first described in the work of Blum [4] using
an illustrative grass-fire analogy. According to Blum a skeleton of an object is
a set of loci, where the fire fronts would meet if all points on the boundary of
the object (made of isotropic, flammable material) were set simultaneously on
fire.
A more formal definition of skeleton is based on the maximal ball concep-
tion [73] (pp.673–674). Let X be a 3D object in continuous space, X ⊂ R3.
A ball B(x,r) centered at position x ∈ X having a radius r, r ≥ 0, is defined as
B(x,r) = {y ∈R3, d(x,y)≤ r}, where d(x,y) is the distance from x to y in R3.
A ball B(x,r)⊂ X is considered to be maximal if and only if there is no larger
ball B′(x′,r′) ⊂ X that contains B. The skeleton is then defined as the set of
centers x of all maximal balls inscribed in X . Maximal balls and the skeleton
of a rectangle and a block are illustrated in Figure 4.1.
4.2 Digital skeletons
In many applications, including medical image processing, skeletonization is
performed on discrete 3D images. The definition of the skeleton in discrete
space is analogous to the one in continuous space, however, some issues may
occur due to discretization. For example, in the discrete case the skeleton lines
27
(a) (b)Figure 4.1. (a) A curve-skeleton of a 2D rectangle shown with blue lines, (b) a surface-
skeleton of a 3D block shown with blue surfaces; in both cases examples of two max-
imal disks/balls are also shown in light gray.
can be wider than one voxel since the diameter of the ball is always an odd
number of voxels (assuming the center of the ball must be one of the voxels).
If the width of the object is an even number of voxels, then the ball will be
maximal when it touches the object boundary only on one side. Another ball
having its center in the neighboring voxel touches the boundary of the object
on the other side. As a result, both centers of these two maximal balls are
included in the skeleton.
To avoid this undesired property, skeletonization in discrete images is per-
formed using digital skeletonization approaches. The most primitive digital
approach simulates the Blum’s grassfire propagation in a discrete grid using
an iterative boundary peeling algorithm under certain topologic and geomet-
ric constraints [64]. Two other categories of digital approaches are based on
fully predicate-kernel based iterative algorithms [50, 56] and distance trans-
forms [5, 6].
Regardless of which algorithm is used for the skeletonization, a digital
skeleton should satisfy as many as possible of the following properties [52]:
• Subset of the original object, i.e., the skeleton should be obtained from
the original object by only removing object voxels (not adding).
• Thin, i.e., the skeleton should be precisely one voxel wide.
• Allows reconstruction of the object, i.e., the information contained by
the skeleton should be sufficient to recover the original object.
• Topologically equivalent to the object, i.e., the skeleton and the orig-
inal object should have the same number of components, tunnels, and
cavities. A cavity is an empty space inside the object and a tunnel can
be intuitively described as a canal passing through the object.
4.3 Our approximate skeleton
So far we considered generating a skeleton from a previously segmented vol-
umetric object. However, in this thesis, we work with a reversed idea of con-
28
structing the skeleton directly from the medical images and subsequently using
this skeleton to obtain the volumetric segmentation of the vasculature. There-
fore, in this thesis and appended publications, the term skeleton refers to a
rather concise and approximate representation of the vasculature.
4.3.1 Description of the approximate skeleton
Our approximate skeleton fulfils the criterion of being thin and roughly cen-
trally located. More precisely, it consists of a set of connected polyline seg-
ments, where the polyline vertices are centrally located within an orthogonal
vascular cross-section (Figure 4.2a) and the straight polylines connecting the
vertices are guaranteed to lie inside the vessel. However the connections are
not necessarily centered (Figure 4.2b). Our skeleton has the same topology
as the vascular tree represented in the digital volume scan. This means that
the skeleton may contain disconnections in the regions where the main artery
becomes blocked and the resolution of the volume is not enough to depict tiny
collateral arteries in the CT scan. The resulting skeleton is also a subset of the
original object and it is possible to reconstruct the vascular tree from it. How-
ever, the difference is in the reconstruction method, which needs to be more
advanced than a simple distance-based expansion. The reconstruction method
we use to reproduce an accurate segmentation of the vascular tree combines
level sets with an implicit 3D model of the vessels [79]. The method takes an
approximated skeleton as an input and generates a 3D vessel model.
The motivation for using such an approximate skeleton is that with this
method the 3D problem of finding a skeleton in a volume is decomposed into a
set of 1D processes of connecting two centrally located vertices (nodes) within
a vascular cross-section and analyzing the connection. To further reduce the
execution time, there is no need to detect the vascular nodes in every vascular
cross-section or every volume slice. Based on the diameter at the detected vas-
cular node some nodes do not need to be saved for further processing, which
results in having more nodes in smaller and more tortuous vessels compared
to larger vessels where the distance between the nodes can be longer. Trans-
ferring the problem to a lower dimensional space allows a high computational
efficiency and opens the possibility for an interactive clinical use.
4.3.2 Evaluation of the approximate skeleton
In this subsection, the metrics used in Papers I and II to evaluate the perfor-
mance of our approximate skeleton are explained, that is, an overlap rate Mo,
a detection rate Md and an average distance error Derr .
Overlap and detection measureThe overlap rate Mo and the detection rate Md are modifications of more tra-
ditional precision and recall measures, respectively. However, since the pro-
29
(a) (b)Figure 4.2. (a) The process of selecting a centrally located polyline vertex. A voxel
lying within an orthogonal vascular cross-section is selected (white square no.1). This
voxel is not well-centered between two horizontal edges so the voxel lying truly in the
middle (white square no.2) becomes a new potential central voxel. The same verifica-
tion is repeated iteratively for vertical and horizontal directions until a truly centrally
located voxel (black square no.3) is found. This voxel is considered a polyline vertex.
Yellow voxels represent three consecutive voxels outside of the vessel cross-section;
(b) An approximate skeleton of a vessel with light blue vertices and dark blue straight
line connections.
posed algorithm produces an approximate skeleton which does not overlap the
reference skeleton completely, measurements that allow a small displacement
had to be used for our evaluation. We considered the skeleton detected by the
proposed algorithm to be correct if the skeleton was included within the refer-
ence segmentation generated by the level set based algorithm proposed in [79]
from the initial reference skeleton. This segmentation algorithm is explained
in detail at the end of the next chapter.
The first metric Mo determines the overlap of the detected skeleton with the
reference vascular segmentation. This measure is important in order to know
if the skeleton detected by our algorithm is a true artery skeleton.
The second metric Md determines the fraction of the reference skeleton that
was successfully detected by our detected skeleton. This rate informs us about
how much of the true artery skeleton was detected by our algorithm.
For evaluation purposes, the detected skeleton, which consists of a set of
connected polyline segments needs to be voxelized. The set of voxels created
by the voxelization of the detected skeleton is denoted Nd . Similarly, the set
of voxels Nr was created by voxelization of the reference skeleton. We define
another subset called Cd of all those voxels from the voxelized detected skele-
ton that lie inside the voxelized reference segmentation Cd = Nd ∩ Sr, where
Sr denotes set of voxels from the voxelized reference segmentation. All sets
and subsets introduced in this paragraph are visually explained on a drawing
of an artery in Figure 4.3.
30
Figure 4.3. Drawing of an artery; (left) a situation where two artery branches were
extracted and one artery branch was missed by the evaluated algorithm is shown;
(middle) a similar situation where another two artery branches were extracted and one
artery branch is missing in the reference skeleton is shown; (right) subsets Cd =Nd ∩Srand Cd_centered = Nd_centered ∩Sr are shown.
The detected skeleton is, in general, less centered, thus can contain fewer
voxels than the reference skeleton within the same area and this may affect
the results. Therefore, we need to introduce yet another two sets of vox-
els Nd_centered and Cd_centered . Nd_centered is made by voxelizing the detected
skeleton that was properly centered and Cd_centered is defined as voxels lying
inside the voxelized reference segmentation that are also a subset of Nd_centered ,
Cd_centered = Nd_centered ∩ Sr. The left subfigure of Figure 4.3 shows Nd and
Nd_centered . Nd_centered is created from the detected segmentation Sd by thin-
ning. A middle artery branch has a sinuous shape. This shape depicts an ex-
aggerated situation and explains why Nd_centered is needed for the calculation
of the Md metric. The middle subfigure shows Nr together with the reference
segmentation Sr and the right subfigure depicts the subsets Cd and Cd_centered .
An overlap rate Mo is similar to the quantity assessment in [45] and is cal-
culated as:
Mo =|Cd ||Nd |
, (4.1)
A detection rate Md is determined as:
Md =|Cd_centered |
|Nr|. (4.2)
Note that the set of detected skeleton voxels Nd can be seen as a sum of true-
positives (TP) and false-positives (FP). Similarly, Cd = Nd ∩Sr then represents
a set of TP and the set of reference skeleton voxels Nr reflects the sum of TP
and false-negatives (FN). If we substitute these variables into the definition of
Mo and Md , then we will get precision and recall measures, respectively.
Average distance error measureThe average distance error metric Derr measures the average distance error
between the detected skeleton and the reference skeleton and defines how far
on average they are from each other. In this measure, the distance is calculated
31
from each voxel of the detected skeleton to the closest voxel of the reference
skeleton and then averaged by the number of voxels. To avoid a bias this mea-
sure is calculated only on those parts of the detected skeleton that overlapped
the reference segmentation.
32
5. Vascular segmentation
This chapter provides an overview of basic methods which form the core
of current state-of-the art for vascular segmentation. The last segmentation
method presented in this chapter was used in our papers in the evaluation. and
was proposed by our collaborators Wang et al. [79].
5.1 SegmentationSegmentation is the process of dividing an image or a volume into regions
that are similar with respect to some characteristic, usually intensity, texture
or location in a space. Ideally, the regions should be meaningful for a specific
task or correspond to distinct anatomical or pathological structures. If the
anatomical structure of interest is a vascular tree we refer to the process of its
delineation as to the vascular segmentation.
Segmentation is often the most important step in medical image analysis
since all futher measurements, feature extractions or visualisations are derived
from the segmentation results. Concurrently, it is also considered the most dif-
ficult step, as there are many challenges. The methods should be able to handle
anatomic variability and complexity and be robust to noise, low contrast be-
tween the structures or presence of different imaging artefacts. Especially,
partial volume effect is an artifact that is very common in medical images. A
possible way to address this difficulty is to allow regions in the segmentation
to overlap. In the next chapter we will discuss a particular group of methods,
called coverage segmentation methods, that allow partial membership.
In this chapter, we focus on standard approaches, that enforce binary deci-
sions in the volume partitioning process - binary segmentation. A voxel pi, j,leither belongs to the object Sk and has assigned the membership value mk = 1
or does not belong to the object and it has assigned the membership value
mk = 0
mk(pi, j,l) =
{1 if pi, j,l ∈ Sk,
0 otherwise.(5.1)
5.2 Preprocessing: Vesselness filterThe presence of nonvascular structures in medical images may affect the qual-
ity of vascular segmentation. Some segmentation approaches, therefore, pre-
process the image in order to enhance tubular vascular structures and suppress
other non-tubular structures e.g., kidneys.
33
(a) plane-like structure (b) tubular structure (c) blob-like structure
Figure 5.1. Example of the structure shapes. Eigenvalue close to 0 represents low
intensity curvature in the direction of the corresponding eigenvector. Large eigenvalue
and its corresponding eigenvector represent the direction of high intensity curvature.
Vessel enhancement approaches are based on the observation that the inten-
sities along the vessel direction are changing slowly (the intensity curvature is
low) compared to the intensities on the cross-sectional plane which are chang-
ing fast (the intensity curvature is high). A measure that characterizes a local
intensity change is the second-order partial derivative and a square matrix of
these derivatives is the Hessian matrix. For a 3D input image, the Hessian
matrix H(p), corresponding to the voxel pi, j,l , is given by
H(p) =
⎡⎢⎢⎢⎣
∂ 2I∂x2
∂ 2I∂x∂y
∂ 2I∂x∂ z
∂ 2I∂y∂x
∂ 2I∂y2
∂ 2I∂y∂ z
∂ 2I∂ z∂x
∂ 2I∂ z∂y
∂ 2I∂ z2
⎤⎥⎥⎥⎦ , (5.2)
where I is the intensity function of the image at the voxel pi, j,l . The eigen-
values λ1,λ2 and λ3 (|λ1| ≤ |λ2| ≤ |λ3|), and their corresponding eigenvectors
e1, e2, e3 of the Hessian matrix are closely related to vascular intensity and
direction. Analyzing them allows us to recognize different local orientation
patterns e.g., blob-like, tubular, plane-like or no preferred direction. See Fig-
ure 5.1 for an example. Ideally, a tubular structure should have eigenvalues
corresponding to |λ1| ≈ 0, |λ1| |λ2|, λ2 ≈ λ3.
In practice, the second-order partial derivatives at voxel pi, j,l are calculated
by convolving the input image I with Gaussian derivative G at scale σ
Hσ (p) = I(p)∗ ∂ 2Gσ (p)∂ p2
. (5.3)
34
(a) σ = (0,2〉 (b) σ = (2,4〉 (c) σ = (4,6〉 (d) σ = (6,8〉Figure 5.2. The maximum intensity projection of the volume preprocessed by the
vesselness filter with different range of σ ; step size between sigmas was set to 0.25.
Vessels as well as bone edges that resemble locally a tubular structure respond to
sigma values that correspond to their sizes.
A vesselness filter proposed by Frangi et al. [17] based on Hessian matrix
theory is then defined at scale σ as:
ν(σ)=
{0 if λ2 > 0 or λ3 > 0,
(1− exp(− R2
A2α2
))exp
(− R2
B2β 2
)(1− exp
(− T 2
2γ2
)) otherwise,
(5.4)
where α , β and γ are sensitivity controlling thresholds for dissimilarity mea-
sures RA, RB and T , respectively. These measures are based on eigenvalues
of H(p) and are defined as
RB =|λ1|√|λ2λ3|
, (5.5)
RA =|λ2||λ3|
, (5.6)
T =√
λ 21 +λ 2
2 +λ 23 . (5.7)
The dissimilarity measure RB distinguishes blob-like structures from the plane-
like and tubular structures while the measure RA differentiates between plane-
like and tubular structures. The last measure T reduces the response of back-
ground voxels. To enhance a tubular object of different sizes, vesselness filter
has to be applied at multiple scales and the final response is performed by
keeping the maximum response over the scales. Figure 5.2 shows an example
of vesselness filter being applied at different scales to a volume image.
However, Hessian-based filters are sensitive to local deformations, such as
bifurcations, stenosis or aneurysms. They may induce subtantial spatial blur
if large scale σ is used (Figure 5.2d) and they may be computationally costly.
35
(a) (b) (c)Figure 5.3. Example of thresholding with the threshold range [150, 525] HU; (a) an
axial 2D slice of the original volume, (b) a thresholded 2D slice, (c) an ISO surface
rendering of the thresholded 3D volume; Vessels as well as bones were segmented
since the range of their intensities overlaps.
A mono-scale approach proposed by Bauer at al. [3] is less sensitive to struc-
ture deformations and uses a vector field obtained from gradient vector flow
diffusion [81].
5.3 Segmentation methods
There are many different approaches to obtain a segmentation and no approach
suits all purposes. A brief overview of the methods follows, however more
detailed overviews of other vascular segmentation techniques are provided in
papers [35, 38].
5.3.1 Thresholding
Thresholding [23] is the simplest segmentation method. It divides the vol-
ume into regions based only on the voxel intensity and it ignores the voxel
spatial position. A suitable intensity value, called threshold value, is selected
either manually or automatically by analyzing the volume histogram. All vox-
els having intensity smaller than the threshold value are then grouped into one
class, e.g., representing the object and all the other voxels are members of the
background class. This technique is well suited for segmenting anatomical
structures that have contrasting intensities compared to the rest of the volume
e.g., bones. To separate object from the background with nonuniform illu-
mination, local adaptive thresholding is well suited. This technique selects
a different threshold value for each voxel in the volume based on the local
36
(a) (b) (c)Figure 5.4. Example of region growing; (a) an axial 2D slice of the original volume,
(b) a thresholded 2D slice, (c) an ISO surface rendering of the thresholded 3D volume
with a seed point marked by the red dot.
intensity characteristics. Thresholding is often part of more complex image
processing pipelines.
When using thresholding for blood vessel segmentation, an intensity in-
terval, called threshold inverval, has to be applied. Only voxels having an
intensity from this interval range are grouped together into the vessel class.
However, intensity range that is typical for the vessels overlaps the bone in-
tensities and therefore thresholding alone is not sufficient approach for vascu-
lar segmentation. Figure 5.3 shows the result of thresholding with a threshold
range [150, 525] HU that was calculated from the volume intensity histogram
using the automatic method presented in Paper I.
5.3.2 Region-growing
Region-growing segmentation [1] starts from a seed voxel or a seed region and
succesively includes neighboring voxels if they fulfil inclusion criteria. These
criteria are usually based on intensity information or on edges detected in the
image. The segmentation growing process continues until no more voxels can
be added to the segmentation. Figure 5.4 shows an example of region-growing
initiated from a seed voxel marked by a red dot in Figure 5.4c. The inclusion
criterion was set to include any neighboring voxel that has intensity within
the range [150, 525] HU. The result compared to the thresholding method
contains only voxels that are connected with the seed. The advantage of this
method is the simplicity and computational efficiency. The disadvantage is the
sensitivity to noise. The method is prone to create holes inside the segmen-
tation, or leak into other structures if the inclusion criteria are locally insuffi-
cient. Many methods are reducing the risk of leakages by introducing some
37
growth-limiting criteria. We used a modification of region-growing algorithm
in Paper III which uses an intensity- and distance-limiting criteria.
Region-growing for vessel segmentation was utilized by Eiho et al. [12],
where the algorithm was performed on one vessel branch at the time so the
inclusion criteria could be adapted according to the local neighborhood char-
acteristics.
5.3.3 Centerline-based methods
Centerline-based segmentation methods either require a centerline on the input
or they detect it as the initial step. Subsequently the centerline is used to extract
the vascular surface through various geometric and spatial constraints.
There are approaches that detect the vessel wall independently in each 2D
cross-sectional plane utilizing 2D active contours [40], ray-casting schemes [26]
or circular dynamic programming [20]. A collection of 2D contours are then
joined into a 3D surface using geometric models. Other approaches are able to
extract the surface directly in 3D utilizing 3D parametric active surfaces [46],
B-spline tensor surfaces [16] or level set evolution [77,79]. A graph-cut-based
approach applied for coronary arteries was presented by Schaap et al. [67].
This method segments the vessel lumen with a Markov Random Field and
then removes falsely segmented regions with a kernel regression approach.
5.3.4 Geometric deformable model
Geometric deformable models were introduced concurrently by Malladi et
al. [43] and Caselles et al. [9]. They combine curve evolution theory [66]
with level set methods [55].
Starting with an initial curve C at t = 0, the image segmentation is per-
formed by evolving the shape of this curve according to the partial differential
equation∂C∂ t
=V (κ)N, (5.8)
which is driven by a speed function V (κ), where κ is the curvature and N is
the inward normal of the curve. The design of the speed function can differ
between the implementations and influences the segmentation performance.
Usually the speed function combines the speed of internal deformation (cur-vature deformation that shrinks the curve and/or constant deformation that
inflates the curve) with the speed dependent on the image data in such a way
that the curve evolution stops at object boundaries.
The evolving curves can be represented implicitly (using only geometric
computations without the need of parametrization) as a higher dimensional
level set function φ(t) defined on the same image domain. The advantage of
38
(a) (b) (c)Figure 5.5. An intersection of the level set function φ (blue surface) with a plane
xy at z=0 (red surface) is illustrated on the top image. This intersection defines an
evolving curve at different time points t. The curve (a dark blue contour) together
with the resulting segmentation (blue area enclosed by the curve) is shown on the
bottom picture.
such a representation is that the topology changes can be handled automati-
cally and multiple objects can be detected simultaneously. In this representa-
tion the curve is a set of points on the image domain for which the function
φ(t) is zero. The curve evolution is then represented by updating the level
set function φ(t) at fixed time points. Figure 5.5 illustrates the principle of
the level set method. The equation 5.8 which specify the curve evolution is
replaced when using the level set method by new evolution equation
∂φ∂ t
=V (κ)|∇φ |. (5.9)
The initial level set function is often based on the signed distance function
D(x,y) from each point on the grid to the zero-level set, φ(x,y,0) = D(x,y).All the formulations were explained for 2D segmentation, however they can
be extended to 3D.
In conventional model based segmentation methods, the curve propagation
is often limited by the statistical shape models. Creating statistical shape mod-
els for vessels is difficult due to large anatomical variation between different
subjects. Therefore, in the work of Wang et al. [79] which is used in our vas-
cular segmentation the curve propagation is limited by a vessel model derived
as a union of several cylinders from the inserted centerline. The method is
detailed in the following subsection.
39
5.3.5 Skeleton guided level set based vessel segmentation
This vessel segmentation method was proposed by our collaborators Wang et
al. [79] and is intended to utilize the vascular centerline tree extracted by the
method described in Paper I.
The method constructs an implicit 3D vessel model φ from the input cen-
terlines as a union of cylinders with a varying radius R(x), where x is a point
on the centerline L. The initial vessel radius is assumed to be 1 voxel uni-
formly for the entire vessel tree. This vessel model is then incorporated in the
speed function and regulates the growth of the vessel contour. The evolution
equation, used in this work, is defined as
∂φ∂ t
= (αvimage +βvmodel + γvinternal)|∇φ(x)|, (5.10)
where α,β ,γ are the weighting factors for every speed term. The speed terms
are defined as
vimage = S−|I(x)−T |, (5.11)
vmodel =−d3m(x), (5.12)
vinternal = κ(x), (5.13)
where I(x) is the intensity of the input image at x, T is the center and S is
the width of the thresholding window. The vessel tree model is represented
by a signed distance map dm, that is generated by a distance transform from
the centerlines, with points on the centerlines set to their initial values −R(x).Then on the surface of the model dm(x) is zero, it is negative inside the model
and positive if it is outside.
After evolving the level set φ , some re-modeling steps are needed. New
centerlines are extracted from the shape given by the zero level set through a
fast marching scheme. New radius function R(x) is estimated for every vessel
branch by fitting a smoothed curve to the distance function measured from the
center points to the current zero level set, dl(x), with x ∈ L. Using the new
centerlines and radius functions, an updated 3D tree model is created. The
steps of the level set propagation and vessel model re-estimation are iteratively
repeated until convergence.
The advantage of this method is that the input centerlines do not need to
be perfectly centered, their approximate position is sufficient. The centerlines
will be re-centered during the algorithm iterations. This enables to use a fast
centerline extraction algorithm to detect the approximate centerline tree pos-
sition quickly and then re-center this position while preforming the vascular
segmentation. The resulting segmentation has, thanks to utilizing the geomet-
ric deformable models, subpixel precision, which can be precious.
40
6. Coverage segmentation
Generalization of the binary segmentation is a fuzzy segmentation based on
fuzzy set theory [82]. Fuzzy segmentation, compared to the binary one, parti-
tions a volume into overlapping regions and allows voxels to belong to more
than one object. This reduces the risk of assigning a voxel to the wrong ob-
ject and preserves a larger amount of the information which results in more
precise measurements and feature extractions. Each voxel in the fuzzy seg-
mentation is associated with a membership value, in the range from zero to
one, which represents the degree of voxel membership to a particular volume
objects. If the membership values correspond to the voxel coverage by an
observed image object, we call such a partitioning a coverage segmentation.
In the following sections we formally define fuzzy set theory and present the
coverage representation. Several methods for the coverage segmentation will
also be covered. For a more detailed description of coverage model and its use
in image processing the reader is refered to [72].
6.1 Fuzzy set theory
A fuzzy subset S of a reference set X is a set of ordered pairs
S = {(x,μS(x))|x ∈ X}, (6.1)
where μS : X → [0,1] is the membership function of S in X [82]. Important
notions related to fuzzy set are support and core. The support of S is set of
points having strictly positive memberships to the set S and the core of fuzzy
set S is defined as the set of points with memberships to S equal to 1.
6.2 Coverage representation
Coverage segmentation uses a representation where the membership function
values correspond to the portion of volume of the voxel covered by the object,
i.e., to the voxel coverage. Formally, coverage segmentation S(I) of an image
I into m components is defined as a set of ordered pairs
S(I) ={(
(i, j, l),α(i, j,l))∣∣∣(i, j, l) ∈ ID
}, (6.2)
41
(a) (b) (c)Figure 6.1. Example of a vessel-like object (a) high resolusion binary representation,
(b) low resolution coverage representation, (c) low resolution binary representation
where
α(i, j,l) = (α1, ...,αm),m
∑k=1
αk = 1, αk =V (pi, j,l
⋂Sk)
V (pi, j,l),
and where Sk ⊂ R3 is the extent of the k-th component (object), V (pi, j,l) de-
notes the volume of a voxel pi, j,l and αk are coverage values. ID ⊆ Z3 is the
discrete image domain. It is simple to compute coverage values αk for voxels
covered by simple continouos geometric objects Sk that can be analytically
defined. However, to extract this information in general, from more complex
and more realistic objects in digital volumes, we have to estimate it from the
image data using coverage segmentation methods. An example of a vessel re-
sembling object is given in Figure 6.1. Coverage segmentation is preserving
the smoothness and other features of the object better than the binary segmen-
tation.
6.3 Coverage segmentation methods
Coverage segmentation of an object is very close to its binary segmentation.
For voxels that are fully covered by, either the object or the background, the
two segmentations are identical. Only for the border voxels, where the par-
tial coverage appear, the values differ. Therefore, the coverage segmentation
methods may focus on adjusting an already existing binary segmentation, in-
stead of re-inventing the whole range of different segmentation methods again.
In the following subsections, we first present a simple coverage segmentation
method based on direct intensity mapping, then one which builds on an ex-
isting binary segmentation, and, finally, one more general method based on
energy minimization, which does not require a priori knowledge about the
binary segmentation of the object in the image.
42
6.3.1 Coverage segmentation based on double thresholding
Coverage segmentation based on double thresholding [70] is the simplest cov-
erage segmentation method. A prerequisite to a successful application of this
method is a volume where the intensity distributions of the object and the
background are well separable; then the range of grey-levels on the transition
between the two classes can be used for estimating coverage values.
The method seeks for a couple of threshold values, f and b, where f is
the intensity estimation of voxels completely covered by the object and bis the intensity estimation of completely covered background voxels. These
thresholds can be found automatically based on the method presented in [70].
The method estimates the values in such a way that the fuzzy boundary be-
tween the object and the background is not more than one voxel thick and the
contrast between them is as large as possible. Voxels darker than b belong
completely to the background and are therefore assigned coverage value zero,
voxels brighter than f belong completely to the foreground and are assigned
coverage value one. The greyscale intensities I(pi, j,l) of the one voxel thick
separating boundary are then normalized between values f and b
α(pi, j,l) =I(pi, j,l)−b
f −b. (6.3)
The new normalised values represent the coverage values of the boundary vox-
els.
6.3.2 Coverage segmentation by local unmixing
Coverage segmentation by local unmixing [71] is based on an existing binary
segmentation. It is assumed that this binary segmentation is trustworthy for all
but boundary voxels. The method identifies the set of these boundary voxels Bas all voxels that are 6-connected to a voxel with a different label. Two major
steps, linear unmixing and thinning, are then performed on the set B.
Linear unmixing stepFor each voxel in this set, the partial coverage value is calculated utilizing a
local linear mixture model. Based on this model, the image intensity I(p) of
the boundary voxel p is assumed, in a noise-free environment, to be a convex
combination of the values ck corresponding to the pure class k, for m classes,
k = 1, . . . ,m:
I(p) =m
∑k=1
αkck,m
∑k=1
αk = 1, αk ≥ 0, (6.4)
where each coefficient αk represents the coverage of the pixel p by a class
k. The pure class representatives ck are usually not known and are estimated
locally from a weighted average of completely covered voxels from the bi-
nary segmentation. In real imaging conditions, equation 6.4 can not be solved
43
exactly due to a presence of noise. The problem has to be reformulated as find-
ing an image intentisity I∗(p) = ∑mk=1 α∗
k ck and I∗(p) is as similar as possible
to I(p). This allows to formulate coverage segmentation as an optimization
problem, with the data fidelity term D(p) = ‖I(p)− I∗(p)‖2. See [71] for
details.
Thinning stepWe assume that the continuous objects have a reasonably smooth and crisp
boundary. This implies that the fuzzy boundary should not be more than one
voxel thick. Due to presence of noise, set B may be thicker and needs to be
thinned in the final step. In the 2D version of the algorithm, the thinning is
performed using an ordered thinning algorithm. In ordered thinning, pixels
that have the smallest difference in intensity to one of the pure classes are
assigned the membership value of that particular pure class. This iteratively
repeats until the resulting set of pixels creates a 4-connected one pixel thick
boundary between the neighbouring objects. In the 3D version of the algo-
rithm, the ordered thinning would be a rather complex procedure when the
topology preserving is required. In Paper V we propose an alternative ap-
proach to the boundary thinning in 3D based on morphological erosion. This
alternative morphological approach can be perfomed equally well in 2D and
offers an alternative method for segmenting thin and elongated objects.
6.3.3 Energy based coverage segmentation
Energy based coverage segmentation [41] performs the whole segmentation
as one optimization task. For each object, the method requires a sample of the
pure class representatives on the input. This can be achieved either by defin-
ing small sample regions of the pure class representatives or by providing a
binary segmentation of each object (the latter is the case for the local unmix-
ing based method [71]). The method then finds the coverage segmentation
S by minimizing the energy function J over the space of possible coverage
segmentations, with several regularization terms
J(S) = D(S)+μP(S)+υT (S)+ξ F(S), (6.5)
where D(S), P(S), T (S) and F(S) is the data term, overall perimeter, bound-
ary thickness and total image fuzziness, respectively. The weighting parame-
ters μ , υ and ξ ≥ 0 give appropriate importance to the different terms. The
data term D(S) itself provides an unmixing segmentation and the rest of the
terms add spatial information which makes the coverage segmentation less
sensitive to noise. The argument S which minimizes J provides a coverage
segmentation with a smooth boundary of each objects and support majority of
object pixels to be classified as pure. This results in improved performance
compared to coverage segmentaion based on local unmixing. However, due to
44
(a) (b) (c) (d) (e)Figure 6.2. Comparison of the coverage segmentation methods performed on a piece
of a retinal vessel, (a) original image with the input binary segmentation marked with
white contour, (b) outcome from the linear unmixing before the thinning step, (c)
thinned result of the local unmixing based coverage segmentation [71], (d) result
of the energy based coverage segmentation [41] with the pure class representatives
marked with red squares, (e) result of our coverage segmentation method proposed in
Paper IV.
iterative optimization, energy based method is more computationally demand-
ing. See [41] for further details.
6.4 Coverage segmentation applied to thin structuresThe presented coverage segmentation methods are based on the asumption
that the boundary of continuous structures is reasonably smooth and the fuzzy
boundary between the neighbouring structures is not more than one pixel
thick. This assumption is correct for thick structures, however, in case of e.g.,
retinal vessels, hairs on the skin or microtubules in the cytoskeleton, which are
often only 1-2 pixels thick in total this assumption may not hold.
The coverage segmentation method based on local unmixing reduces, in the
ordered thinning step, the thickness of the structure boundary as long as the
topology stays preserved. This means that all partially covered pixels from the
tips of long vessel branches are removed until the nearest pure structure pixel
is reached. This is undesired, since important sub-voxel information, wich we
aim at capturing by coverage representation, is lost.
The difficulty with the energy based coverage segmentation is mainly a
strong similarity between the object and the background intensities and a dif-
ficult selection of proper regularization parameters. This leads to a noisy and
incorrect coverage segmentation.
A comparison between different coverage segmentation methods, performed
on a 2D image containing a segment of thin and long retinal vessel is shown
in Figure 6.2. Crisp segmentation incorporates many partially covered pixels,
which is clearly visible in comparison with Figure 6.2b. Only a few bright pix-
els are fully covered by the object and all the remaining ones are mixed. Such
an oversegmentation, as presented in Figure 6.2a leads to an object that will
45
unavoidably provide poor measures (e.g., object area, length, thickness, etc).
Figure 6.2c clearly shows undesirable effects of the method presented in [71],
where topological thinning does not ensure preservation of partially covered
elongated structures. Figure 6.2d indicates deficiencies of the energy based
method [41], leading to a noisy and oversegmented result. All these issues are
successfully overcome by our proposed method presented in Paper IV, suited
for coverage segmentation of thin structures. This algorithm was also applied
to artefacts removal in our Related Paper 3, which led to improved skin lession
segmentation in dermoscopic images.
46
7. Deep neural networks
This chapter gives a brief introduction to deep neural networks for supervised
learning with the main focus on convolutional neural networks (CNN) to pro-
vide a background for Paper VI. As the topic of deep learning in neural net-
works is very broad, to get a deeper understanding we recommend exhaustive
review papers such as [37,68] or the books by Nielsen [51] and Goodfellow et
al. [24].
7.1 Deep learning in neural networks
Deep learning belongs to the class of machine learning algorithms which fo-
cuses on pattern analysis, classification, feature extraction and transforma-
tion [11]. Humans are remarkably good in these activities, and therefore, it
is not surprising that visual cortex (an area of the brain which is responsible
for recognizing patterns) inspired the way how deep learning algorithms work.
Study on a visual cortex showed that simple cells respond to certain visual in-
puts, e.g., the presence of edges of a certain orientation. These simple cells are
organised in a columnar architecture and all cells together are able to produce
a visual perception [32, 33].
Similarly, neural networks for deep learning consists of many layers of lin-
ear and non-linear information processing ordered into a cascade. The signal
path traverses from the first (input) layer, to the last (output) layer. This direc-
tion of passing through the network is called forward pass. Between the input
and the output layer there are one or more layers referred to as hidden layers.
The first hidden layer detects low level features (edges, colors,...), the second
hidden layer detects features which are a combination of the previous ones
(corners, basic shapes,...) and the more we advance through the network the
more complex and abstract these features become. The last layer can easily
detect or classify high-level features, e.g., faces, persons, dogs, etc.
Each layer consists of elements called neurons. Outputs from the neurons in
one layer are the inputs to the neurons in the next layer. Each neuron receives
a number of input values xi and calculates their weighted sum using weights
wi j and adds a bias b j to it. The resulting value is a parameter for an activation
function f (a linear or non-linear transformation) which decides the output
value y j. This can be summarized with formula
y j = f (∑i
wi jxi +b j). (7.1)
47
Network training is the process of finding appropriate weights wi j and a bias b jfor each neuron automatically. Training requires a labeled training dataset and
is usually done using back-propagation. During the training process, labeled
training samples are classified by the network and resulting classifications are
compared to the labels. Then a loss function, which specifies the final error
of the network performance, is calculated. The loss function is minimized
by passing the gradient of the error back through the network and adjusting
the weights and biases by an amount proportional to their contribution to the
error. The contribution to the error (the rate of change of error with respect
to the parameter) is calculated by applying the chain rule of differentiation.
Passing through the network in the direction from the output layer back to
the input layer is called backward pass. After the gradient of the error with
respect to all the parameters are calculated the parameters are updated. The
parameter update is usually done by stochastic gradient descent or its variant.
The forward pass, the backward pass and the parameter update are repeated
until a satisfactory network performance is reached [34, 51].
The networks usually consist of different types of layers which can be or-
dered in the cascade in many different ways. The order and type of the layers
define the network architecture. One type of architecture is a convolutional
neural network.
7.2 Convolutional neural networks
Convolutional neural network (CNN) is a neural network architecture special-
ized in processing data that has a grid-like topology, e.g., images, therefore it
is used for computer vision and image analysis applications [24].
The CNN architecture contains at least one layer that employs a mathemat-
ical operation called convolution. This layer is referred to as convolutional
layer. The traditional CNN architecture consists of more than one convolu-
tional layer interleaved with nonlinear and pooling layers and the network
ends with fully connected layers. In the following subsections we will briefly
describe the most popular types of layers.
Compared to a regular neural network, the layers in CNN have neurons ar-
ranged not in a vector, but rather in a volume of three dimensions: width,
height and depth. The first hidden layer receives a flat and shallow input
volume of activations (outputs of the neurons) having the same dimensions
as the input image (e.g., 32×32×3 for a color image of size 32×32). The
CNN architecture transforms one volume of activations to another one result-
ing into a single vector of class scores, arranged along the depth dimension
(e.g., 1×1×6) [39]. An example of simple CNN architecture illustrates this
transformation of volume of activations in Figure 7.1.
48
Figure 7.1. Simple CNN network architecture consisting of two convolutional layers
interleaved with two pooling layers and one final fully-connected layer. The input
activation volume of size 32×32×3 transformed to 1×1×6 vector of output classes.
7.2.1 Convolutional layer
The convolutional layer is the core building block of a CNN. It consists of a
set of 2D learnable convolution filters (kernels). The size of these filters is
relatively small, usually between 1×1 to 11×11 pixels, and is referred to as
a receptive field of the filter. The receptive field is a user-defined parameter.
A neuron in this layer is connected only with those neurons from the previous
layer which lie in its receptive field [39].
Each filter slides (convolves) across the input volume and at every spatial
position it computes the dot product between the weights of the filter and the
input values. The result is one 2D activation map per filter and represents the
responses of that filter at every spatial position. In convolutional layers, the fil-
ter weights are shared, i.e., they are independent regarding the position where
they are applied and the update of the weights during the backpropagation is
consistent across the full image. Shared weigths and small receptive fields re-
sult in several orders of magnitude less weights compared to fully connected
networks and allow a more efficient computation [24].
In addition to receptive fields, there are other user-defined parameters as-
sociated with each layer that control the size of the output activation volume:
the depth, stride and zero-padding. The depth corresponds to the number of
filters used in each layer. The stride specifies the horizontal and vertical slide
of the filter. Zero-padding enlarge the input volume by certain number of ze-
ros around the border and controls the spatial size of the output volume. It is
mostly used to preserve the input volume size.
7.2.2 Pooling layer
The pooling layer down-samples the filter response from the previous layer by
applying a non-linear function. A common type of the layer is max poolingwhich applies the max function. In max pooling layers, the input for the layer
is partitioned into a non-overlapping rectangles and the maximum value of
49
each partition is the new output value. The reason behind this is that only the
most dominant features will propagate further through the network and their
relative position compared to the other dominant features is more important
than their exact position. By reducing the number of features, this layer helps
prevent overfitting. The user-defined paramters in this layer are: the type, sizeand stride.
7.2.3 Rectified linear unit (ReLU)
The ReLU is one type of activation function which determines the output
through a non-linear transformation defined as
f (x) = max(0,x), (7.2)
where x is the input to the ReLU. Other types of activation functions are, e.g.,
the sigmoid function or the hyperbolic tangent function.
7.2.4 Batch normalization layer
The batch normalization layer normalizes the activations of the previous layer
at each batch (a subset of the training set), i.e., it applies a transformation that
results in the mean activation being close to 0 and standard deviation of the
activation being close to 1.
7.2.5 Fully-connected layer
The last layer is always a fully-connected layer. This layer, like the regular
neural network layers, connects each neuron in the layer to every single activa-
tion from the previous layer. Fully connected layers result in one-dimensional
output and therefore there can not be a convolution layer placed after a fully-
connected one.
7.3 Our CNN classifier
In Paper VI we designed a CNN architecture for classification purposes. The
input to this network are 2D images representing a potential vascular cross-
section. Each image has a constant size 31×31 pixels and depicts vessels
or non-vessels of different sizes. The output of the network gives a decision
whether the 2D image is truly a vascular cross-section or not. Further details
and the architecture description of the proposed CNN classifier are provided
in Chapter 8 and Paper VI.
50
8. Contributions
This chapter summarizes the methods and results presented in the appended
publications. The background and context to this work was provided in the
previous chapters.
8.1 Fast vascular skeleton extraction algorithm (Papers Iand II)
Papers I and II present a fast and fully automatic vascular skeleton extraction
algorithm. This algorithm is intended to be a part of a complete arterial tree
segmentation framework, where the approximate vascular skeleton is used as a
seed region for subsequent segmentation algorithms. In Paper I, we developed
a method that focuses on extracting a skeleton from healthy arteries of vary-
ing sizes. Paper II expands the algorithm and focuses on diseased peripheral
arteries, i.e., arteries with severe calcifications and collateral arteries.
MethodNext, we discuss the algorithm for both methods, the one for heatlhy arteries
and the one for diseased ones.
Skeleton of healthy arteries: The method takes a 3D unprocessed CTA
scan as input and produces a graph representing an approximate vascular skele-
ton. The nodes of the graph represent centrally located voxels within arterial
cross-sections and edges of the graph represent direct connections between the
nodes within a vascular branch. The approximate skeleton of large and small
arteries is extracted in two algorithm levels starting with the skeleton of large
arteries. Each level consists of four main steps:
1. A parameter selection step automatically selects appropriate parameters
based on vascular morphology and the intensity histogram of the data.
2. An artery node detection step detects voxels that are centrally located
within potential arterial cross-sections and applies a set of knowledge-
based filters to reduce false-positives.
3. An artery node connection step iteratively connects the nodes utilizing
the distance and intensity information and constructs a graph structure
representing the approximate vascular skeleton.
4. An anatomy based analysis step identifies and removes spurious graphs
or graph segments and keeps only those graphs that correspond to the
vascular tree.
51
Skeleton of diseased arteries: The method for extracting a skeleton of
diseased arteries expands the previous version of the algorithm by adding two
new algorithm levels thereby creating a cascading structure. The first level,
where the skeleton of large arteries is detected, serves as a reliable basis and
new graph segments are appended to this basis in the subsequent levels. The
first additional level extracts the skeleton of arteries which contain severe cal-
cifications and the second additional level detects the skeleton of tiny arteries
in the lower extremities and collateral arteries. The same four main steps that
were presented in the previous version of the algorithm are performed in every
level of the expanded version. The main difference is in the selection of the
parameter values which are chosen to be well suited for detecting the desired
appearance of diseased arteries. The cascading structure of the algorithm pro-
vides a possibility to add more new levels, where each level might be adapted
to a new vascular disease.
ResultsNext, we discuss the results for each of the proposed methods, the one for
heatlhy arteries and the one for diseased ones.
Skeleton of healthy arteries: The method was evaluated on 25 CTA scans
of the lower limbs taken from the clinical routine. The proposed method
achieved a very good overlap rate and a good detection rate compared to a
united skeleton, which is a union of the reference skeleton and the resulting
skeleton corrected by the radiologist. However, the algorithm had difficulties
to detect the skeleton of diseased vascular segments. The resulting average
distance error between the detected and united skeleton was 1.35 mm which
reflects the fact that the approximate skeleton consists of polyline segments
instead of arc segments. The average computational time of the algorithm
was 88 seconds per CTA volume using a single-threaded implementation. It
is possible to parallelize the algorithm and by utilizing four CPU cores with
2 threads per core, the average running time dropped to 29 seconds. This is a
considerable improvement compared to ca. 15-20 min per CTA volume which
Figure 8.1. Detected skeleton printed in red colour is overlayed with a volume region
depicting bones of a right foot displayed in isosurface rendering mode with intensity
level equal to 600 HU. The vascular skeleton was successfully extracted even from
very distal arteries in the right foot.
52
Figure 8.2. Comparison of the skeletons: (a) maximum intensity projection (MIP)
of the volume with MIP of three cutouts with manually removed bones are shown.
The cutouts depict calcified iliac artery (top) and occluded femoral arteries (middle,
bottom), (b) the reference skeleton, (c) the resulting skeleton from Paper I, (d) the re-
sulting skeleton from Paper II. Resulting skeleton in (d) contains more small branches
compared to the two other skeletons. However, it also detected a false artery segment
marked with a red arrow and did not connect occluded femoral arteries.
was the time the radiologist needed to extract the reference skeleton using a
semi-automatic method.
Skeleton of diseased arteries: The extended version of the algorithm was
evaluated on the same dataset as the previous version of the method in order
to make results comparable. The new results showed the improvement in the
detection rate, which was also confirmed by a visual assessment. The result-
ing skeletons were more complete with longer branches and contained less
disconnections. An example of a successful extraction of skeleton from distal
arteries in the right foot is shown in Figure 8.1. A connection between two
occluded arterial parts, however, was not always created because the resolu-
tion did not provide sufficient intensity evidence to identify this connection.
The increased sensitivity of the algorithm to detect small artery skeletons in-
creased the number of false artery segments compared to the previous version
of the method. This is reflected by a decreased overlap rate value. The false
artery segments were detected mainly on surface of bone or in regions where
the beam hardening artifacts were present. The average running time of the
53
Figure 8.3. The histogram of a single case is shown with the three relevant ranges
detected automatically (fat, muscle and blood vessels). Logarithmic histogram shows
a peak that represents bone tissue having intensities ca. within the range [500,1500]
HU.
algorithm was 196 s per CTA volume using a single-threaded implementation
and 70 seconds per CTA volume using the parallelized version, which suits
the needs of clinical examination. Figure 8.2 depicts a comparision of the ref-
erence skeleton with the two resulting skeletons obtained by the two proposed
versions of the skeleton extraction algorithm.
8.2 Automatic adaptation to intensity variation betweenscans (Paper I)
Paper I features a parameter selection step that automatically selects the in-
tensity ranges of tissues relevant to our proposed algorithm (i.e., fat, muscles
and blood) for each CTA scan. The Hounsfield scale should be relatively stan-
dardised between CT machines for most soft tissues. However, the HU value
of contrast-mixed blood may vary considerably between the scans. This is
caused mainly due to variations in timing between injection and image ac-
quisition and variations in hemodynamics. We propose a volume intensity
histogram analysis based on Gaussian curve fitting.
MethodA smoothed histogram of CTA volume intensities can be, in general, described
as a succession of the following peaks: a peak reflecting air inside the volume,
two peaks representing fat and muscles and a small peak describing blood
54
vessel intensities. A peak representing bone tissue is clearly visible only if we
plot a logarithmic histogram. Figure 8.3 shows the histogram and logarithmic
histogram of a single CTA volume. To find the intensity ranges corresponding
to fat, muscles and blood, a sum of Gaussian curves is fitted to the image
histogram using the non-linear least squares fitting method. If any of the peaks
is too flat for the Gaussian curve to fit the data reasonably well, fallback ranges
are used. The fallback ranges might be particularly useful when the method is
applied to a cropped CT volume, where all tissues are not necessarily present
in sufficient quantities.
ResultsThe automatic parameter selection was tested on 25 CTA scans and the de-
tected intensity ranges roughly corresponded to the typical HU values of par-
ticular tissue reported in the literature. In eight cases the fallback range for
blood intensities was used and we observed that in these cases the proportion
of arteries containing calcifications was higher than in the rest of the volumes.
The high proportion of calcification may have caused that the typical blood
vessel intensities were not present in a sufficient amount to be properly re-
flected in the volume histogram.
8.3 Skeleton-guided onion-kernel based segmentation(Paper III)
Paper III proposes a rapid airway-tree segmentation algorithm. The develop-
ment of this algorithm is part of a joint project on acute respiratory distress
syndrome (ARDS). This project includes researchers from France, Colombia,
Poland and Sweden. The main goal of the project is to reduce the mortality
rate in ARDS by analyzing the lung aeration on CT images in order to learn to
predict the response of the lungs to different ventilation strategies. A more de-
tailed statement on this problem is provided in Section 3.2.3. The airway-tree
segmentation is needed for the removal of non-parenchymal structures and for
providing anatomical landmarks which are important for the lung registration
and subsequent delineation. This segmentation method was developed for the
purpose of airway segmentation, however, it is possible to use it for vessels
segmentation without the need of any modifications other than appropriate
parametrization.
MethodThe airway-tree segmentation algorithm consists of two main steps:
1. An airway-tree skeleton detection step follows the same main steps as
the vascular skeleton detection algorithm presented in Paper I. However,
to successfully address a challenging intensity heterogeneity of ARDS
55
(a) (b) (c)Figure 8.4. Onion-kernel region-growing, (a) the input skeleton is voxelized, (b) each
skeleton voxel becomes a seed; the segmentation is not allowed to grow backwards
to fill the cavities (the dark green border delineates the region that would be filled
otherwise, (c) the final segmentation.
the parameters and the set of knowledge-based filters have to be modified
to reflect the appearance of the airways in the diseased lungs. Three
classes representing three different levels of voxel aeration are defined
and each potential airway node is treated by following the rules defined
by the class corresponding to its intensity. For each airway node the
radius and intensity range corresponding to the aeration class of the node
is saved.
2. A modified onion-kernel region-growing segmentation step is initiated
from the detected airway-tree skeleton. The poly-line segments are vox-
elized (Figure 8.4a) and each voxel becomes a seed for an ordered region-
growing propagation (Figure 8.4b). From the seed voxel the algorithm
propagates only outwards, with the same speed in all directions and adds
a new voxel only if it is 26-connected to another voxel segmented in the
earlier layer. This ensures locally convex segmentation without segmen-
tation “overhangs” (Figure 8.4c) and limits propagating sideways when
a leakage into parenchyma occurs. To restrict the size of the leakage, the
propagation is limited also by distance and intensity constraints.
ResultsThe airway-tree segmentation algorithm was tested against a reference seg-
mentation based on a traditional region growing algorithm introduced by Mori
et al. [49]. A qualitative assessment was performed on 70 thoracic CT vol-
umes of piglets with induced ARDS acquired at various volume and pressure
settings. The proposed method detected a larger number of small branches
compared to the reference segmentation, particularly in images acquired at
low-pressure conditions (See Figure 8.5 bottom row). The reason for obtain-
ing notably better results in images with lower PEEP is a locally increased
contrast between dense parenchyma and the airway lumen. A higher num-
ber of detected branches leads to more bifurcation points which can serve as
anatomical landmarks.
56
Figure 8.5. Qualitative airway segmentation results, (a) Coronal slice of a thoracic
CT showing the differences in parenchyma intensities between high and low PEEP,
(b) result of the reference method, (c) result of the reference method overlayed with a
skeleton from the proposed method (d) result of the proposed method.
Since the proposed airway-tree segmentation algorithm is meant to be a
part of a larger lung registration and segmentation framework, the quantita-
tive evaluation of the algorithm was performed indirectly by comparing two
registration-based lung-segmentation methods. A method using a hybrid reg-
istration, where intensity information was enriched with airway-tree landmark
correspondences yields to improved lung segmentation compared to a tradi-
tional intensity-driven registration.
In addition, two hybrid registration methods were compared. One method
used landmark correspondences extracted from the proposed segmentation and
the other one used the landmark correspondences from the reference segmen-
tation. This two registration methods produced comparable results, which re-
flects the fact that even though the number of landmarks is notably greater
using the proposed method, the spatial distribution of the landmarks cover an
area similar to the area of landmarks obtained using the reference method.
This points to a limitation of the hybrid registration.
The complete proposed segmentation method achieves the average com-
putation time of 43 seconds per thoracic CT volume when running a single-
threaded implementation. The skeleton extraction algorithm takes 41 seconds
on average and the onion-kernel region growing algorithm takes an average
time of only 2 seconds.
57
8.4 Coverage segmentation of thin structures (Papers IVand V)
Coverage segmentation methods, as introduced in Chapter 6, are based on
the assumption that the fuzzy boundary between the neighboring structures
is only one voxel thick. This assumption may not hold for 1-2 pixel thick
elongated structures, e.g., retinal vessels, hairs on the skin or microtubules
in the cytoskeleton. In Paper IV, we developed a 2D method for coverage
segmentation, especially suited for precise segmentation of thin structures.
In Paper V, we expanded this method to 3D. In addition, we suggested an
implementation that enables lower memory consumption and lower processing
time.
MethodNext, we discuss the algorithm for the 2D and 3D versions of the method.
2D version: The method starts with an original grayscale image and an
existing binary segmentation of an object. To achieve a coverage segmentation
it performs four steps:
1. A linear unmixing step is utilized the same way as in [71] to obtain
preliminary coverage values.
2. An upsampling and centre of gravity attraction step combines informa-
tion available from the linear unmixing with the information from the
original image and estimates a new binary segmentation at increased
spatial resolution. Coverage information indicates how large area of a
pixel is covered by the object. This information allows to estimate the
number of fully covered pixels in the new high-resolution image corre-
sponding to this coverage value. Information from the original grayscale
image is needed for estimating the position of these pixels by using a lo-
cal centre of gravity attraction.
3. A minipixel shifting step corrects holes and protrusions that may appear
in the high-resolution binary segmentation by locally re-arranging the
pixel positions guided by the original binary segmentation.
4. The final down-sampling step converts the high-resolution binary seg-
mentation into a coverage segmentation of the input image size.
3D version: A coverage segmentation of 3D thin structures is an expansion
of the 2D method and consists of analogous steps. The main difference is in the
upsampling step. An input volume upsampled by an integer factor of n (where
n = 3) would lead to very large volumes to store and to process. Therefore,
our algorithm proposes a modification, where a sliding cube of size 4×4×4
is used to scan through the volume and it up-samples only a couple of voxels
within the cube at a time. This modification radically descreases the execution
time, where the modified version is 12 times faster than the original one. The
two versions exhibit similar performance in terms of the average absolute error
of the coverage values.
58
(a) (b)Figure 8.6. Average absolute error of coverage values of object border pixels obtained
by (a) a 2D version of the method tested on a thin star, (b) a 3D version of the method
tested on thin conical spiral, at noise levels ranging 0%-40%. Examples of the tested
objects are shown in the upper left corner.
ResultsNext, we discuss the results for the 2D and 3D versions of the method.
2D version: The evaluation of the 2D version of the method was performed
qualitatively using retinal vessel images. See Figure 6.2 for an example. The
proposed method is confirmed to outperform previous versions, [41, 71], in
case of thin objects, i.e., the method does not produce unintuitively thick
fuzzy boundaries, does not shrink the vessels and is not sensitive to similar
intensities of noise and vessels. Quantitative evaluation regarding sensitiv-
ity to noise was performed observing two types of synthetic objects (thin star
and thin rounded square) randomly placed at 50 different positions within an
image at each observed noise level. We evaluated the average absolute error
of the coverage values over 50 repetitions at nine different levels of additive
uncorrelated Gaussian noise with standard deviation σ up to 40% of the fore-
ground/background contrast. The graph in Figure 8.6 (a) illustrates the results
and shows that the proposed method improved the performance, especially in
the presence of noise. All considered coverage segmentation methods, even
in the presence of noise, outperform crisp segmentation applied to noise-free
objects. The proposed method outperforms all other methods as soon as the
noise level exceeds 5%, and is better than the noise free crisp segmentation
even when the image is corrupted by as much as 25% of noise.
3D version: The evaluation of the 3D version of the method was performed
qualitatively on real 3D CTA images and quantitatively by measuring the av-
erage absolute error of coverage values at increasing noise levels using two
synthetic objects. The graph in Figure 8.6 (b) confirms very good perfor-
mance of the method in the 3D case as well. The accuracy of voxel coverage
59
values estimated by the proposed method is higher even in the presence of up
to 35% of additive uncorrelated Gaussian noise, than what is achieved by crisp
segmentation in a noise free-case.
The algorithm for coverage segmentation of thin structures was applied also
to thin artefact removal which lead to improved skin lession segmentation in
dermoscopic images, as presented in reference [42] which is listed as Related
Paper 3 in this thesis.
8.5 High resolution crisp reconstruction (Papers IV andV)
Papers IV and V provide a method for high resolution binary reconstruction
of a structure. High-resolution binary images are a by-product of the coverage
segmentation method and are obtained by skipping the final down-sampling
step of the method. To obtain a high-resolution volume image, the 3D version
of the method has to run without the sliding-cube modification.
ResultsThe 2D method was compared with the nearest neighbor up-sampling method
on 15 manually-segmented ground truth segmentations of retinal images. Pre-
cision, recall and f-score, as well as specific features describing vasculature
(i.e., connectivity, area and length), are computed based on pixel-wise com-
parisons. It is shown that the proposed method outperforms the basic upsam-
pling method. Same conclusion can be derived for the 3D method which was
compared with the nearest neighbor up-sampling using 20 synthetic 3D con-
ical spirals by measuring precision, recall and f-score. Figure 8.7 shows an
example of resulting images both for 2D and 3D case.
8.6 Boundary thinning in 3D (Paper V)
Coverage segmentation by local unmixing contains a boundary thinning step
where a set of coverage pixels is reduced to a one-pixel-thick fuzzy bound-
ary between the structures. In 2D, this step is performed by ordered thinning,
however, in 3D the topology-preserving thinning procedures are rather com-
plex. In Paper V we propose an alternative approach to boundary thinning
that can be efficiently implemented in 3D. The method uses a binary erosion
operation and a set of constraints for deciding whether a voxel assigned a par-
tial coverage should be excluded from the boundary and be fully part of the
object/background or not. This alternative approach prevents the undesired
shrinking of thin protrusions, which was observed with the previously pro-
posed method.
60
Figure 8.7. Comparison of the high-resolution crisp reconstruction methods, (a)
ground truth segmentation, (b) result from nearest neighbor up-sampling, (c) result
from the proposed method. For the 3D case the maximum intensity projection (MIP)
is shown.
8.7 Convolutional neural network classifier (Paper VI)
In Paper VI we propose an alternative method for vascular skeleton extraction,
where a convolutional neural networks (CNN) classifier was used to distin-
guish between true and false 2D artery cross-sections. The classifier replaced
the set of knowledge-based filters used in Papers I and II in order to reduce the
large number of false positive nodes.
MethodThe method is a modified version of the algorithm presented in Paper I and
focuses on improving the node classification step. We replaced the original
set of knowledge-based filters with a CNN classifier. The other steps of the
method remain unchanged.
The classifier receives a 2D image of a potential vascular cross-section of
size 31×31 pixels from an orthogonal slice on the input and based on the
trained CNN, it classifies the image into two categories: vessel and non-
vessel. The developed CNN classifier consists of four convolutional layers,
with a max-pooling layer placed after every second convolutional layer. The
illustration of the model is provided in Figure 8.8. The first convolutional
layer contains 32 kernels of size 3×3×1 and is padded with a two-pixels thick
frame of zeros in order to keep the spatial sizes of the patches unaltered after
the first convolutional layer. The second and third convolutional layers consist
of 32 and 64 kernels, respectively, of size 3×3×32. The last convolutional
layer contains 64 kernels of size 3×3×64. The max-pooling layer reduces the
61
Figure 8.8. An overview of the CNN classifier proposed in Paper VI, showing the
output of each convolution filter applied to an example patch of a vessel. Here, the
greyscale intensities are shown in false color for suitable visualization.
size of feature maps by selecting the maximum feature response in windows
of size 2×2 and stride of 2.
ResultsWe evaluated the proposed CNN classifier qualitatively in terms of preci-
sion, recall, and F-score on the model-evaluation subset containing labeled
2D patches not used in the model development process. The precision of the
proposed CNN classifier was 0.81 for medium-sized vessels compared to the
precision of only 0.28 yielded by the method utilizing knowledge-based filters.
These numbers confirm that the CNN classifier successfully reduced the num-
ber of false positives. As a consequence, the algorithm pipeline of the skeleton
extraction method that utilizes the CNN classifier was simplified compared to
the algorithm presented in Paper I.
Qualitative evaluation was performed by visual comparison of the result-
ing skeletons extracted from 21 CTA volumes of the lower limbs comparing
the algorithm proposed in Paper I with the proposed algorithm. Figure 8.9
shows the comparison for two representative volumes. The results after the
second step confirm that the classification by CNN leads to a fewer number of
false-positives than the other method. Final results of volume A (Figure 8.9a)
illustrate that the proposed method detects more vascular branches, and com-
pared to the method based on the knowledge-based filters, does not contain
spurious graph segments in the pelvic region. On the other hand, occasion-
ally CNN classifier discarded a larger amount of true vessel candidates, which
lead to missing a complete vessel branch. Volume B (Figure 8.9b) illustrates
such situation. The missed vessels were either very small or they were very
diseased and the CNN classifier would need larger training dataset to classify
them correctly. In volumes where the vessel appearance resembles the appear-
ance of vessels in the training dataset the proposed method performed very
well.
Regarding the computation time, the CNN classifier takes ca. 30 seconds to
classify a set of 130,000 images compared to ca. 22 seconds which are needed
to process the same amount of images by the set of knowledge-based filters.
62
(a) Volume A (b) Volume B
Figure 8.9. Results after each algorithm step shown for two volumes. The set of po-
tential nodes (black) is the same for both algorithms. In the second step classifiers
remove a portion of non-vessel nodes from this set. In the third step nodes that re-
mained are connected and create skeletons. In the final step anatomy based analysis
cleans the skeletons from spurious branches.
63
9. Conclusion and future perspective
We conclude this thesis by providing a short description of how our main
objective and specific aims were fulfilled and we suggest some directions for
the future work.
9.1 Summary of contributions
In this thesis, we proposed a fast and fully-automatic vascular skeleton ex-
traction algorithm. The algorithm detects approximate vascular skeletons in
healthy (Paper I), as well as in diseased (Paper II) arteries. This algorithm,
in combination with the segmentation method presented by our collaborators
Wang et al. [79], creates a complete arterial tree segmentation framework of-
fering a segmentation with subpixel precision. In Paper III, we suggested a
new segmentation method that benefits in speed from the input skeletons and
is based on a modified onion-kernel region-growing segmentation in order to
rapidly segment anatomical tree structures resulting in a binary segmentation.
To increase the precision of this segmentation we introduced two coverage
segmentation methods that work in 2D images and 3D volumes, Papers IV
and V, respectively. These coverage segmentation methods work by turning
the binary segmentation into a fuzzy segmentation and are particularly suited
for application in thin and elongated structures, such as blood vessels. In Pa-
per VI, a convolutional neural network classifier was designed to replace a set
of simple knowledge-based filters (used in Papers I, II and III) reducing the
false-positive rate in vascular cross-sectional classification.
These contributions are well in line with the main objective of the thesis: to
develop fast and automatic methods for approximate skeleton extraction and
segmentation of tubular structures from the volume CT datasets.
Conclusion of specific aims:I The proposed algorithm for vascular skeleton extraction provides
an approximate skeleton of large CTA scans of the lower limbs
in approximately 29 seconds per scan when running on four CPU
cores. This running time is compatible with the clinical needs. In
addition, the method adapts its internal parameters automatically
for each scan based on the volume’s intensity histogram analysis.
The method was tested on 25 CTA scans and achieved an average
overlap rate of 97% and an average detection rate of 71%.
65
II The skeleton extraction algorithm is extended by two new levels
into a cascading structure, where one level improves the skeleton
extraction of strongly calcified vessels and the other level detects a
larger number of small arteries with longer branches. The method
was tested on the same CTA datasets as the previous method and
achieved an average overlap rate of 89% and an average detection
rate of 82%.
III The method for airway-tree segmentation detects a larger num-
ber of small branches, especially in thoracic images acquired at
low-pressure conditions, when compared to the reference algorithm
based on region-growing that is initiated with a single seed. Anatom-
ical landmarks extracted from each branch bifurcation provide use-
ful additional information to enable the lung segmentation in im-
ages with poor contrast when performing a hybrid registration.
IV The presented coverage segmentation method is designed to pre-
serve thin and elongated protrusions of the objects. The qualitative
evaluation confirmed the preservation of these protrusions. When
comparing the sensitivity to noise the presented method showed
better performance than all other available methods for coverage
segmentation.
V The method was successfully extended to segment thin structures in
3D images. We propose a modification that radically decreases the
execution time thus allowing it to cope with large 3D data. In the
quantitative evaluation, the accuracy of estimated coverage values
of the voxels by the proposed method was higher even in presence
of up to 35% of noise, compared to what was achieved by crisp
segmentation in a noise free-case.
VI A classifier based on CNN greatly reduces the false positives in the
vascular cross-sectional classification. The CNN classifier yielded
an f-score of 0.82 for the medium size vessels which is two times
better than what was achieved with the knowledge-based filters.
The f-score for smaller and bigger vessels was 0.65 and 0.72, re-
spectively.
9.2 Future work
Several ideas for a continuation can be drawn from our work. One idea is
to expand the algorithm for vascular skeleton detection from CTA to MRA
as this imaging modality is widely used in clinical examination and produces
volume images. Since the grey-level intensity scale in MRA is not normalized
and standardized between the machines as it is in CTA, the intensity volume
histogram analysis will be a crucial part of the algorithm. We expect that after
66
the parameters are properly selected, the conceptual idea of the rest of the
algorithm should work without major modifications.
One of the main challenges in vascular skeleton detection and segmentation
is posed by tiny and diseased arteries and their large variability in appearance
depending on the type and severity of the disease. In Paper II we proposed
a cascading algorithm, where two levels of the algorithm were specialized in
detecting two specific diseases. One way to cope with this challenge is to add
more new levels which would be specialized in other diseases like aneurysms
or thrombosis. Another way is to use machine learning algorithms. We have
already shown in Paper VI that the CNN classifier trained only on 4 CTA vol-
umes yields acceptable precision and recall in classification of middle-sized
vessels. If more training samples were provided, containing an adequate rep-
resentation of both heathy and diseased arteries of several types and sizes the
results have a potential to be largely improved. The classifier can be futher
enhanced by taking multiple patches from different axes oriented planes or
to expand to 3D and be trained on 3D clusters of the candidates. The use of
machine learning does not need to be limited to application to classification,
the detection step, can also be replaced by machine learning-based methods.
However, one challenge will be to overcome the lack of proper training data
for very small vessels, since these vessels are hard to segment manually and
it is possible that the training data contains forgotten small vessels annotated
as non-vessels. In addition, tiny vessels having only 1-2 pixels in diameter do
not provide enough information for the learning process and additional infor-
mation about spatial position within the volume or relative to other voxels that
are potential vessel candidates would need to be taken into account.
In Paper III we proposed an airway-tree segmentation algorithm suited for
cases suffering from ARDS. In diseased lungs, bronchi may contain mucus,
which consists mainly of liquid and may cause bronchi to appear disconnected
on the CT scan. Currently the algorithm detects both of the disconnected parts
of the bronchus, however, for the future it could be beneficial to include a
deeper anatomical analysis based on the validation of angles at the bifurcation
to close these gaps and provide even more anatomical landmarks. This could
potentialy remove the length limitation in the connections between the nodes.
Finally, the principle of extracting the skeleton from a structure can be ex-
tended to extraction of the structure surface. Similarly designed knowledge-
based filters can be used to detect a point on the outer surface and cluster it
with other points that have similar intensity and significant gradient creating
a mesh. This may be useful for rapid segmentation of anatomical structures,
such as lungs or colon.
67
Summary in Swedish
Kärlsjukdomar är en av de vanligaste dödsorsakerna i utvecklade länder [54],
där tidig diagnos och korrekt behandling är vitalt för att minska antalet döds-
fall. Tack vara medicinska bildupptagningstekniker kan kärlens sjukdomstill-
stånd undersökas och utvärderas utan behov av öppen kirurgi, vilket mar-
kant reducerar risken för komplikationer och samtidigt ökar patientkomfor-
ten. Dock har de teknologiska framstegen i icke-invasiv vaskulär medicinsk
avbildning lett till en ständigt ökande mängd stora och komplexa dataset som
behöver visualiseras, analyseras samt tolkas. Genomgång av dessa stora da-
tamängder är ett monotont, felkänsligt och tidsödande arbete vilket lett till ett
stort intresse för automatiserade bildbehandlingsmetoder utvecklade för att as-
sistera röntgenläkaren till en snabb och korrekt diagnos och därigenom minska
arbetsbördan.
Kärlsegmentering har en nyckelroll bland dessa bildbehandlingstekniker
då de kan bidra till att erhålla precisa mått på sjukdomstillstånd, understödja
planering av behandling samt användas för övervakning. Det kliniska använ-
dandet av metoder för kärlsegmentering står dock inför ett antal utmaningar.
Segmenteringsalgoritmerna måste i hastighet vara jämförbara med interakti-
va undersökningar av bildvolymen. Algoritmerna ska inte kräva mer än ett
minimum av interaktion och samtidigt ge en segmentering av kärl av varie-
rande storlek i det kompletta kärlträdet. Dessutom behöver algoritmen vara
tillräckligt robust för att korrekt segmentera både friska och patologiska kärl,
de senare av stort kliniskt intresse. Dessa mål uppfylls inte av någon av de
algoritmer som tidigare presenterats.
I denna avhandling fokuserar vi på att lösa ovanstående problem. Segmen-
tering av rörformade strukturer såsom blodkärl underlättas om ett approxima-
tivt kärlskelett finns att tillgå innan segmenteringen påbörjas. Vi presenterar
en snabb och helautomatisk algoritm för skapandet av ett kärlträdskelett som
ger en startpunkt för en efterföljand ytgenerering vilket accelererar och under-
lättar segmenteringsprocessen. Denna algoritm för kärlträdskelettextrahering
är designad för att hitta det kompletta kärlträdet inklusive patologiska kärl
och kärl av varierande storlek. Vi introducerar en snabb segmenteringsalgo-
ritm som utnyttjar kärlträdskelettet för att skapa en binär segmentering. Vi har
utvecklat metoder som utgående från en initial binär segmentering skapar en
mer exakt täckande segmentering med subpixelprecision. Dessutom föreslår
vi en klassificerare baserad på ett så kallat faltat neuralt nätverk (convolution
neural networks classifier, CNN) för att ytterligare förbättra algoritmen för
extrahering av kärlträdskelett.
69
De i avhandlingen nämnda metoderna designades speciellt med tanke på
kärlsegmentering, men de kan även appliceras för segmentering av andra ana-
tomiska trädstrukturer, där segmentering av bronkträd i lunga har implemen-
terats och validerats i Artikel III. De metoder för kärlsegmentering som fö-
reslagits i denna avhandling har utvärderats på angiografiska datortomografi-
volymer (Computed Tomography Angiography, CTA) av nedre extremiteter,
utvalda utmanande fall ur den kliniska rutinverksamheten. De metoder som
presenterats för segmentering av bronkträdet har validerats på griskultingar
med inducerad Acute Respiratory Distress Syndrom (ARDS).
70
Acknowledgements
Focus on the journey,
not the destination.
Joy is found not in finishing
an activity but in doing it.
Greg Anderson
Walking the stony, winding path leading to the writing of this thesis would
not have been possible –nor would I have enjoyed the journey so much– with-
out a great and supportive company. I would like to express my great appreci-
ation to those who accompanied me on this journey. My special “Thank you”
belongs to:
• Hans Frimmel, my main supervisor, for always being very supportive,
for trusting me even when I did not. For guiding me through my doctoral
studies with ease and wit. For your help when I needed it and for your
knowledge.
• Ewert Bengtsson, my assistant supervisor, for keeping your door always
open for me. For your support, helpful discussions and for sharing your
huge knowledge and experience.
• Örjan Smedby, my assistant supervisor, for providing me with your
knowledge and insight into medicine and radiology. For always find-
ing time for me in your busy schedule.
• My collaborators, Chunliang Wang, Nataša Sladoje, Joakim Lindblad,
Tomáš Majtner, Duván Gómez, Marcela Hernández Hoyos, MaciejOrkisz, Anindya Gupta, and Ida-Maria Sintorn, for fruitful discus-
sions, for your hard work, good collaboration and valuable opinions and
comments. I learnt a lot by working with you!
• Marcela Hernández Hoyos, for your hospitality and supervision in Colom-
bia. Being a part of your team was a trully valuable experience.
• Carolina Wählby, Ingela Nyström, Nataša Sladoje, Joakim Lindblad,
Maciej Orkisz, Maxime Bombrun, Christophe Avenel, Nadia AssraouiAvenel, Sajith Sadanandan Kecheril, and Anindya Gupta, for proof-
reading and commenting on selected chapters of my thesis. A special
thanks to Carlos Pérez Penichet for constructive criticism and proof-
reading the whole thesis.
71
• Lena Nordström, for always being very kind and helpful and for taking
care of everything and everybody at Vi2. Your work is much appreciated.
• Astrid Raidl, for taking care of my computer and for carefully checking
the logs on the servers :). I will not scare you with my unfamiliar IP
address anymore.
• The late Olle Ericsson, for taking care of my computer and for being a
nice office mate.
• All my office mates, Azadeh Fakhrzadeh, Omer Ishaq, Fei Liu, FredrikWahlberg, Vlada Curic, Christophe Avenel, Ingrid Carlbom, ChristerKiselman, and Sajith Sadanandan Kecheril, for nice talks and making
my time in the office very enjoyable.
• All the past and present colleagues and friends at Vi2 for creating a great
and inspiring research environment.
• All my colleagues who became my very good friends, Alexandra Pacure-anu, Maxime Bombrun, Andre Liebscher, Damian Matuszewski, Elis-abeth Linnér, Pontus Olsson, Azadeh Fakhrzadeh, Vlada Curic, AmitSuveer, Anindya Gupta, Leslie Solorzano, Gabriele Partel, and EvaBreznik.
• Leslie Solorzano and Duván Gómez, for taking care of me in Colombia.
Leslie, I hope I have been able to repay, at least partially, your kindness
and great advice. Duván, thank you for showing me around Medellin
and Lyon.
• All my Uppsala friends. Special thanks to Carine, for sharing your
passion for knitting and scouting with me. Carine and Tobias, for all the
trips and nice times we had together. Anastasija, for all the adventures
in the nature we undertook together.
• Christophe Avenel, for being such a true friend, for helping me out
during the hard times and for sharing the joy during the good times.
Christophe and Nadia Avenel, for all the unforgettable moments. You
mean so much to me.
• Šamot, for everything you did for me and for your love. You taught me
to believe in myself. Without you, I would not achieve so much and I
am very grateful.
• Carlos, for being my close friend, for your love and support. I am learn-
ing every day with you.
• My family, for supporting me and believing in me. My mother, Andrea,
for being always here for me. Mirko, because I know that my mother is
in the best hands. My brother, Adrián, for taking care of everything at
home while I am away and for being the best brother I could have ever
wished for. Miška and Maya, for your positivity and contagious smile.
72
References
[1] R. Adams and L. Bischof. Seeded region growing. IEEE Transactions onPattern Analysis and Machine Intelligence, 16(6):641–647, 1994.
[2] A. Artigas, G. R. Bernard, J. Carlet, D. Dreyfuss, L. Gattinoni, L. Hudson,
M. Lamy, J. J. Marini, M. A. Matthay, M. R. Pinsky, et al. The
American–European consensus conference on ARDS, part 2: ventilatory,
pharmacologic, supportive therapy, study design strategies, and issues related to
recovery and remodeling. American Journal of Respiratory and Critical CareMedicine, 157(4):1332–1347, 1998.
[3] Ch. Bauer and H. Bischof. A novel approach for detection of tubular objects
and its application to medical image analysis. In Joint Pattern RecognitionSymposium, pages 163–172. Springer, 2008.
[4] H. Blum. A transformation for extracting new descriptions of shape. In Modelsfor the Perception of Speech and Visual Form, pages 362–380, MIT Press,
Cambridge, MA, 1967.
[5] G. Borgefors, I. Nyström, and G. Sanniti di Baja. Computing skeletons in three
dimensions. Pattern Recognition, 32(7):1225–1236, 1999.
[6] G. Borgefors, I. Nyström, G. Sanniti di Baja, and S. Svensson. Simplification of
3D skeletons using distance information. In International Symposium onOptical Science and Technology, pages 300–309. International Society for
Optics and Photonics, 2000.
[7] T. Boskamp, D. Rinck, F. Link, B. Kümmerlen, G. Stamm, and P. Mildenberger.
New vessel analysis tool for morphometric quantification and visualization of
vessels in CT and MR imaging data sets. Radiographics, 24(1):287–297, 2004.
[8] J. T. Bushberg and J. M. Boone. The Essential Physics of Medical Imaging.
Lippincott Williams & Wilkins, 2011.
[9] V. Caselles, F. Catté, T. Coll, and F. Dibos. A geometric model for active
contours in image processing. Numerische Mathematik, 66(1):1–31, 1993.
[10] N. D. Cornea, D. Silver, and P. Min. Curve-skeleton properties, applications,
and algorithms. IEEE Transactions on Visualization & Computer Graphics,
(3):530–548, 2007.
[11] L. Deng and D. Yu. Deep learning: Methods and applications. Foundations andTrends in Signal Processing, 7(3–4):197–387, 2014.
[12] S. Eiho, H. Sekiguchi, N. Sugimoto, T. Hanakawa, and S. Urayama.
Branch-based region growing method for blood vessel segmentation. In
Proceedings of International Society for Photogrammetry and Remote SensingCongress, pages 796–801, 2004.
[13] T. T. Elvins. A survey of algorithms for volume visualization. ACMSIGGRAPH Computer Graphics, 26(3):194–201, 1992.
[14] V. Fanelli and V. M. Ranieri. Mechanisms and clinical consequences of acute
lung injury. Annals of the American Thoracic Society, 12(Supplement
1):S3–S8, 2015.
73
[15] ARDS Definition Task Force. Acute respiratory distress syndrome. Jama,
307(23):2526–2533, 2012.
[16] A. F. Frangi, W. J. Niessen, R. M. Hoogeveen, T. van Walsum, and M. A.
Viergever. Model-based quantitation of 3-D magnetic resonance angiographic
images. IEEE Transactions on Medical Imaging, 18(10):946–956, 1999.
[17] A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale
vessel enhancement filtering. In Medical Image Computing andComputer-Assisted Intervention–MICCAI’98, pages 130–137. Springer, 1998.
[18] H. Frimmel, J. Näppi, and H. Yoshida. Centerline-based colon segmentation for
CT colonography. Medical Physics, 32(8):2665–2672, 2005.
[19] E. Galkina and K. Ley. Immune and inflammatory mechanisms of
atherosclerosis. Annual review of immunology, 27:165–197, 2009.
[20] D. Geiger, A. Gupta, L. A. Costa, and J. Vlontzos. Dynamic programming for
detecting, tracking, and matching deformable contours. IEEE Transactions onPattern Analysis and Machine Intelligence, 17(3):294–302, 1995.
[21] R. W. Gill. Pulsed doppler with B-mode imaging for quantitative blood flow
measurement. Ultrasound in Medicine & Biology, 5(3):223–235, 1979.
[22] O. Glasser. Wilhelm Conrad Röntgen and the early history of the Roentgen rays.
Number 1. Norman Publishing, 1993.
[23] R. Gonzalez and R. Woods. Digital image processing. Pearson Prentice Hall,
2008.
[24] I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. Book in preparation
for MIT Press, 2016.
[25] H. Gray. Anatomy of the human body. Lea & Febiger, 1918.
[26] M. A. Gülsün and H. Tek. Robust vessel tree modeling. In Medical ImageComputing and Computer-Assisted Intervention–MICCAI 2008, pages
602–611. Springer, 2008.
[27] J. C. P. Heggie, N. A. Liddell, and K. P. Maher. Applied imaging technology. St.
Vincent’s Hospital Melbourne, 4th ed. edition, 2001.
[28] S. J. Hessel, D. F. Adams, and H. L. Abrams. Complications of angiography.
Radiology, 138(2):273–281, 1981.
[29] W. R. Hiatt. Medical treatment of peripheral arterial disease and claudication.
The New England Journal of Medicine, 344(21):1608–21, 2001.
[30] E. A. Hoffman. Effect of body orientation on regional lung expansion: a
computed tomographic approach. Journal of Applied Physiology,
59(2):468–480, 1985.
[31] G. N. Hounsfield. Computerized transverse axial scanning (tomography): Part
1. Description of system. The British Journal of Radiology,
46(552):1016–1022, 1973.
[32] D. H. Hubel and T. N. Wiesel. Receptive fields of single neurones in the cat’s
striate cortex. The Journal of physiology, 148(3):574–591, 1959.
[33] D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and
functional architecture in the cat’s visual cortex. The Journal of physiology,
160(1):106–154, 1962.
[34] O. Ishaq. Image Analysis and Deep Learning for Applications in Microscopy.
PhD thesis, Acta Universitatis Upsaliensis, 2016.
[35] C. Kirbas and F. K. H. Quek. A review of vessel extraction techniques and
74
algorithms. ACM Computing Surveys, 36(2):81–121, 2004.
[36] P. A. Krieger. A Visual Analogy Guide to Human Anatomy and Physiology.
Morton Publishing Company, 2013.
[37] Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature,
521(7553):436–444, 2015.
[38] D. Lesage, E. D. Angelini, I. Bloch, and G. Funka-Lea. A review of 3D vessel
lumen segmentation techniques: Models, features and extraction schemes.
Medical Image Analysis, 13(6):819–845, 2009.
[39] F.-F. Li, A. Karpathy, and J. Johnson. Notes to Stanford CS class CS231n:
Convolutional Neural Networks for Visual Recognition, 2017.
[40] R. Li and S. Ourselin. Accurate curvilinear modelling for precise measurements
of tubular structures. In Digital Image Computing: Techniques andApplications; Proceedings of the 7th Biennial Australian Pattern RecognitionSociety Conference, DICTA 2003, page 243, 2003.
[41] J. Lindblad and N. Sladoje. Coverage segmentation based on linear unmixing
and minimization of perimeter and boundary thickness. Pattern RecognitionLetters, 33(6):728 – 738, 2012.
[42] T. Majtner, K. Lidayová, S. Yildirim-Yayilgan, and J. Y. Hardeberg. Improving
skin lesion segmentation in dermoscopic images by thin artefacts removal
methods. In Visual Information Processing (EUVIP), 2016 6th EuropeanWorkshop on, pages 1–6. IEEE, 2016.
[43] R. Malladi, J. A. Sethian, and B. C. Vemuri. Topology-independent shape
modeling scheme. In SPIE’s 1993 International Symposium on Optics,Imaging, and Instrumentation, pages 246–258. International Society for Optics
and Photonics, 1993.
[44] S. S. Mao, N. Ahmadi, B. Shah, D. Beckmann, A. Chen, L. Ngo, F. R. Flores,
Y. L. Gao, and M. J. Budoff. Normal thoracic aorta diameter on cardiac
computed tomography in healthy asymptomatic adults: Impact of age and
gender. Academic Radiology, 15(7):827–834, 2008.
[45] C. T. Metz, M. Schaap, A. C. Weustink, N. R. Mollet, T. van Walsum, and W. J.
Niessen. Coronary centerline extraction from CT coronary angiography images
using a minimum cost path approach. Medical Physics, 36(12):5568–5579,
2009.
[46] J. Mille and L. D. Cohen. Deformable tree models for 2D and 3D branching
structures extraction. In 2009 IEEE Computer Society Conference on ComputerVision and Pattern Recognition Workshops, pages 149–156, 2009.
[47] M. Miyazaki and V. S. Lee. Nonenhanced MR angiography. Radiology,
248(1):20–43, 2008.
[48] A. Morales Pinzón. Lung Segmentation and Airway Tree Matching: Applicationto Aeration Quantification in CT Images of Subjects with ARDS. PhD thesis,
Université de Lyon, 2016.
[49] K. Mori, J. Hasegawa, J. Toriwaki, H. Anno, and K. Katada. Recognition of
bronchus in three-dimensional X-ray CT images with applications to virtualized
bronchoscopy system. In Proceedings of the 13th International Conference onPattern Recognition, volume 3, pages 528–532. IEEE, 1996.
[50] G. Németh, P. Kardos, and K. Palágyi. Thinning combined with
iteration-by-iteration smoothing for 3D binary images. Graphical Models,
75
73(6):335–345, 2011.
[51] M. A. Nielsen. Neural networks and deep learning. URL:http://neuralnetworksanddeeplearning. com/.(visited: 30.03. 2017), 2015.
[52] I. Nyström. On Quantitative Shape Analysis of Digital Volume Images. PhD
thesis, Acta Universitatis Upsaliensis, 1997.
[53] J. W. Olin and B. A. Sealove. Peripheral artery disease: current insight into the
disease and its diagnosis and management. Mayo Clinic Proceedings,
85(7):678–92, 2010.
[54] World Health Organization. World Health Statistics 2016: Monitoring Healthfor the Sustainable Development Goals (SDGs). World Health Organization,
2016.
[55] S. Osher and J. A. Sethian. Fronts propagating with curvature-dependent speed:
algorithms based on Hamilton-Jacobi formulations. Journal of ComputationalPhysics, 79(1):12–49, 1988.
[56] K. Palágyi and A. Kuba. A parallel 3D 12-subiteration thinning algorithm.
Graphical Models and Image Processing, 61(4):199–221, 1999.
[57] P. Pignoli, E. Tremoli, A. Poli, P. Oreste, and R. Paoletti. Intimal plus medial
thickness of the arterial wall: a direct measurement with ultrasound imaging.
Circulation, 74(6):1399–406, 1986.
[58] B. Preim and D. Bartz. Visualization in medicine: theory, algorithms, andapplications. Morgan Kaufmann, 2007.
[59] B. Preim and Ch. P. Botha. Visual Computing for Medicine: Theory,Algorithms, and Applications. Newnes, 2013.
[60] M. R. Prince, E. K. Yucel, J. A. Kaufman, D. C. Harrison, and S. C. Geller.
Dynamic gadolinium-enhanced three-dimensional abdominal MR
arteriography. Journal of Magnetic Resonance Imaging, 3(6):877–881, 1993.
[61] J.-Ch. Richard, C. Pouzot, A. Morales Pinzón, J. S. T. González, M. Orkisz,
B. Neyran, M. Hernández Hoyos, F. Lavenne, and C. Guerin. Reliability of the
nitrogen washin-washout technique to assess end-expiratory lung volume at
variable PEEP and tidal volumes. Intensive care medicine experimental,2(1):1–14, 2014.
[62] G. D. Rubin, P. J. Walker, M. D. Dake, S. Napel, R. B. Jeffrey, C. H. McDonnell,
R. S. Mitchell, and D. C. Miller. Three-dimensional spiral computed
tomographic angiography: an alternative imaging modality for the abdominal
aorta and its branches. Journal of Vascular Surgery, 18(4):656–64, 1993.
[63] P. K. Saha, G. Borgefors, and G. Sanniti di Baja. A survey on skeletonization
algorithms and their applications. Pattern Recognition Letters, 76:3–12, 2016.
[64] P. K. Saha, B. B. Chaudhuri, and D. D. Majumder. A new shape preserving
parallel thinning algorithm for 3D digital images. Pattern Recognition,
30(12):1939–1955, 1997.
[65] P. K. Sahoo, S. Soltani, and A. K. C. Wong. A survey of thresholding
techniques. Computer Vision, Graphics, and Image Processing, 41(2):233–260,
1988.
[66] G. Sapiro and A. Tannenbaum. Affine invariant scale-space. InternationalJournal of Computer Vision, 11(1):25–44, 1993.
[67] M. Schaap, L. Neefjes, C. Metz, A. van der Giessen, A. Weustink, N. Mollet,
J. Wentzel, T. van Walsum, and W. Niessen. Coronary lumen segmentation
76
using graph cuts and robust kernel regression. In International Conference onInformation Processing in Medical Imaging, pages 528–539. Springer, 2009.
[68] J. Schmidhuber. Deep learning in neural networks: An overview. NeuralNetworks, 61:85–117, 2015.
[69] K. Siddiqi and S. Pizer. Medial representations: mathematics, algorithms andapplications, volume 37. Springer Science & Business Media, 2008.
[70] N. Sladoje and J. Lindblad. High-precision boundary length estimation by
utilizing gray-level information. IEEE Transactions on Pattern Analysis andMachine Intelligence, 31(2):357–363, 2009.
[71] N. Sladoje and J. Lindblad. Pixel coverage segmentation for improved feature
estimation. In Proceedings of the 15th International Conference on ImageAnalysis and Processing, ICIAP ’09, pages 929–938. Springer-Verlag, 2009.
[72] N. Sladoje and J. Lindblad. The coverage model and its use in image
processing. In Selected Topics on Image Processing and Cryptology, ZbornikRadova (Collection of Papers), volume 15, pages 39–117. Mathematical
Institute of the Serbian Academy of Sciences and Arts, Belgrade, 2012.
[73] M. Sonka, V. Hlavac, and R. Boyle. Image processing, analysis, and machinevision. Cengage Learning, 2014.
[74] H. C. Stary. Natural history and histological classification of atherosclerotic
lesions: an update. Arteriosclerosis, Thrombosis, and Vascular Biology,
20(5):1177–8, 2000.
[75] J. S. Suri, K. Liu, L. Reden, and S. Laxminarayan. A review on MR vascular
image processing: skeleton versus nonskeleton approaches: Part II. IEEETransactions on Information Technology in Biomedicine: a publication of theIEEE Engineering in Medicine and Biology Society, 6(4):338–350, 2002.
[76] G. J. Tortora and B. Derrickson. Introduction to the human body: the essentialsof anatomy and physiology. Wiley, 2011.
[77] C. M. van Bemmel, O. Wink, B. Verdonck, M. A. Viergever, and W. J. Niessen.
Blood pool contrast-enhanced MRA: improved arterial visualization in the
steady state. IEEE Transactions on Medical Imaging, 22(5):645–652, 2003.
[78] W. J. van Rooij, M. E. Sprengers, A. N. de Gast, J. P. Peluso, and M. Sluzewski.
3D rotational angiography: the new gold standard in the detection of additional
intracranial aneurysms. American Journal of Neuroradiology, 29(5):976–9,
2008.
[79] Ch. Wang, R. Moreno, and Ö. Smedby. Vessel segmentation using implicit
model-guided level sets. In Proceedings of MICCAI Workshop 3DCardiovascular Imaging: a MICCAI segmentation Challenge – Medical ImageComputing and Computer-Assisted Intervention. 2012.
[80] Ch. Wang and Ö. Smedby. Integrating automatic and interactive methods for
coronary artery segmentation: let the PACS workstation think ahead.
International Journal of Computer Assisted Radiology and Surgery,
5(3):275–85, 2010.
[81] Ch. Xu and J. L. Prince. Snakes, shapes, and gradient vector flow. IEEETransactions on Image Processing, 7(3):359–369, 1998.
[82] L. A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, 1965.
77
Errata
Paper IV
Coverage Segmentation of Thin Structures by Linear Unmixing and LocalCentre of Gravity Attraction
II. Background• Original text “where Sk ⊂ R
2 is” with character S in calligraphic
font should be replaced by “where Sk ⊂ R2 is” with character S in
ordinary font.
• Original text “The continuous sets Sk are” with character S in cal-
ligraphic font should be replaced “The continuous sets Sk are” with
character S in ordinary font.
Paper V
Coverage Segmentation of 3D Thin Structures
II. Background• Original text “where Sk ⊂ R
3 is” with character S in calligraphic
font should be replaced by “where Sk ⊂ R3 is” with character S in
ordinary font.
• Original text The continuous sets Sk are” with character S in cal-
ligraphic font should be replaced by “The continuous sets Sk are”
with character S in ordinary font.
79
Acta Universitatis UpsaliensisDigital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1496
Editor: The Dean of the Faculty of Science and Technology
A doctoral dissertation from the Faculty of Science andTechnology, Uppsala University, is usually a summary of anumber of papers. A few copies of the complete dissertationare kept at major Swedish research libraries, while thesummary alone is distributed internationally throughthe series Digital Comprehensive Summaries of UppsalaDissertations from the Faculty of Science and Technology.(Prior to January, 2005, the series was published under thetitle “Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology”.)
Distribution: publications.uu.seurn:nbn:se:uu:diva-318796
ACTAUNIVERSITATIS
UPSALIENSISUPPSALA
2017