fast methods for vascular segmentation based on ...1085403/fulltext01.pdf · "skeleton-based...

80
ACTA UNIVERSITATIS UPSALIENSIS UPPSALA 2017 Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1496 Fast Methods for Vascular Segmentation Based on Approximate Skeleton Detection KRISTÍNA LIDAYOVÁ ISSN 1651-6214 ISBN 978-91-554-9874-0 urn:nbn:se:uu:diva-318796

Upload: others

Post on 21-May-2020

11 views

Category:

Documents


0 download

TRANSCRIPT

ACTAUNIVERSITATIS

UPSALIENSISUPPSALA

2017

Digital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1496

Fast Methods for VascularSegmentation Based onApproximate Skeleton Detection

KRISTÍNA LIDAYOVÁ

ISSN 1651-6214ISBN 978-91-554-9874-0urn:nbn:se:uu:diva-318796

Dissertation presented at Uppsala University to be publicly examined in ITC 2446,Lägerhyddsvägen 2, Uppsala, Monday, 22 May 2017 at 10:15 for the degree of Doctor ofPhilosophy. The examination will be conducted in English. Faculty examiner: ProfessorAlejandro F. Frangi (The University of Sheffield).

AbstractLidayová, K. 2017. Fast Methods for Vascular Segmentation Based on ApproximateSkeleton Detection. Digital Comprehensive Summaries of Uppsala Dissertations from theFaculty of Science and Technology 1496. 79 pp. Uppsala: Acta Universitatis Upsaliensis.ISBN 978-91-554-9874-0.

Modern medical imaging techniques have revolutionized health care over the last decades,providing clinicians with high-resolution 3D images of the inside of the patient's body withoutthe need for invasive procedures. Detailed images of the vascular anatomy can be capturedby angiography, providing a valuable source of information when deciding whether a vascularintervention is needed, for planning treatment, and for analyzing the success of therapy.However, increasing level of detail in the images, together with a wide availability of imagingdevices, lead to an urgent need for automated techniques for image segmentation and analysisin order to assist the clinicians in performing a fast and accurate examination.

To reduce the need for user interaction and increase the speed of vascular segmentation, wepropose a fast and fully automatic vascular skeleton extraction algorithm. This algorithm firstanalyzes the volume's intensity histogram in order to automatically adapt the internal parametersto each patient and then it produces an approximate skeleton of the patient's vasculature.The skeleton can serve as a seed region for subsequent surface extraction algorithms. Furtherimprovements of the skeleton extraction algorithm include the expansion to detect the skeletonof diseased arteries and the design of a convolutional neural network classifier that reduces falsepositive detections of vascular cross-sections. In addition to the complete skeleton extractionalgorithm, the thesis presents a segmentation algorithm based on modified onion-kernel regiongrowing. It initiates the growing from the previously extracted skeleton and provides a rapidbinary segmentation of tubular structures. To provide the possibility of extracting precisemeasurements from this segmentation we introduce a method for obtaining a segmentationwith subpixel precision out of the binary segmentation and the original image. This method isespecially suited for thin and elongated structures, such as vessels, since it does not shrink thelong protrusions. The method supports both 2D and 3D image data.

The methods were validated on real computed tomography datasets and are primarily intendedfor applications in vascular segmentation, however, they are robust enough to work with otheranatomical tree structures after adequate parameter adjustment, which was demonstrated on anairway-tree segmentation.

Keywords: medical image analysis, automatic skeleton extraction, vascular segmentation,coverage segmentation, convolutional neural network classifier, CT angiography

Kristína Lidayová, Department of Information Technology, Division of Visual Informationand Interaction, Box 337, Uppsala University, SE-751 05 Uppsala, Sweden. Department ofInformation Technology, Computerized Image Analysis and Human-Computer Interaction,Box 337, Uppsala University, SE-75105 Uppsala, Sweden.

© Kristína Lidayová 2017

ISSN 1651-6214ISBN 978-91-554-9874-0urn:nbn:se:uu:diva-318796 (http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-318796)

To my dear family

List of papers

This thesis is based on the following papers, which are referred to in the text

by their Roman numerals.

I K. Lidayová, H. Frimmel, C. Wang, E. Bengtsson and Ö. Smedby.

"Fast Vascular Skeleton Extraction Algorithm", Pattern RecognitionLetters, Vol. 76, pp. 67-75, 2016.

Lidayová developed the method, designed and performed the

experiments and wrote the paper

II K. Lidayová, H. Frimmel, E. Bengtsson and Ö. Smedby. "Improved

Centerline Tree Detection of Diseased Peripheral Arteries with a

Cascading Algorithm for Vascular Segmentation", Journal of MedicalImaging, accepted for publication, 2017

Lidayová developed the method, designed and performed the

experiments and wrote the paper

III K. Lidayová, D. A. Gómez Betancur, H. Frimmel, M. Hernández

Hoyos, M. Orkisz and Ö. Smedby. "Airway-Tree Segmentation in

Subjects with Acute Respiratory Distress Syndrome", in ScandinavianConference on Image Analysis (SCIA), accepted for publication, 2017.

Lidayová developed the method and wrote a major part of the paper

IV K. Lidayová, J. Lindblad, N. Sladoje and H. Frimmel. "Coverage

Segmentation of Thin Structures by Linear Unmixing and Local Centre

of Gravity Attraction", in 8th International Symposium on Image andSignal Processing and Analysis (ISPA), pp. 83-88, 2013.

Lidayová developed the method, designed and performed the

experiments and wrote the paper

V K. Lidayová, J. Lindblad, N. Sladoje, H. Frimmel, C. Wang and Ö.

Smedby. "Coverage Segmentation of 3D Thin Structures", in

International Conference on Image Processing Theory, Tools andApplications (IPTA), pp. 23-28, 2015.

Lidayová developed the method, designed and performed the

experiments and wrote the paper

VI K. Lidayová, A. Gupta, H. Frimmel, I.-M. Sintorn, E. Bengtsson and

Ö. Smedby. "Classification of Cross-sections for Vascular Skeleton

Extraction Using Convolutional Neural Networks", submitted for

conference publication, 2017.

Lidayová and Gupta contributed equally to the method development,

experiments design and performance and the paper writing

Reprints were made with permission from the publishers.

Related work

In addition to the papers included in this thesis, the author has also written or

contributed to the following publications:

1 K. Lidayová, H. Frimmel, Ch. Wang, E. Bengtsson, Ö Smedby.

"Skeleton-based fast, fully automated generation of vessel tree

structure for clinical evaluation of blood vessel systems",

Skeletonization and its Application, Academic Press and Newnes,

Elsevier, Oxford, pp. 345-382, book chapter in print

2 K. Lidayová, H. Frimmel, E. Bengtsson, Ö Smedby. "Fast Vessel

Centerline Tree Extraction Algorithm", in Proceedings of SSBA 2014,

Swedish Society for Automated Image Analysis, pp. 86-90, 2014

3 T. Majtner, K. Lidayová, S. Yildirim-Yayilgan, J. Y. Hardeberg.

"Improving Skin Lesion Segmentation in Dermoscopic Images by Thin

Artefacts Removal Methods", in 6th European Workshop on VisualInformation Processing (EUVIP), pp. 1-6, 2016

Contents

Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.1 Objectives of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.2 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Medical digital imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1 Early medical imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2 Computed tomography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3 Digitization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Medical context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1 Anatomy of blood vessels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.1 Peripheral Artery Disease . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.2 Diagnosing PAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.1.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.1.4 CTA dataset of the lower limbs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2 Anatomy of the lungs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2.1 Acute Respiratory Distress Syndrome . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2.2 Treatment of ARDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2.4 Dataset of thoracic CT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4 Vascular skeletons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.1 Definition of a skeleton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.2 Digital skeletons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3 Our approximate skeleton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.3.1 Description of the approximate skeleton . . . . . . . . . . . . . . . . . . . . . . . 29

4.3.2 Evaluation of the approximate skeleton . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Vascular segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.1 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.2 Preprocessing: Vesselness filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.3 Segmentation methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.3.1 Thresholding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.3.2 Region-growing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.3.3 Centerline-based methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.3.4 Geometric deformable model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.3.5 Skeleton guided level set based vessel segmentation . . . . 40

6 Coverage segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6.1 Fuzzy set theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6.2 Coverage representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6.3 Coverage segmentation methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

6.3.1 Coverage segmentation based on double thresholding . 43

6.3.2 Coverage segmentation by local unmixing . . . . . . . . . . . . . . . . . . . . 43

6.3.3 Energy based coverage segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

6.4 Coverage segmentation applied to thin structures . . . . . . . . . . . . . . . . . . . . . . . 45

7 Deep neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

7.1 Deep learning in neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

7.2 Convolutional neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

7.2.1 Convolutional layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

7.2.2 Pooling layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

7.2.3 Rectified linear unit (ReLU) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

7.2.4 Batch normalization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

7.2.5 Fully-connected layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

7.3 Our CNN classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

8 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

8.1 Fast vascular skeleton extraction algorithm (Papers I and II) . . . . 51

8.2 Automatic adaptation to intensity variation between scans

(Paper I) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

8.3 Skeleton-guided onion-kernel based segmentation (Paper III) . . . 55

8.4 Coverage segmentation of thin structures (Papers IV and V) . . . . . 58

8.5 High resolution crisp reconstruction (Papers IV and V) . . . . . . . . . . . . . 60

8.6 Boundary thinning in 3D (Paper V) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

8.7 Convolutional neural network classifier (Paper VI) . . . . . . . . . . . . . . . . . . . . . 61

9 Conclusion and future perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

9.1 Summary of contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

9.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Summary in Swedish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Errata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Abbreviations

2D Two-Dimensional

3D Three-Dimensional

ABI Ankle-Brachial Index

ARDS Acute Respiratory Distress Syndrome

CPR Curved Planar Reformation

CT Computed Tomography

CTA Computed Tomography Angiography

DVR Direct Volume Rendering

HU Hounsfield Units

MIP Maximum Intensity Projection

PACS Picture Archiving and Communication System

PAD Peripheral Artery Disease

PVE Partial Volume Effect

11

1. Introduction

Vascular diseases are among the leading causes of death in developed coun-

tries [54]. An early diagnosis and the right treatment are crucial to reduce

the number of fatal cases. Thanks to medical imaging techniques the state of

diseased vessels can be examined and accurately evaluated, without the need

for open surgery. This greatly reduces the risk of complications and improves

patient comfort. However, technological advances in non-invasive vascular

medical imaging have lead to ever-increasing amounts of large and complex

datasets that need to be visualized, analyzed, and interpreted. Processing of

such a large amount of data is a monotonous, error-prone and time-consuming

task. Hence, there is a large interest in developing automated image process-

ing techniques to reduce the workload and assist the radiologist in performing

a quick and accurate diagnosis.

Vascular segmentation plays a distinctive role among these image process-

ing techniques since it can aid in obtaining precise measurements for disease

severity assessment, therapy planning and monitoring. However, there are nu-

merous challenges related to utilizing vascular segmentation methods in clin-

ical routine. Segmentation algorithms need to be fast enough to run at speeds

compatible with interactive exploration of the volume. The algorithms should

require only minimal human interaction and support segmentation of vessels

of different sizes over the complete vascular tree. In addition, these algo-

rithms should be robust enough to accurately segment healthy vessels as well

as pathological vessels, which are of the greatest clinical interest. These goals

are currently not met by existing algorithms.

In this thesis, we focus on achieving these goals. The process of segmenting

tubular structures, such as vessels, can benefit from detecting an approximate

vascular skeleton prior to segmentation. We propose a fast and fully auto-

matic algorithm for vascular skeleton detection that provides an initial base

for subsequent surface extraction and thereby accelerates and facilitates the

segmentation process. The skeleton extraction algorithm is designed to detect

the complete vascular tree, including pathological vessels and vessels of dif-

ferent sizes. We also introduce a fast segmentation algorithm which leverages

this skeleton and produces a binary segmentation. We developed methods that,

provided with an initial binary segmentation, create a more accurate coverage

segmentation with subpixel precision. In addition, we propose a convolutional

neural network (CNN) classifier to further improve the skeleton extraction al-

gorithm.

These methods were designed having the main application of vascular seg-

mentation in mind, however, they can be successfully applied for segmenting

13

other anatomical tree structures, such as airway trees. The methods proposed

in this thesis were evaluated on challenging Computed Tomography Angiog-

raphy (CTA) datasets of the lower limbs taken from the clinical routine and on

thoracic Computed Tomography (CT) volumes of piglets with induced acute

respiratory distress syndrome (ARDS).

1.1 Objectives of the thesisThe main objective of this thesis was to develop fast and automatic meth-

ods for approximate skeleton extraction and precise segmentation of tubular

structures from volume CT datasets. The work is divided into two main appli-

cations: detection of arteries in CTA datasets of the lower limbs and an airway

tree segmentation from subjects with ARDS. My thesis studies concentrated

on several aspects of the main objective. Each aspect was addressed by one

publication. These aspects can be detailed as:

Specific aims:I To develop a fast and fully automatic method for vascular skeleton

extraction.

II To extend and improve the vascular skeleton extraction method for

pathological arteries with focus on calcifications and collateral ar-

teries.

III To suggest a method for airway tree segmentation with the main

objective of obtaining a large number of branches that can serve as

anatomical landmarks needed for further image processing.

IV To propose a method for coverage segmentation of thin 2D struc-

tures that will not shrink the long protrusions.

V To extend the coverage segmentation method of thin structures to

support segmentation of 3D elongated structures from volume im-

ages.

VI To develop a classifier based on CNN to reduce the false positives in

the vascular cross-sectional classification and to compare the clas-

sifier with a previously utilized set of knowledge-based filters.

1.2 Thesis outlineThe thesis is structured as follows: Chapter 2 gives an introduction to medical

digital imaging and to the digitization process. Chapter 3 provides a medi-

cal background for two diseases: peripheral artery disease (PAD) and ARDS.

Chapters 4 and 5 introduce methods for vascular skeleton extraction and vas-

cular segmentation, respectively. Chapter 6 focuses on coverage segmentation

techniques and Chapter 7 deals with deep neural networks. The main con-

tributions of the thesis are presented and summarized in Chapter 8. Finally,

Chapter 9 concludes the thesis and discusses possible future perspectives.

14

2. Medical digital imaging

This chapter provides a brief overview of the main milestones in radiographic

medical imaging, from analogue to digital techniques. We explain how digi-

tal volume images are acquired, how they are represented in a computer and

which imaging artifacts are common in CT.

2.1 Early medical imaging

Medical imaging began in 1895 after the German physicist Wilhelm Rönt-

gen discovered X-rays [22]. Röntgen quickly realized that this new radiation,

invisible to the human eye, could penetrate different materials and could be

recorded on photographic plates. He demonstrated it with the first X-ray image

of the human anatomy - the hand possibly belonging to his wife (Figure 2.1).

This important discovery laid the foundations of radiography.

Early radiography captured radiographic images using an X-ray source on

one side of the patient and an X-ray detector, usually a photographic film, on

the other side. The X-rays emitted from the source were transmitted through

the patient’s body where a fraction of the X-rays was absorbed by various

tissues. The attenuated X-rays were then recorded by the X-ray detector [8].

Figure 2.1. A radiograph of the hand taken by Röntgen in December 1895. Used with

permission of the Deutsches Röntgen Museum, Remscheid, Germany.

15

Figure 2.2. Orthogonal cross-sectional slices (axial, sagittal and coronal) of a 3D CT

volume representing the pelvic region. The image intensity corresponds to the tissue

type, dense tissue (e.g. bones) absorb more X-rays and appear as bright regions on a

CT scan compared to sparse tissue (e.g. soft tissue, air) which appear as gray or dark

regions.

2.2 Computed tomography

Computers were introduced in medical imaging in the early 1970s, when the

English engineer Godfrey Hounsfield invented X-ray Computed Tomography

(CT) [31]. Compared to the traditional radiography images, CT images are

produced by passing X-rays through the patient body at a large number of an-

gles. The X-ray source and the detector rotate around the patient and collect

multiple projection data. The data are processed by applying mathematical

reconstruction algorithms and synthesized by a computer into a set of cross-

sectional tomographic images. These images are stacked on top of each other

to create a 3D volume image of the anatomy of interest [8]. From the 3D

volume different orientations of 2D cross-sectional images (slices) can be vi-

sualized; three principal orientations given the position of a human are: axial,

sagittal and coronal (Figure 2.2).

2.3 Digitization

Early radiography images were captured on a photographic film and thereby

allowed a continuous representation of the human anatomy. The captured in-

formation represented a physical quantity, i.e., the darkness of the picture was

proportional to the number of X-rays that hit the film layer.

In X-ray CT the analogue information from the detectors is, from inception,

converted into digital form to process, store and transfer images. The process

of converting the analogue information into the digital form is digitization and

generally consists of two concurrent processes - sampling and quantization

[27]. Sampling limits the spatial resolution by dividing the continuous image

into an array of image elements. Quantization restricts brightness resolution

to a fixed set of discrete values for each image element. Figures 2.3 and 2.4

illustrate sampling and quantization, respectively.

16

(a) (b) (c)Figure 2.3. A coronal CT slice from Figure 2.2 displayed with different spatial resolu-

tions. The larger the number of pixels, the closer the spatial resolution of the digitized

image approximates the resolution of the original object. (a) 64 × 64 pixels, (b) 32 ×32 pixels, (c) 16 × 16 pixels. Anatomical structures clearly visible in image (a) are

due to partial volume effect blurred in image (c).

(a) (b) (c)Figure 2.4. A coronal CT slice from Figure 2.2 displayed at different image quantiza-

tion levels. The more levels of gray are used, the closer the brightness information in

the digitized image approximates the original brightness. (a) 16 levels of grayscale (4

bits), (b) 8 levels of grayscale (3 bits), (c) 4 levels of grayscale (2 bits).

The image element is called a pixel in the case of a two-dimensional image

and a voxel in the case of a three-dimensional volume image. A pixel size

is usually isotropic, whereas, in medical volume images it is common to have

anisotropic voxel size. Each voxel can be viewed as cube or rectangular prism,

having 6 faces, 12 edges and 8 corners. A set of voxels that share a common

face with a voxel p is known as the 6-neighborhood of p denoted by N6(p).Similarly, N18(p) and N26(p) are 18- and 26-neighborhoods of p and have

a common edge and corner with p, respectively. If another voxel q is m-

connected to p it means that q ∈ Nm(p).The brightness of the voxel is called intensity. In the case of CT images

the intensity of a voxel is related to the attenuation of X-ray radiation in the

tissue it represents. X-ray attenuation coefficients are normalized to water

17

Table 2.1. Interval values of Hounsfield units for selected tissue types.

Tissue Type HU Value Interval

Air -1000

Lung tissue -900 to -170

Fat tissue -220 to -30

Water 0

Muscle 10 to 40

Blood 40

Trabecular bone 300 to 500

Cortical bone 600 to 3000

as a reference material and are referred to as Hounsfield units (HU) or CT-

numbers [27]. In the Hounsfield scale, water represents a value of 0 HU and

air represents a value of -1000 HU. Muscle and blood are both around +40 HU.

In order to differentiate between blood vessels and surrounding muscle tissue,

a contrast agent needs to be injected into the bloodstream before the scan.

By carefully timing the acquisition, a CT scan can be obtained in the arterial

phase, when the contrast medium has not yet reached veins therefore only

arteries are visible, or in the venous phase, when both arteries and veins are

filled with the contrast agent. In this thesis, we work with CTA datasets that

were acquired in the arterial phase. Typical Hounsfield units of other tissue

types are presented in Table 2.1 [27, 59] but might differ slightly between

different CT machines.

Digitization leads, due to the limited spatial resolution, to an imaging ar-

tifact called partial volume effect (PVE). This artifact occurs when a voxel

contains more than one tissue type. The resulting HU value of such a voxel

is calculated as a weighted average of the HU values of individual compo-

nents. As a consequence, the intensities across tissue boundaries are blurred

and the visualization of small structures is limited. Other artifacts that are

often present in CT volume images are noise, motion blur caused by patient

movement during the scanning process, and beam hardening artifacts (a rela-

tive increase in X-ray energy after passing through a dense object (e.g. metallic

implant)).

In this thesis, we work mostly with CT angiography volume images depict-

ing the lower limbs and having the average size of 512×512×1711 voxels and

a bit depth of 16 bits (65 536 levels of gray). The voxel size is anisotropic, on

average having the length of 0.78×0.78×0.7 mm. This implies that the small-

est vessel that we can clearly view in our data should be at least 1.4 mm in

diameter. Smaller vessels (0.7 - 1.4 mm in diameter) can possibly be detected

if they are well aligned with the voxel grid and they cover one complete voxel.

Vessels smaller than 0.7 mm in diameter will be blurred and their intensity

averaged with the neighboring tissue intensity due to PVE.

18

3. Medical context

This chapter introduces the medical background needed to understand the

severity of the diseases which represent the main interest of this thesis, i.e.,

peripheral artery diseases and acute respiratory distress syndrome.

3.1 Anatomy of blood vessels

The blood vessels are part of the cardiovascular system. Together with the

heart they distribute the blood throughout the body, deliver materials such

as oxygen, nutrients, and hormones to all cells and carry away the wastes.

They contribute to homeostasis, stabilize temperature and pH, and fight dis-

eases [76].

There are five main types of blood vessels in the human body: arteries, ar-

terioles, capillaries, venules and veins. Together they form closed circulatory

routes for blood to travel from the heart to body organs and back again. In the

systemic circuit, arteries carry oxygenated blood away from the heart. They

divide into smaller arteries and arterioles until they reach a tissue or organ

when they branch into numerous microscopic capillaries. After the gas, nu-

trients and wastes exchange, capillaries merge together to form venules and

form bigger veins. Veins carry deoxygenated blood back to the heart.

There is also a second, pulmonary, circuit. In this circuit deoxygenated

blood is pumped from the heart to the lungs where gas exchange occurs and

oxygenated blood is carried back to the heart to be pumped out again in the

systemic circuit.

The largest artery in the body is the aorta with a diameter of two to four

centimeters [44]. The aorta starts at the heart, forms the aortic arch and

continues downwards until it splits at the aortic bifurcation into the left and

right common iliac arteries. Common iliac arteries soon split again into ex-

ternal and internal iliac arteries, which distribute the blood to the lower limbs

(through femoral and popliteal arteries and their branches) and the pelvis, re-

spectively [36]. An illustration of peripheral arteries is given in Figure 3.1.

3.1.1 Peripheral Artery Disease

Peripheral artery disease (PAD) is a common circulatory disease, which oc-

cludes the abdominal aorta, iliac, and lower-extremity arteries [53]. The most

19

Figure 3.1. Ilustration of the peripheral arteries. Used with permission of Sheri Amsel,

www.exploringnature.org; (adapted).

common cause of the occlusion is due to atherosclerosis, which is character-

ized by a combined process involving a long-term accumulation of a plaque

(e.g., cholesterol) on the arterial wall and inflammation in the wall [19]. The

plaque cumulation results in a reduction of the arterial lumen which impairs

the blood flow and limits the oxygen supply to the organs and to the lower

limbs [74]. Until the time when an artery gets completely occluded, minor

collateral arteries may grow and dilate to offer an alternative path around the

blocked artery. In late stages, the atherosclerosis is also commonly associated

with calcifications in the arterial wall.

3.1.2 Diagnosing PAD

A physician can usually diagnose peripheral artery disease based on the med-

ical and family histories, reported symptoms and physical exam. A common

test is ankle-brachial index (ABI), which compares the blood pressure in the

ankle to the blood pressure in the arm [29]. However, whenever drug and

life-style interventions are not adequate, and revascularization therapy is con-

sidered, a detailed description of the state of each major artery in the lower

limbs is needed. Such a description can be obtained using different medical

20

imaging techniques. A brief description of these techniques follows:

Catheter angiography is the oldest way of imaging the arteries and it in-

volves the introduction of a catheter into the arterial system. Once the catheter

is in place, an iodinated radiopaque contrast medium is injected through the

tube and images are acquired. The image acquisition is performed by digital

subtraction angiography, however, other imaging techniques are also applica-

ble. The advantages of catheter angiography are the possibility to supplement

the imaging with pressure measurements carried out through the catheter and

combining diagnosis and treatment in a single procedure. Factors that limit

the use of catheter angiography are potential medical complications caused by

the catheter and contrast medium [28].

Digital Subtraction Angiography (DSA) utilizes fluoroscopy X-ray-based

technique for image acquisition. A pre-contrast image is subtracted from later

post-contrast images, which removes nonvascular structures from the images

and enhances blood contrast. Nowadays, it is only used with intraarterial in-

jection, i.e., catheter angiography. DSA offers very high spatial resolution

but gives a two-dimensional projection of the three-dimensional anatomy. An

exception is rotational angiography which is mostly used for intracranial ves-

sels [78].

Ultrasonography utilizes high-frequency ultrasound waves to capture live

images. It is capable of identifying stenoses (vessel narrowing) as well as ar-

terial wall thickening [57]. With the use of Doppler flow measurements, the

blood flow disturbance can be identified and quantified [21]. Ultrasound ex-

aminations are commonly used for selecting candidates for revascularization

or for clinical follow-up after intervention.

Computed Tomography Angiography (CTA) employs iodine-based con-

trast media and X-rays just like in catheter angiography, but the injection is

made intravenously. This reduces patient discomfort and risk of complica-

tions relative to catheter angiography. The image acquisition is performed by

CT, resulting in a detailed 3D image containing information about vessels as

well as surrounding tissues [62]. CTA is well suited for studying calcifica-

tions, composition of the vessel wall and lumen. The main factors limiting the

use of CTA are the radiation dose and potential negative effects of the contrast

medium.

Magnetic Resonance Angiography (MRA) uses a powerful magnetic field,

pulses of radio wave energy and, most commonly, an intravenous contrast in-

jection to acquire a detailed 3D image [60]. The injected contrast medium is

gadolinium-based which is less likely to cause allergic reaction than iodine-

based contrast-agents. The patient is not exposed to any ionising radiation as

21

used in DSA and CTA, therefore it is possible to acquire a set of images taken

at several phases (arterial or venous). In patients sensitive to contrast agents

several alternative MRA techniques, not depending on a contrast agent, are

available [47].

3.1.3 Problem statement

Medical imaging techniques like CTA provide a clinician with a detailed 3D

image of the vascular anatomy. Hundreds of 2D images stacked on top of each

other are, however, hard to interpret and simple visualization techniques such

as maximum intensity projection (MIP) or direct volume rendering (DVR) are

often of great help. Both of these techniques are based on casting simulated

rays through the volume; in MIP the brightest value along the ray is displayed,

whereas DVR uses various optical concepts such as emittance, reflectance and

absorption to calculate the displayed intensity [13]. For displaying tubular

structures, curved planar reformation (CPR) may be a more suitable tech-

nique. This technique generates longitudinal cross-sections along a previously

generated vascular centerline to show the vascular lumen, wall and a few mil-

limeters of surrounding tissue in a curved plane. Obtaining vascular center-

lines needed for CPR by manual interaction is feasible, but very tedious and

time-consuming to fit into a tight clinical workflow. A fast vascular centerline

extraction algorithm would improve the usefulness of CPR.

In clinical cases when a clinician is deciding whether to perform an in-

tervention, precise information and measurements are needed and vascular

segmentation is therefore required. Numerous methods for vascular lumen or

wall segmentation have already been presented in the literature [35, 38, 75].

For widespread use in clinical routine, however, time efficiency is extremely

important. Regardless of how much interaction the algorithm requires, most

working radiologists are not likely to accept computation times above 1 or

2 minutes. State-of-the-art algorithms do not meet this requirement. A pos-

sible exception could be if the segmentation process is initiated completely

automatically once the image data arrive in the picture archiving and commu-nication system (PACS) and only limited editing work requires attention of the

user [80].

Vascular pathologies, which are of great clinical interest, pose another chal-

lenge for the segmentation algorithms. Typical for peripheral artery disease is

the presence of calcifications and occluded arteries which are often accom-

panied by dilated collateral vessels. These collateral vessels tend to have

a tortuous and irregular course and are typically difficult to handle for cur-

rent segmentation algorithms. Calcifications with high radiopacity, also cause

problems for segmentation algorithms that might be adapted to the much lower

intensities of the contrast-filled lumen. Other challenges include narrow and

22

irregular lumen and intensity variations along the arteries caused by stenosis

or incomplete contrast filling.

In this thesis, we contribute in solving the above-mentioned problems by

proposing a vascular skeleton extraction method that is designed to detect

healthy arteries of different sizes as well as to cope with challenges intro-

duced by PAD. The algorithm might be combined with other segmentation

algorithms and initiate the segmentation process automatically providing the

approximate skeleton as a seed region once the image data arrive in the PACS.

The algorithm is thought to be suited for the use in CPR if the current center-

line representation using polyline segments would be replaced by splines.

3.1.4 CTA dataset of the lower limbs

The dataset used in the studies presented in Papers I, II, V and VI was obtained

from the clinical routine of the Radiology department of the University Hos-

pital in Linköping, Sweden. The dataset contains CTA from the abdominal

aorta to the feet and was acquired with a clinical question of arterial stenosis

or occlusion. CTA was acquired on a Siemens Somatom Flash scanner, using

settings of 80 kV and 228 mAs. The protocol included automated injection

of 75-100 ml of iopromide (Ultravist) 370 mg J/ml at a rate of 3.7-4.0 ml/s,

followed by 70 ml of isotonic saline injected at the same rate, and images were

acquired in the arterial phase.

3.2 Anatomy of the lungsThe lungs are the essential organ of the respiratory system and are responsible

for providing oxygen to the body’s cells and removing carbon dioxide from

them. The air is brought from the external environment through the upper

and lower respiratory tracts to the lung alveoli in a process called inspiration,

the reverse process being expiration. The lower respiratory tract consists of

the trachea, which branches into two primary bronchi that enter the lungs.

Inside the lungs the primary bronchi are further divided into smaller bronchi,

bronchioles and terminal bronchioles, until they reach the last bifurcation -

respiratory bronchioles. Respiratory bronchioles consist of alveolar ducts and

alveolar sacs which are covered with alveoli. Alveoli are the main sites of gas

exchange with the blood. [76].

All structures that form a passageway for air (from the upper respiratory

track until the terminal bronchioles) are called airways and are usually not

considered to be part of the lungs. Respiratory bronchioles and their branches

make up the primary lobules which are the anatomical units of the lungs. Sev-

eral primary lobules create a secondary lobule and those are the main com-

ponents of the parenchyma. The parenchyma is surrounded by the sub-serous

areolar tissue and together with the pleura, which is a thin sack covering and

23

Figure 3.2. Ilustration of the lung anatomy. Source: https://patient.info/health/the-lungs-and-respiratory-tract; (adapted).

protecting the lungs, form the main parts of the lungs [25]. An illustration of

the anatomy of the lungs is given in Figure 3.2.

3.2.1 Acute Respiratory Distress Syndrome

Acute Respiratory Distress Syndrome (ARDS) is a life-threatening respiratory

condition that is characterized by an inflammation in the lungs which leads to

increased pulmonary vascular permeability and collapse of alveoli [14]. As

a consequence, alveoli are not able to fill with air, and oxygen can not be

provided in a sufficient amount into the bloodstream. The lack of oxygen

may lead to a multiple organ dysfunction syndrome and, in the worst case,

to death. The explicit cause of this symptom is not known, however, some

clinical conditions, such as trauma, pneumonia or sepsis, are associated with

development of ARDS [15].

3.2.2 Treatment of ARDS

The treatment of ARDS requires the use of mechanical ventilation in inten-

sive care units. Mechanical ventilation assists the patient with breathing and

can be controled by two main parameters: tidal volume (Vt) and positive end-expiratory pressure (PEEP). Tidal volume is the amount of air that is pushed

by the ventilator to the patient lungs and PEEP is the positive airway pressure

maintained until the end of expiration. Different ventilation strategies have

different objectives in focus, e.g., to minimize oxygen toxicity, to re-open col-

lapsed alveoli, or to prevent atelectasis [2]. Lung-protective strategies usually

use low Vt values compared to hypoxemia-reducing strategies, where higher

24

(a) (b) (c)Figure 3.3. Example slices of thoracic CT acquired at low PEEP value (2mmH2O)

showing lungs with ARDS induced, (a) axial slice with red and blue lines showing the

location of the coronal slices framed with the same color, (b) coronal slice showing

reduced contrast between lung and surrounding structures, (c) coronal slice showing

higher lung boundary contrast.

Vt and PEEP are utilized. Deciding the right strategy is challenging, since the

response of the lungs to the treatment is very hard to predict.

3.2.3 Problem statement

Analysis of lung aeration is essential in order to understand and to improve

ventilation strategies and to reduce the mortality rate in ARDS [61]. Lung aer-

ation refers to the amount of air and its distribution inside the lung parenchyma.

In order to obtain this information, the lung parenchyma needs to be seg-

mented, which implies lung segmentation and non-parenchymal structures

(vessels, bronchi and nerves) removal. However, many traditional lung seg-

mentation algorithms do not perform well in 3D images of lungs acquired in

the presence of ARDS. These images are often completely or partly lacking

contrast at the outer boundary of the lungs, especially in low PEEP conditions.

Figure 3.3 shows an example of such a case.

ARDS studies performed on animal models make it possible to obtain a

sequence of volume images for one subject acquired at different ventilation

conditions. The problem of missing contrast in low PEEP condition might

then be addressed by gradually warping the lung segmentation obtained from

the most-contrasted volume in the sequence towards the least contrasted one

by means of intensity registration. This approach was successfully presented

in [48].

Human ARDS studies, compared to animal studies, can not afford a large

number of acquisitions per subject due to minimization of radiation doses in

patients. Having a sequence of very few images (commonly two or three) in-

troduces two new challenges: (1) large deformations of lung structures and

(2) considerable intensity changes inside the lung between the two images

25

in the sequence. Traditional gray-level based registration is not powerful

enough to overcome these large differences and additional information, e.g.,

in a form of landmark correspondences between the volumes, should be pro-

vided. The anatomical landmarks might be obtained by utilizing an airway-

tree segmentation and extracting the points where airway branches bifurcate.

Once the successful lung segmentation is reached and the regions containing

non-parenchymal structures are removed, the lung aeration can be obtained.

The amount of air in each parenchyma voxel is easily calculated from the

thoracic CT scan using the voxel’s HU value. The percentage of air A(pi, j,l)in a voxel p at position (i, j, l) is defined as

A(pi, j,l) =I(pi, j,l)−HUtissue

HUair −HUtissue, (3.1)

where I(pi, j,l) is the intensity of the voxel pi, j,l , HUair represent the Hounsfield

values for air and HUtissue is HU of tissue. Tissue is mainly composed of water

and is considered to have value 0 HU in this application.

Our contribution in solving this problem was the development of fast airway

tree segmentation, that handles the intensity heterogeneity of ARDS pathology

and segments a large number of branches. Each branch bifurcation provides an

anatomic landmark needed for improving the lung registration and subsequent

delineation. The airway-tree segmentation also serves for airway removal.

3.2.4 Dataset of thoracic CT

The dataset utilized for the development and evaluation of the airway-tree seg-

mentation algorithm presented in Paper III was obtained from the Hôpital de la

Croix-Rousse, Lyon, France, where the research team of Réanimation Médi-

cale is performing a research project based on an animal model with induced

ARDS. The use of piglets for this study was approved by the institutional

review board for the care of animal subjects. The dataset contains 3D tho-

racic CT volumes of piglets obtained at different ventilation conditions. In our

work we utilized volumes acquired with a constant tidal volume Vt (5ml/kg)

and PEEP of different values ranging from 2 to 20 cmH2O. A pair of volumes

was available for each of the ventilation parameter settings; one volume was

acquired at the end of inspiration and another one at the end of expiration. The

animal model is used instead of images from humans since there is a lack of

availability of such image sequences. However, the sequence of few images

is simulated by selecting very different ventilation conditions (extreme and

intermediate) and testing the lung registration on this sparse sequence.

26

4. Vascular skeletons

This chapter defines the morphological skeleton in the continuous as well as

the discrete cases and explains what the approximate skeleton used in our

algorithms represents and how this skeleton is evaluated. For a more detailed

discussion on different skeletonization principles and approaches please refer

to [63, 69].

4.1 Definition of a skeleton

A skeleton is a thinned representation of an object that preserves the topologic

and geometric properties of the object and reduces its dimensionality. The pro-

cess of obtaining a skeleton is called skeletonization. Skeletonization reduces

a 2D object to a curve-skeleton (centerline) consisting of only 1D structures.

A 3D object may be simplified either to a surface-skeleton consisting of 1D

and 2D structures or reduced to the curve-skeleton either directly from the

object or via the surface-skeleton.

The notion of the skeleton was first described in the work of Blum [4] using

an illustrative grass-fire analogy. According to Blum a skeleton of an object is

a set of loci, where the fire fronts would meet if all points on the boundary of

the object (made of isotropic, flammable material) were set simultaneously on

fire.

A more formal definition of skeleton is based on the maximal ball concep-

tion [73] (pp.673–674). Let X be a 3D object in continuous space, X ⊂ R3.

A ball B(x,r) centered at position x ∈ X having a radius r, r ≥ 0, is defined as

B(x,r) = {y ∈R3, d(x,y)≤ r}, where d(x,y) is the distance from x to y in R3.

A ball B(x,r)⊂ X is considered to be maximal if and only if there is no larger

ball B′(x′,r′) ⊂ X that contains B. The skeleton is then defined as the set of

centers x of all maximal balls inscribed in X . Maximal balls and the skeleton

of a rectangle and a block are illustrated in Figure 4.1.

4.2 Digital skeletons

In many applications, including medical image processing, skeletonization is

performed on discrete 3D images. The definition of the skeleton in discrete

space is analogous to the one in continuous space, however, some issues may

occur due to discretization. For example, in the discrete case the skeleton lines

27

(a) (b)Figure 4.1. (a) A curve-skeleton of a 2D rectangle shown with blue lines, (b) a surface-

skeleton of a 3D block shown with blue surfaces; in both cases examples of two max-

imal disks/balls are also shown in light gray.

can be wider than one voxel since the diameter of the ball is always an odd

number of voxels (assuming the center of the ball must be one of the voxels).

If the width of the object is an even number of voxels, then the ball will be

maximal when it touches the object boundary only on one side. Another ball

having its center in the neighboring voxel touches the boundary of the object

on the other side. As a result, both centers of these two maximal balls are

included in the skeleton.

To avoid this undesired property, skeletonization in discrete images is per-

formed using digital skeletonization approaches. The most primitive digital

approach simulates the Blum’s grassfire propagation in a discrete grid using

an iterative boundary peeling algorithm under certain topologic and geomet-

ric constraints [64]. Two other categories of digital approaches are based on

fully predicate-kernel based iterative algorithms [50, 56] and distance trans-

forms [5, 6].

Regardless of which algorithm is used for the skeletonization, a digital

skeleton should satisfy as many as possible of the following properties [52]:

• Subset of the original object, i.e., the skeleton should be obtained from

the original object by only removing object voxels (not adding).

• Thin, i.e., the skeleton should be precisely one voxel wide.

• Allows reconstruction of the object, i.e., the information contained by

the skeleton should be sufficient to recover the original object.

• Topologically equivalent to the object, i.e., the skeleton and the orig-

inal object should have the same number of components, tunnels, and

cavities. A cavity is an empty space inside the object and a tunnel can

be intuitively described as a canal passing through the object.

4.3 Our approximate skeleton

So far we considered generating a skeleton from a previously segmented vol-

umetric object. However, in this thesis, we work with a reversed idea of con-

28

structing the skeleton directly from the medical images and subsequently using

this skeleton to obtain the volumetric segmentation of the vasculature. There-

fore, in this thesis and appended publications, the term skeleton refers to a

rather concise and approximate representation of the vasculature.

4.3.1 Description of the approximate skeleton

Our approximate skeleton fulfils the criterion of being thin and roughly cen-

trally located. More precisely, it consists of a set of connected polyline seg-

ments, where the polyline vertices are centrally located within an orthogonal

vascular cross-section (Figure 4.2a) and the straight polylines connecting the

vertices are guaranteed to lie inside the vessel. However the connections are

not necessarily centered (Figure 4.2b). Our skeleton has the same topology

as the vascular tree represented in the digital volume scan. This means that

the skeleton may contain disconnections in the regions where the main artery

becomes blocked and the resolution of the volume is not enough to depict tiny

collateral arteries in the CT scan. The resulting skeleton is also a subset of the

original object and it is possible to reconstruct the vascular tree from it. How-

ever, the difference is in the reconstruction method, which needs to be more

advanced than a simple distance-based expansion. The reconstruction method

we use to reproduce an accurate segmentation of the vascular tree combines

level sets with an implicit 3D model of the vessels [79]. The method takes an

approximated skeleton as an input and generates a 3D vessel model.

The motivation for using such an approximate skeleton is that with this

method the 3D problem of finding a skeleton in a volume is decomposed into a

set of 1D processes of connecting two centrally located vertices (nodes) within

a vascular cross-section and analyzing the connection. To further reduce the

execution time, there is no need to detect the vascular nodes in every vascular

cross-section or every volume slice. Based on the diameter at the detected vas-

cular node some nodes do not need to be saved for further processing, which

results in having more nodes in smaller and more tortuous vessels compared

to larger vessels where the distance between the nodes can be longer. Trans-

ferring the problem to a lower dimensional space allows a high computational

efficiency and opens the possibility for an interactive clinical use.

4.3.2 Evaluation of the approximate skeleton

In this subsection, the metrics used in Papers I and II to evaluate the perfor-

mance of our approximate skeleton are explained, that is, an overlap rate Mo,

a detection rate Md and an average distance error Derr .

Overlap and detection measureThe overlap rate Mo and the detection rate Md are modifications of more tra-

ditional precision and recall measures, respectively. However, since the pro-

29

(a) (b)Figure 4.2. (a) The process of selecting a centrally located polyline vertex. A voxel

lying within an orthogonal vascular cross-section is selected (white square no.1). This

voxel is not well-centered between two horizontal edges so the voxel lying truly in the

middle (white square no.2) becomes a new potential central voxel. The same verifica-

tion is repeated iteratively for vertical and horizontal directions until a truly centrally

located voxel (black square no.3) is found. This voxel is considered a polyline vertex.

Yellow voxels represent three consecutive voxels outside of the vessel cross-section;

(b) An approximate skeleton of a vessel with light blue vertices and dark blue straight

line connections.

posed algorithm produces an approximate skeleton which does not overlap the

reference skeleton completely, measurements that allow a small displacement

had to be used for our evaluation. We considered the skeleton detected by the

proposed algorithm to be correct if the skeleton was included within the refer-

ence segmentation generated by the level set based algorithm proposed in [79]

from the initial reference skeleton. This segmentation algorithm is explained

in detail at the end of the next chapter.

The first metric Mo determines the overlap of the detected skeleton with the

reference vascular segmentation. This measure is important in order to know

if the skeleton detected by our algorithm is a true artery skeleton.

The second metric Md determines the fraction of the reference skeleton that

was successfully detected by our detected skeleton. This rate informs us about

how much of the true artery skeleton was detected by our algorithm.

For evaluation purposes, the detected skeleton, which consists of a set of

connected polyline segments needs to be voxelized. The set of voxels created

by the voxelization of the detected skeleton is denoted Nd . Similarly, the set

of voxels Nr was created by voxelization of the reference skeleton. We define

another subset called Cd of all those voxels from the voxelized detected skele-

ton that lie inside the voxelized reference segmentation Cd = Nd ∩ Sr, where

Sr denotes set of voxels from the voxelized reference segmentation. All sets

and subsets introduced in this paragraph are visually explained on a drawing

of an artery in Figure 4.3.

30

Figure 4.3. Drawing of an artery; (left) a situation where two artery branches were

extracted and one artery branch was missed by the evaluated algorithm is shown;

(middle) a similar situation where another two artery branches were extracted and one

artery branch is missing in the reference skeleton is shown; (right) subsets Cd =Nd ∩Srand Cd_centered = Nd_centered ∩Sr are shown.

The detected skeleton is, in general, less centered, thus can contain fewer

voxels than the reference skeleton within the same area and this may affect

the results. Therefore, we need to introduce yet another two sets of vox-

els Nd_centered and Cd_centered . Nd_centered is made by voxelizing the detected

skeleton that was properly centered and Cd_centered is defined as voxels lying

inside the voxelized reference segmentation that are also a subset of Nd_centered ,

Cd_centered = Nd_centered ∩ Sr. The left subfigure of Figure 4.3 shows Nd and

Nd_centered . Nd_centered is created from the detected segmentation Sd by thin-

ning. A middle artery branch has a sinuous shape. This shape depicts an ex-

aggerated situation and explains why Nd_centered is needed for the calculation

of the Md metric. The middle subfigure shows Nr together with the reference

segmentation Sr and the right subfigure depicts the subsets Cd and Cd_centered .

An overlap rate Mo is similar to the quantity assessment in [45] and is cal-

culated as:

Mo =|Cd ||Nd |

, (4.1)

A detection rate Md is determined as:

Md =|Cd_centered |

|Nr|. (4.2)

Note that the set of detected skeleton voxels Nd can be seen as a sum of true-

positives (TP) and false-positives (FP). Similarly, Cd = Nd ∩Sr then represents

a set of TP and the set of reference skeleton voxels Nr reflects the sum of TP

and false-negatives (FN). If we substitute these variables into the definition of

Mo and Md , then we will get precision and recall measures, respectively.

Average distance error measureThe average distance error metric Derr measures the average distance error

between the detected skeleton and the reference skeleton and defines how far

on average they are from each other. In this measure, the distance is calculated

31

from each voxel of the detected skeleton to the closest voxel of the reference

skeleton and then averaged by the number of voxels. To avoid a bias this mea-

sure is calculated only on those parts of the detected skeleton that overlapped

the reference segmentation.

32

5. Vascular segmentation

This chapter provides an overview of basic methods which form the core

of current state-of-the art for vascular segmentation. The last segmentation

method presented in this chapter was used in our papers in the evaluation. and

was proposed by our collaborators Wang et al. [79].

5.1 SegmentationSegmentation is the process of dividing an image or a volume into regions

that are similar with respect to some characteristic, usually intensity, texture

or location in a space. Ideally, the regions should be meaningful for a specific

task or correspond to distinct anatomical or pathological structures. If the

anatomical structure of interest is a vascular tree we refer to the process of its

delineation as to the vascular segmentation.

Segmentation is often the most important step in medical image analysis

since all futher measurements, feature extractions or visualisations are derived

from the segmentation results. Concurrently, it is also considered the most dif-

ficult step, as there are many challenges. The methods should be able to handle

anatomic variability and complexity and be robust to noise, low contrast be-

tween the structures or presence of different imaging artefacts. Especially,

partial volume effect is an artifact that is very common in medical images. A

possible way to address this difficulty is to allow regions in the segmentation

to overlap. In the next chapter we will discuss a particular group of methods,

called coverage segmentation methods, that allow partial membership.

In this chapter, we focus on standard approaches, that enforce binary deci-

sions in the volume partitioning process - binary segmentation. A voxel pi, j,leither belongs to the object Sk and has assigned the membership value mk = 1

or does not belong to the object and it has assigned the membership value

mk = 0

mk(pi, j,l) =

{1 if pi, j,l ∈ Sk,

0 otherwise.(5.1)

5.2 Preprocessing: Vesselness filterThe presence of nonvascular structures in medical images may affect the qual-

ity of vascular segmentation. Some segmentation approaches, therefore, pre-

process the image in order to enhance tubular vascular structures and suppress

other non-tubular structures e.g., kidneys.

33

(a) plane-like structure (b) tubular structure (c) blob-like structure

Figure 5.1. Example of the structure shapes. Eigenvalue close to 0 represents low

intensity curvature in the direction of the corresponding eigenvector. Large eigenvalue

and its corresponding eigenvector represent the direction of high intensity curvature.

Vessel enhancement approaches are based on the observation that the inten-

sities along the vessel direction are changing slowly (the intensity curvature is

low) compared to the intensities on the cross-sectional plane which are chang-

ing fast (the intensity curvature is high). A measure that characterizes a local

intensity change is the second-order partial derivative and a square matrix of

these derivatives is the Hessian matrix. For a 3D input image, the Hessian

matrix H(p), corresponding to the voxel pi, j,l , is given by

H(p) =

⎡⎢⎢⎢⎣

∂ 2I∂x2

∂ 2I∂x∂y

∂ 2I∂x∂ z

∂ 2I∂y∂x

∂ 2I∂y2

∂ 2I∂y∂ z

∂ 2I∂ z∂x

∂ 2I∂ z∂y

∂ 2I∂ z2

⎤⎥⎥⎥⎦ , (5.2)

where I is the intensity function of the image at the voxel pi, j,l . The eigen-

values λ1,λ2 and λ3 (|λ1| ≤ |λ2| ≤ |λ3|), and their corresponding eigenvectors

e1, e2, e3 of the Hessian matrix are closely related to vascular intensity and

direction. Analyzing them allows us to recognize different local orientation

patterns e.g., blob-like, tubular, plane-like or no preferred direction. See Fig-

ure 5.1 for an example. Ideally, a tubular structure should have eigenvalues

corresponding to |λ1| ≈ 0, |λ1| |λ2|, λ2 ≈ λ3.

In practice, the second-order partial derivatives at voxel pi, j,l are calculated

by convolving the input image I with Gaussian derivative G at scale σ

Hσ (p) = I(p)∗ ∂ 2Gσ (p)∂ p2

. (5.3)

34

(a) σ = (0,2〉 (b) σ = (2,4〉 (c) σ = (4,6〉 (d) σ = (6,8〉Figure 5.2. The maximum intensity projection of the volume preprocessed by the

vesselness filter with different range of σ ; step size between sigmas was set to 0.25.

Vessels as well as bone edges that resemble locally a tubular structure respond to

sigma values that correspond to their sizes.

A vesselness filter proposed by Frangi et al. [17] based on Hessian matrix

theory is then defined at scale σ as:

ν(σ)=

{0 if λ2 > 0 or λ3 > 0,

(1− exp(− R2

A2α2

))exp

(− R2

B2β 2

)(1− exp

(− T 2

2γ2

)) otherwise,

(5.4)

where α , β and γ are sensitivity controlling thresholds for dissimilarity mea-

sures RA, RB and T , respectively. These measures are based on eigenvalues

of H(p) and are defined as

RB =|λ1|√|λ2λ3|

, (5.5)

RA =|λ2||λ3|

, (5.6)

T =√

λ 21 +λ 2

2 +λ 23 . (5.7)

The dissimilarity measure RB distinguishes blob-like structures from the plane-

like and tubular structures while the measure RA differentiates between plane-

like and tubular structures. The last measure T reduces the response of back-

ground voxels. To enhance a tubular object of different sizes, vesselness filter

has to be applied at multiple scales and the final response is performed by

keeping the maximum response over the scales. Figure 5.2 shows an example

of vesselness filter being applied at different scales to a volume image.

However, Hessian-based filters are sensitive to local deformations, such as

bifurcations, stenosis or aneurysms. They may induce subtantial spatial blur

if large scale σ is used (Figure 5.2d) and they may be computationally costly.

35

(a) (b) (c)Figure 5.3. Example of thresholding with the threshold range [150, 525] HU; (a) an

axial 2D slice of the original volume, (b) a thresholded 2D slice, (c) an ISO surface

rendering of the thresholded 3D volume; Vessels as well as bones were segmented

since the range of their intensities overlaps.

A mono-scale approach proposed by Bauer at al. [3] is less sensitive to struc-

ture deformations and uses a vector field obtained from gradient vector flow

diffusion [81].

5.3 Segmentation methods

There are many different approaches to obtain a segmentation and no approach

suits all purposes. A brief overview of the methods follows, however more

detailed overviews of other vascular segmentation techniques are provided in

papers [35, 38].

5.3.1 Thresholding

Thresholding [23] is the simplest segmentation method. It divides the vol-

ume into regions based only on the voxel intensity and it ignores the voxel

spatial position. A suitable intensity value, called threshold value, is selected

either manually or automatically by analyzing the volume histogram. All vox-

els having intensity smaller than the threshold value are then grouped into one

class, e.g., representing the object and all the other voxels are members of the

background class. This technique is well suited for segmenting anatomical

structures that have contrasting intensities compared to the rest of the volume

e.g., bones. To separate object from the background with nonuniform illu-

mination, local adaptive thresholding is well suited. This technique selects

a different threshold value for each voxel in the volume based on the local

36

(a) (b) (c)Figure 5.4. Example of region growing; (a) an axial 2D slice of the original volume,

(b) a thresholded 2D slice, (c) an ISO surface rendering of the thresholded 3D volume

with a seed point marked by the red dot.

intensity characteristics. Thresholding is often part of more complex image

processing pipelines.

When using thresholding for blood vessel segmentation, an intensity in-

terval, called threshold inverval, has to be applied. Only voxels having an

intensity from this interval range are grouped together into the vessel class.

However, intensity range that is typical for the vessels overlaps the bone in-

tensities and therefore thresholding alone is not sufficient approach for vascu-

lar segmentation. Figure 5.3 shows the result of thresholding with a threshold

range [150, 525] HU that was calculated from the volume intensity histogram

using the automatic method presented in Paper I.

5.3.2 Region-growing

Region-growing segmentation [1] starts from a seed voxel or a seed region and

succesively includes neighboring voxels if they fulfil inclusion criteria. These

criteria are usually based on intensity information or on edges detected in the

image. The segmentation growing process continues until no more voxels can

be added to the segmentation. Figure 5.4 shows an example of region-growing

initiated from a seed voxel marked by a red dot in Figure 5.4c. The inclusion

criterion was set to include any neighboring voxel that has intensity within

the range [150, 525] HU. The result compared to the thresholding method

contains only voxels that are connected with the seed. The advantage of this

method is the simplicity and computational efficiency. The disadvantage is the

sensitivity to noise. The method is prone to create holes inside the segmen-

tation, or leak into other structures if the inclusion criteria are locally insuffi-

cient. Many methods are reducing the risk of leakages by introducing some

37

growth-limiting criteria. We used a modification of region-growing algorithm

in Paper III which uses an intensity- and distance-limiting criteria.

Region-growing for vessel segmentation was utilized by Eiho et al. [12],

where the algorithm was performed on one vessel branch at the time so the

inclusion criteria could be adapted according to the local neighborhood char-

acteristics.

5.3.3 Centerline-based methods

Centerline-based segmentation methods either require a centerline on the input

or they detect it as the initial step. Subsequently the centerline is used to extract

the vascular surface through various geometric and spatial constraints.

There are approaches that detect the vessel wall independently in each 2D

cross-sectional plane utilizing 2D active contours [40], ray-casting schemes [26]

or circular dynamic programming [20]. A collection of 2D contours are then

joined into a 3D surface using geometric models. Other approaches are able to

extract the surface directly in 3D utilizing 3D parametric active surfaces [46],

B-spline tensor surfaces [16] or level set evolution [77,79]. A graph-cut-based

approach applied for coronary arteries was presented by Schaap et al. [67].

This method segments the vessel lumen with a Markov Random Field and

then removes falsely segmented regions with a kernel regression approach.

5.3.4 Geometric deformable model

Geometric deformable models were introduced concurrently by Malladi et

al. [43] and Caselles et al. [9]. They combine curve evolution theory [66]

with level set methods [55].

Starting with an initial curve C at t = 0, the image segmentation is per-

formed by evolving the shape of this curve according to the partial differential

equation∂C∂ t

=V (κ)N, (5.8)

which is driven by a speed function V (κ), where κ is the curvature and N is

the inward normal of the curve. The design of the speed function can differ

between the implementations and influences the segmentation performance.

Usually the speed function combines the speed of internal deformation (cur-vature deformation that shrinks the curve and/or constant deformation that

inflates the curve) with the speed dependent on the image data in such a way

that the curve evolution stops at object boundaries.

The evolving curves can be represented implicitly (using only geometric

computations without the need of parametrization) as a higher dimensional

level set function φ(t) defined on the same image domain. The advantage of

38

(a) (b) (c)Figure 5.5. An intersection of the level set function φ (blue surface) with a plane

xy at z=0 (red surface) is illustrated on the top image. This intersection defines an

evolving curve at different time points t. The curve (a dark blue contour) together

with the resulting segmentation (blue area enclosed by the curve) is shown on the

bottom picture.

such a representation is that the topology changes can be handled automati-

cally and multiple objects can be detected simultaneously. In this representa-

tion the curve is a set of points on the image domain for which the function

φ(t) is zero. The curve evolution is then represented by updating the level

set function φ(t) at fixed time points. Figure 5.5 illustrates the principle of

the level set method. The equation 5.8 which specify the curve evolution is

replaced when using the level set method by new evolution equation

∂φ∂ t

=V (κ)|∇φ |. (5.9)

The initial level set function is often based on the signed distance function

D(x,y) from each point on the grid to the zero-level set, φ(x,y,0) = D(x,y).All the formulations were explained for 2D segmentation, however they can

be extended to 3D.

In conventional model based segmentation methods, the curve propagation

is often limited by the statistical shape models. Creating statistical shape mod-

els for vessels is difficult due to large anatomical variation between different

subjects. Therefore, in the work of Wang et al. [79] which is used in our vas-

cular segmentation the curve propagation is limited by a vessel model derived

as a union of several cylinders from the inserted centerline. The method is

detailed in the following subsection.

39

5.3.5 Skeleton guided level set based vessel segmentation

This vessel segmentation method was proposed by our collaborators Wang et

al. [79] and is intended to utilize the vascular centerline tree extracted by the

method described in Paper I.

The method constructs an implicit 3D vessel model φ from the input cen-

terlines as a union of cylinders with a varying radius R(x), where x is a point

on the centerline L. The initial vessel radius is assumed to be 1 voxel uni-

formly for the entire vessel tree. This vessel model is then incorporated in the

speed function and regulates the growth of the vessel contour. The evolution

equation, used in this work, is defined as

∂φ∂ t

= (αvimage +βvmodel + γvinternal)|∇φ(x)|, (5.10)

where α,β ,γ are the weighting factors for every speed term. The speed terms

are defined as

vimage = S−|I(x)−T |, (5.11)

vmodel =−d3m(x), (5.12)

vinternal = κ(x), (5.13)

where I(x) is the intensity of the input image at x, T is the center and S is

the width of the thresholding window. The vessel tree model is represented

by a signed distance map dm, that is generated by a distance transform from

the centerlines, with points on the centerlines set to their initial values −R(x).Then on the surface of the model dm(x) is zero, it is negative inside the model

and positive if it is outside.

After evolving the level set φ , some re-modeling steps are needed. New

centerlines are extracted from the shape given by the zero level set through a

fast marching scheme. New radius function R(x) is estimated for every vessel

branch by fitting a smoothed curve to the distance function measured from the

center points to the current zero level set, dl(x), with x ∈ L. Using the new

centerlines and radius functions, an updated 3D tree model is created. The

steps of the level set propagation and vessel model re-estimation are iteratively

repeated until convergence.

The advantage of this method is that the input centerlines do not need to

be perfectly centered, their approximate position is sufficient. The centerlines

will be re-centered during the algorithm iterations. This enables to use a fast

centerline extraction algorithm to detect the approximate centerline tree pos-

sition quickly and then re-center this position while preforming the vascular

segmentation. The resulting segmentation has, thanks to utilizing the geomet-

ric deformable models, subpixel precision, which can be precious.

40

6. Coverage segmentation

Generalization of the binary segmentation is a fuzzy segmentation based on

fuzzy set theory [82]. Fuzzy segmentation, compared to the binary one, parti-

tions a volume into overlapping regions and allows voxels to belong to more

than one object. This reduces the risk of assigning a voxel to the wrong ob-

ject and preserves a larger amount of the information which results in more

precise measurements and feature extractions. Each voxel in the fuzzy seg-

mentation is associated with a membership value, in the range from zero to

one, which represents the degree of voxel membership to a particular volume

objects. If the membership values correspond to the voxel coverage by an

observed image object, we call such a partitioning a coverage segmentation.

In the following sections we formally define fuzzy set theory and present the

coverage representation. Several methods for the coverage segmentation will

also be covered. For a more detailed description of coverage model and its use

in image processing the reader is refered to [72].

6.1 Fuzzy set theory

A fuzzy subset S of a reference set X is a set of ordered pairs

S = {(x,μS(x))|x ∈ X}, (6.1)

where μS : X → [0,1] is the membership function of S in X [82]. Important

notions related to fuzzy set are support and core. The support of S is set of

points having strictly positive memberships to the set S and the core of fuzzy

set S is defined as the set of points with memberships to S equal to 1.

6.2 Coverage representation

Coverage segmentation uses a representation where the membership function

values correspond to the portion of volume of the voxel covered by the object,

i.e., to the voxel coverage. Formally, coverage segmentation S(I) of an image

I into m components is defined as a set of ordered pairs

S(I) ={(

(i, j, l),α(i, j,l))∣∣∣(i, j, l) ∈ ID

}, (6.2)

41

(a) (b) (c)Figure 6.1. Example of a vessel-like object (a) high resolusion binary representation,

(b) low resolution coverage representation, (c) low resolution binary representation

where

α(i, j,l) = (α1, ...,αm),m

∑k=1

αk = 1, αk =V (pi, j,l

⋂Sk)

V (pi, j,l),

and where Sk ⊂ R3 is the extent of the k-th component (object), V (pi, j,l) de-

notes the volume of a voxel pi, j,l and αk are coverage values. ID ⊆ Z3 is the

discrete image domain. It is simple to compute coverage values αk for voxels

covered by simple continouos geometric objects Sk that can be analytically

defined. However, to extract this information in general, from more complex

and more realistic objects in digital volumes, we have to estimate it from the

image data using coverage segmentation methods. An example of a vessel re-

sembling object is given in Figure 6.1. Coverage segmentation is preserving

the smoothness and other features of the object better than the binary segmen-

tation.

6.3 Coverage segmentation methods

Coverage segmentation of an object is very close to its binary segmentation.

For voxels that are fully covered by, either the object or the background, the

two segmentations are identical. Only for the border voxels, where the par-

tial coverage appear, the values differ. Therefore, the coverage segmentation

methods may focus on adjusting an already existing binary segmentation, in-

stead of re-inventing the whole range of different segmentation methods again.

In the following subsections, we first present a simple coverage segmentation

method based on direct intensity mapping, then one which builds on an ex-

isting binary segmentation, and, finally, one more general method based on

energy minimization, which does not require a priori knowledge about the

binary segmentation of the object in the image.

42

6.3.1 Coverage segmentation based on double thresholding

Coverage segmentation based on double thresholding [70] is the simplest cov-

erage segmentation method. A prerequisite to a successful application of this

method is a volume where the intensity distributions of the object and the

background are well separable; then the range of grey-levels on the transition

between the two classes can be used for estimating coverage values.

The method seeks for a couple of threshold values, f and b, where f is

the intensity estimation of voxels completely covered by the object and bis the intensity estimation of completely covered background voxels. These

thresholds can be found automatically based on the method presented in [70].

The method estimates the values in such a way that the fuzzy boundary be-

tween the object and the background is not more than one voxel thick and the

contrast between them is as large as possible. Voxels darker than b belong

completely to the background and are therefore assigned coverage value zero,

voxels brighter than f belong completely to the foreground and are assigned

coverage value one. The greyscale intensities I(pi, j,l) of the one voxel thick

separating boundary are then normalized between values f and b

α(pi, j,l) =I(pi, j,l)−b

f −b. (6.3)

The new normalised values represent the coverage values of the boundary vox-

els.

6.3.2 Coverage segmentation by local unmixing

Coverage segmentation by local unmixing [71] is based on an existing binary

segmentation. It is assumed that this binary segmentation is trustworthy for all

but boundary voxels. The method identifies the set of these boundary voxels Bas all voxels that are 6-connected to a voxel with a different label. Two major

steps, linear unmixing and thinning, are then performed on the set B.

Linear unmixing stepFor each voxel in this set, the partial coverage value is calculated utilizing a

local linear mixture model. Based on this model, the image intensity I(p) of

the boundary voxel p is assumed, in a noise-free environment, to be a convex

combination of the values ck corresponding to the pure class k, for m classes,

k = 1, . . . ,m:

I(p) =m

∑k=1

αkck,m

∑k=1

αk = 1, αk ≥ 0, (6.4)

where each coefficient αk represents the coverage of the pixel p by a class

k. The pure class representatives ck are usually not known and are estimated

locally from a weighted average of completely covered voxels from the bi-

nary segmentation. In real imaging conditions, equation 6.4 can not be solved

43

exactly due to a presence of noise. The problem has to be reformulated as find-

ing an image intentisity I∗(p) = ∑mk=1 α∗

k ck and I∗(p) is as similar as possible

to I(p). This allows to formulate coverage segmentation as an optimization

problem, with the data fidelity term D(p) = ‖I(p)− I∗(p)‖2. See [71] for

details.

Thinning stepWe assume that the continuous objects have a reasonably smooth and crisp

boundary. This implies that the fuzzy boundary should not be more than one

voxel thick. Due to presence of noise, set B may be thicker and needs to be

thinned in the final step. In the 2D version of the algorithm, the thinning is

performed using an ordered thinning algorithm. In ordered thinning, pixels

that have the smallest difference in intensity to one of the pure classes are

assigned the membership value of that particular pure class. This iteratively

repeats until the resulting set of pixels creates a 4-connected one pixel thick

boundary between the neighbouring objects. In the 3D version of the algo-

rithm, the ordered thinning would be a rather complex procedure when the

topology preserving is required. In Paper V we propose an alternative ap-

proach to the boundary thinning in 3D based on morphological erosion. This

alternative morphological approach can be perfomed equally well in 2D and

offers an alternative method for segmenting thin and elongated objects.

6.3.3 Energy based coverage segmentation

Energy based coverage segmentation [41] performs the whole segmentation

as one optimization task. For each object, the method requires a sample of the

pure class representatives on the input. This can be achieved either by defin-

ing small sample regions of the pure class representatives or by providing a

binary segmentation of each object (the latter is the case for the local unmix-

ing based method [71]). The method then finds the coverage segmentation

S by minimizing the energy function J over the space of possible coverage

segmentations, with several regularization terms

J(S) = D(S)+μP(S)+υT (S)+ξ F(S), (6.5)

where D(S), P(S), T (S) and F(S) is the data term, overall perimeter, bound-

ary thickness and total image fuzziness, respectively. The weighting parame-

ters μ , υ and ξ ≥ 0 give appropriate importance to the different terms. The

data term D(S) itself provides an unmixing segmentation and the rest of the

terms add spatial information which makes the coverage segmentation less

sensitive to noise. The argument S which minimizes J provides a coverage

segmentation with a smooth boundary of each objects and support majority of

object pixels to be classified as pure. This results in improved performance

compared to coverage segmentaion based on local unmixing. However, due to

44

(a) (b) (c) (d) (e)Figure 6.2. Comparison of the coverage segmentation methods performed on a piece

of a retinal vessel, (a) original image with the input binary segmentation marked with

white contour, (b) outcome from the linear unmixing before the thinning step, (c)

thinned result of the local unmixing based coverage segmentation [71], (d) result

of the energy based coverage segmentation [41] with the pure class representatives

marked with red squares, (e) result of our coverage segmentation method proposed in

Paper IV.

iterative optimization, energy based method is more computationally demand-

ing. See [41] for further details.

6.4 Coverage segmentation applied to thin structuresThe presented coverage segmentation methods are based on the asumption

that the boundary of continuous structures is reasonably smooth and the fuzzy

boundary between the neighbouring structures is not more than one pixel

thick. This assumption is correct for thick structures, however, in case of e.g.,

retinal vessels, hairs on the skin or microtubules in the cytoskeleton, which are

often only 1-2 pixels thick in total this assumption may not hold.

The coverage segmentation method based on local unmixing reduces, in the

ordered thinning step, the thickness of the structure boundary as long as the

topology stays preserved. This means that all partially covered pixels from the

tips of long vessel branches are removed until the nearest pure structure pixel

is reached. This is undesired, since important sub-voxel information, wich we

aim at capturing by coverage representation, is lost.

The difficulty with the energy based coverage segmentation is mainly a

strong similarity between the object and the background intensities and a dif-

ficult selection of proper regularization parameters. This leads to a noisy and

incorrect coverage segmentation.

A comparison between different coverage segmentation methods, performed

on a 2D image containing a segment of thin and long retinal vessel is shown

in Figure 6.2. Crisp segmentation incorporates many partially covered pixels,

which is clearly visible in comparison with Figure 6.2b. Only a few bright pix-

els are fully covered by the object and all the remaining ones are mixed. Such

an oversegmentation, as presented in Figure 6.2a leads to an object that will

45

unavoidably provide poor measures (e.g., object area, length, thickness, etc).

Figure 6.2c clearly shows undesirable effects of the method presented in [71],

where topological thinning does not ensure preservation of partially covered

elongated structures. Figure 6.2d indicates deficiencies of the energy based

method [41], leading to a noisy and oversegmented result. All these issues are

successfully overcome by our proposed method presented in Paper IV, suited

for coverage segmentation of thin structures. This algorithm was also applied

to artefacts removal in our Related Paper 3, which led to improved skin lession

segmentation in dermoscopic images.

46

7. Deep neural networks

This chapter gives a brief introduction to deep neural networks for supervised

learning with the main focus on convolutional neural networks (CNN) to pro-

vide a background for Paper VI. As the topic of deep learning in neural net-

works is very broad, to get a deeper understanding we recommend exhaustive

review papers such as [37,68] or the books by Nielsen [51] and Goodfellow et

al. [24].

7.1 Deep learning in neural networks

Deep learning belongs to the class of machine learning algorithms which fo-

cuses on pattern analysis, classification, feature extraction and transforma-

tion [11]. Humans are remarkably good in these activities, and therefore, it

is not surprising that visual cortex (an area of the brain which is responsible

for recognizing patterns) inspired the way how deep learning algorithms work.

Study on a visual cortex showed that simple cells respond to certain visual in-

puts, e.g., the presence of edges of a certain orientation. These simple cells are

organised in a columnar architecture and all cells together are able to produce

a visual perception [32, 33].

Similarly, neural networks for deep learning consists of many layers of lin-

ear and non-linear information processing ordered into a cascade. The signal

path traverses from the first (input) layer, to the last (output) layer. This direc-

tion of passing through the network is called forward pass. Between the input

and the output layer there are one or more layers referred to as hidden layers.

The first hidden layer detects low level features (edges, colors,...), the second

hidden layer detects features which are a combination of the previous ones

(corners, basic shapes,...) and the more we advance through the network the

more complex and abstract these features become. The last layer can easily

detect or classify high-level features, e.g., faces, persons, dogs, etc.

Each layer consists of elements called neurons. Outputs from the neurons in

one layer are the inputs to the neurons in the next layer. Each neuron receives

a number of input values xi and calculates their weighted sum using weights

wi j and adds a bias b j to it. The resulting value is a parameter for an activation

function f (a linear or non-linear transformation) which decides the output

value y j. This can be summarized with formula

y j = f (∑i

wi jxi +b j). (7.1)

47

Network training is the process of finding appropriate weights wi j and a bias b jfor each neuron automatically. Training requires a labeled training dataset and

is usually done using back-propagation. During the training process, labeled

training samples are classified by the network and resulting classifications are

compared to the labels. Then a loss function, which specifies the final error

of the network performance, is calculated. The loss function is minimized

by passing the gradient of the error back through the network and adjusting

the weights and biases by an amount proportional to their contribution to the

error. The contribution to the error (the rate of change of error with respect

to the parameter) is calculated by applying the chain rule of differentiation.

Passing through the network in the direction from the output layer back to

the input layer is called backward pass. After the gradient of the error with

respect to all the parameters are calculated the parameters are updated. The

parameter update is usually done by stochastic gradient descent or its variant.

The forward pass, the backward pass and the parameter update are repeated

until a satisfactory network performance is reached [34, 51].

The networks usually consist of different types of layers which can be or-

dered in the cascade in many different ways. The order and type of the layers

define the network architecture. One type of architecture is a convolutional

neural network.

7.2 Convolutional neural networks

Convolutional neural network (CNN) is a neural network architecture special-

ized in processing data that has a grid-like topology, e.g., images, therefore it

is used for computer vision and image analysis applications [24].

The CNN architecture contains at least one layer that employs a mathemat-

ical operation called convolution. This layer is referred to as convolutional

layer. The traditional CNN architecture consists of more than one convolu-

tional layer interleaved with nonlinear and pooling layers and the network

ends with fully connected layers. In the following subsections we will briefly

describe the most popular types of layers.

Compared to a regular neural network, the layers in CNN have neurons ar-

ranged not in a vector, but rather in a volume of three dimensions: width,

height and depth. The first hidden layer receives a flat and shallow input

volume of activations (outputs of the neurons) having the same dimensions

as the input image (e.g., 32×32×3 for a color image of size 32×32). The

CNN architecture transforms one volume of activations to another one result-

ing into a single vector of class scores, arranged along the depth dimension

(e.g., 1×1×6) [39]. An example of simple CNN architecture illustrates this

transformation of volume of activations in Figure 7.1.

48

Figure 7.1. Simple CNN network architecture consisting of two convolutional layers

interleaved with two pooling layers and one final fully-connected layer. The input

activation volume of size 32×32×3 transformed to 1×1×6 vector of output classes.

7.2.1 Convolutional layer

The convolutional layer is the core building block of a CNN. It consists of a

set of 2D learnable convolution filters (kernels). The size of these filters is

relatively small, usually between 1×1 to 11×11 pixels, and is referred to as

a receptive field of the filter. The receptive field is a user-defined parameter.

A neuron in this layer is connected only with those neurons from the previous

layer which lie in its receptive field [39].

Each filter slides (convolves) across the input volume and at every spatial

position it computes the dot product between the weights of the filter and the

input values. The result is one 2D activation map per filter and represents the

responses of that filter at every spatial position. In convolutional layers, the fil-

ter weights are shared, i.e., they are independent regarding the position where

they are applied and the update of the weights during the backpropagation is

consistent across the full image. Shared weigths and small receptive fields re-

sult in several orders of magnitude less weights compared to fully connected

networks and allow a more efficient computation [24].

In addition to receptive fields, there are other user-defined parameters as-

sociated with each layer that control the size of the output activation volume:

the depth, stride and zero-padding. The depth corresponds to the number of

filters used in each layer. The stride specifies the horizontal and vertical slide

of the filter. Zero-padding enlarge the input volume by certain number of ze-

ros around the border and controls the spatial size of the output volume. It is

mostly used to preserve the input volume size.

7.2.2 Pooling layer

The pooling layer down-samples the filter response from the previous layer by

applying a non-linear function. A common type of the layer is max poolingwhich applies the max function. In max pooling layers, the input for the layer

is partitioned into a non-overlapping rectangles and the maximum value of

49

each partition is the new output value. The reason behind this is that only the

most dominant features will propagate further through the network and their

relative position compared to the other dominant features is more important

than their exact position. By reducing the number of features, this layer helps

prevent overfitting. The user-defined paramters in this layer are: the type, sizeand stride.

7.2.3 Rectified linear unit (ReLU)

The ReLU is one type of activation function which determines the output

through a non-linear transformation defined as

f (x) = max(0,x), (7.2)

where x is the input to the ReLU. Other types of activation functions are, e.g.,

the sigmoid function or the hyperbolic tangent function.

7.2.4 Batch normalization layer

The batch normalization layer normalizes the activations of the previous layer

at each batch (a subset of the training set), i.e., it applies a transformation that

results in the mean activation being close to 0 and standard deviation of the

activation being close to 1.

7.2.5 Fully-connected layer

The last layer is always a fully-connected layer. This layer, like the regular

neural network layers, connects each neuron in the layer to every single activa-

tion from the previous layer. Fully connected layers result in one-dimensional

output and therefore there can not be a convolution layer placed after a fully-

connected one.

7.3 Our CNN classifier

In Paper VI we designed a CNN architecture for classification purposes. The

input to this network are 2D images representing a potential vascular cross-

section. Each image has a constant size 31×31 pixels and depicts vessels

or non-vessels of different sizes. The output of the network gives a decision

whether the 2D image is truly a vascular cross-section or not. Further details

and the architecture description of the proposed CNN classifier are provided

in Chapter 8 and Paper VI.

50

8. Contributions

This chapter summarizes the methods and results presented in the appended

publications. The background and context to this work was provided in the

previous chapters.

8.1 Fast vascular skeleton extraction algorithm (Papers Iand II)

Papers I and II present a fast and fully automatic vascular skeleton extraction

algorithm. This algorithm is intended to be a part of a complete arterial tree

segmentation framework, where the approximate vascular skeleton is used as a

seed region for subsequent segmentation algorithms. In Paper I, we developed

a method that focuses on extracting a skeleton from healthy arteries of vary-

ing sizes. Paper II expands the algorithm and focuses on diseased peripheral

arteries, i.e., arteries with severe calcifications and collateral arteries.

MethodNext, we discuss the algorithm for both methods, the one for heatlhy arteries

and the one for diseased ones.

Skeleton of healthy arteries: The method takes a 3D unprocessed CTA

scan as input and produces a graph representing an approximate vascular skele-

ton. The nodes of the graph represent centrally located voxels within arterial

cross-sections and edges of the graph represent direct connections between the

nodes within a vascular branch. The approximate skeleton of large and small

arteries is extracted in two algorithm levels starting with the skeleton of large

arteries. Each level consists of four main steps:

1. A parameter selection step automatically selects appropriate parameters

based on vascular morphology and the intensity histogram of the data.

2. An artery node detection step detects voxels that are centrally located

within potential arterial cross-sections and applies a set of knowledge-

based filters to reduce false-positives.

3. An artery node connection step iteratively connects the nodes utilizing

the distance and intensity information and constructs a graph structure

representing the approximate vascular skeleton.

4. An anatomy based analysis step identifies and removes spurious graphs

or graph segments and keeps only those graphs that correspond to the

vascular tree.

51

Skeleton of diseased arteries: The method for extracting a skeleton of

diseased arteries expands the previous version of the algorithm by adding two

new algorithm levels thereby creating a cascading structure. The first level,

where the skeleton of large arteries is detected, serves as a reliable basis and

new graph segments are appended to this basis in the subsequent levels. The

first additional level extracts the skeleton of arteries which contain severe cal-

cifications and the second additional level detects the skeleton of tiny arteries

in the lower extremities and collateral arteries. The same four main steps that

were presented in the previous version of the algorithm are performed in every

level of the expanded version. The main difference is in the selection of the

parameter values which are chosen to be well suited for detecting the desired

appearance of diseased arteries. The cascading structure of the algorithm pro-

vides a possibility to add more new levels, where each level might be adapted

to a new vascular disease.

ResultsNext, we discuss the results for each of the proposed methods, the one for

heatlhy arteries and the one for diseased ones.

Skeleton of healthy arteries: The method was evaluated on 25 CTA scans

of the lower limbs taken from the clinical routine. The proposed method

achieved a very good overlap rate and a good detection rate compared to a

united skeleton, which is a union of the reference skeleton and the resulting

skeleton corrected by the radiologist. However, the algorithm had difficulties

to detect the skeleton of diseased vascular segments. The resulting average

distance error between the detected and united skeleton was 1.35 mm which

reflects the fact that the approximate skeleton consists of polyline segments

instead of arc segments. The average computational time of the algorithm

was 88 seconds per CTA volume using a single-threaded implementation. It

is possible to parallelize the algorithm and by utilizing four CPU cores with

2 threads per core, the average running time dropped to 29 seconds. This is a

considerable improvement compared to ca. 15-20 min per CTA volume which

Figure 8.1. Detected skeleton printed in red colour is overlayed with a volume region

depicting bones of a right foot displayed in isosurface rendering mode with intensity

level equal to 600 HU. The vascular skeleton was successfully extracted even from

very distal arteries in the right foot.

52

Figure 8.2. Comparison of the skeletons: (a) maximum intensity projection (MIP)

of the volume with MIP of three cutouts with manually removed bones are shown.

The cutouts depict calcified iliac artery (top) and occluded femoral arteries (middle,

bottom), (b) the reference skeleton, (c) the resulting skeleton from Paper I, (d) the re-

sulting skeleton from Paper II. Resulting skeleton in (d) contains more small branches

compared to the two other skeletons. However, it also detected a false artery segment

marked with a red arrow and did not connect occluded femoral arteries.

was the time the radiologist needed to extract the reference skeleton using a

semi-automatic method.

Skeleton of diseased arteries: The extended version of the algorithm was

evaluated on the same dataset as the previous version of the method in order

to make results comparable. The new results showed the improvement in the

detection rate, which was also confirmed by a visual assessment. The result-

ing skeletons were more complete with longer branches and contained less

disconnections. An example of a successful extraction of skeleton from distal

arteries in the right foot is shown in Figure 8.1. A connection between two

occluded arterial parts, however, was not always created because the resolu-

tion did not provide sufficient intensity evidence to identify this connection.

The increased sensitivity of the algorithm to detect small artery skeletons in-

creased the number of false artery segments compared to the previous version

of the method. This is reflected by a decreased overlap rate value. The false

artery segments were detected mainly on surface of bone or in regions where

the beam hardening artifacts were present. The average running time of the

53

Figure 8.3. The histogram of a single case is shown with the three relevant ranges

detected automatically (fat, muscle and blood vessels). Logarithmic histogram shows

a peak that represents bone tissue having intensities ca. within the range [500,1500]

HU.

algorithm was 196 s per CTA volume using a single-threaded implementation

and 70 seconds per CTA volume using the parallelized version, which suits

the needs of clinical examination. Figure 8.2 depicts a comparision of the ref-

erence skeleton with the two resulting skeletons obtained by the two proposed

versions of the skeleton extraction algorithm.

8.2 Automatic adaptation to intensity variation betweenscans (Paper I)

Paper I features a parameter selection step that automatically selects the in-

tensity ranges of tissues relevant to our proposed algorithm (i.e., fat, muscles

and blood) for each CTA scan. The Hounsfield scale should be relatively stan-

dardised between CT machines for most soft tissues. However, the HU value

of contrast-mixed blood may vary considerably between the scans. This is

caused mainly due to variations in timing between injection and image ac-

quisition and variations in hemodynamics. We propose a volume intensity

histogram analysis based on Gaussian curve fitting.

MethodA smoothed histogram of CTA volume intensities can be, in general, described

as a succession of the following peaks: a peak reflecting air inside the volume,

two peaks representing fat and muscles and a small peak describing blood

54

vessel intensities. A peak representing bone tissue is clearly visible only if we

plot a logarithmic histogram. Figure 8.3 shows the histogram and logarithmic

histogram of a single CTA volume. To find the intensity ranges corresponding

to fat, muscles and blood, a sum of Gaussian curves is fitted to the image

histogram using the non-linear least squares fitting method. If any of the peaks

is too flat for the Gaussian curve to fit the data reasonably well, fallback ranges

are used. The fallback ranges might be particularly useful when the method is

applied to a cropped CT volume, where all tissues are not necessarily present

in sufficient quantities.

ResultsThe automatic parameter selection was tested on 25 CTA scans and the de-

tected intensity ranges roughly corresponded to the typical HU values of par-

ticular tissue reported in the literature. In eight cases the fallback range for

blood intensities was used and we observed that in these cases the proportion

of arteries containing calcifications was higher than in the rest of the volumes.

The high proportion of calcification may have caused that the typical blood

vessel intensities were not present in a sufficient amount to be properly re-

flected in the volume histogram.

8.3 Skeleton-guided onion-kernel based segmentation(Paper III)

Paper III proposes a rapid airway-tree segmentation algorithm. The develop-

ment of this algorithm is part of a joint project on acute respiratory distress

syndrome (ARDS). This project includes researchers from France, Colombia,

Poland and Sweden. The main goal of the project is to reduce the mortality

rate in ARDS by analyzing the lung aeration on CT images in order to learn to

predict the response of the lungs to different ventilation strategies. A more de-

tailed statement on this problem is provided in Section 3.2.3. The airway-tree

segmentation is needed for the removal of non-parenchymal structures and for

providing anatomical landmarks which are important for the lung registration

and subsequent delineation. This segmentation method was developed for the

purpose of airway segmentation, however, it is possible to use it for vessels

segmentation without the need of any modifications other than appropriate

parametrization.

MethodThe airway-tree segmentation algorithm consists of two main steps:

1. An airway-tree skeleton detection step follows the same main steps as

the vascular skeleton detection algorithm presented in Paper I. However,

to successfully address a challenging intensity heterogeneity of ARDS

55

(a) (b) (c)Figure 8.4. Onion-kernel region-growing, (a) the input skeleton is voxelized, (b) each

skeleton voxel becomes a seed; the segmentation is not allowed to grow backwards

to fill the cavities (the dark green border delineates the region that would be filled

otherwise, (c) the final segmentation.

the parameters and the set of knowledge-based filters have to be modified

to reflect the appearance of the airways in the diseased lungs. Three

classes representing three different levels of voxel aeration are defined

and each potential airway node is treated by following the rules defined

by the class corresponding to its intensity. For each airway node the

radius and intensity range corresponding to the aeration class of the node

is saved.

2. A modified onion-kernel region-growing segmentation step is initiated

from the detected airway-tree skeleton. The poly-line segments are vox-

elized (Figure 8.4a) and each voxel becomes a seed for an ordered region-

growing propagation (Figure 8.4b). From the seed voxel the algorithm

propagates only outwards, with the same speed in all directions and adds

a new voxel only if it is 26-connected to another voxel segmented in the

earlier layer. This ensures locally convex segmentation without segmen-

tation “overhangs” (Figure 8.4c) and limits propagating sideways when

a leakage into parenchyma occurs. To restrict the size of the leakage, the

propagation is limited also by distance and intensity constraints.

ResultsThe airway-tree segmentation algorithm was tested against a reference seg-

mentation based on a traditional region growing algorithm introduced by Mori

et al. [49]. A qualitative assessment was performed on 70 thoracic CT vol-

umes of piglets with induced ARDS acquired at various volume and pressure

settings. The proposed method detected a larger number of small branches

compared to the reference segmentation, particularly in images acquired at

low-pressure conditions (See Figure 8.5 bottom row). The reason for obtain-

ing notably better results in images with lower PEEP is a locally increased

contrast between dense parenchyma and the airway lumen. A higher num-

ber of detected branches leads to more bifurcation points which can serve as

anatomical landmarks.

56

Figure 8.5. Qualitative airway segmentation results, (a) Coronal slice of a thoracic

CT showing the differences in parenchyma intensities between high and low PEEP,

(b) result of the reference method, (c) result of the reference method overlayed with a

skeleton from the proposed method (d) result of the proposed method.

Since the proposed airway-tree segmentation algorithm is meant to be a

part of a larger lung registration and segmentation framework, the quantita-

tive evaluation of the algorithm was performed indirectly by comparing two

registration-based lung-segmentation methods. A method using a hybrid reg-

istration, where intensity information was enriched with airway-tree landmark

correspondences yields to improved lung segmentation compared to a tradi-

tional intensity-driven registration.

In addition, two hybrid registration methods were compared. One method

used landmark correspondences extracted from the proposed segmentation and

the other one used the landmark correspondences from the reference segmen-

tation. This two registration methods produced comparable results, which re-

flects the fact that even though the number of landmarks is notably greater

using the proposed method, the spatial distribution of the landmarks cover an

area similar to the area of landmarks obtained using the reference method.

This points to a limitation of the hybrid registration.

The complete proposed segmentation method achieves the average com-

putation time of 43 seconds per thoracic CT volume when running a single-

threaded implementation. The skeleton extraction algorithm takes 41 seconds

on average and the onion-kernel region growing algorithm takes an average

time of only 2 seconds.

57

8.4 Coverage segmentation of thin structures (Papers IVand V)

Coverage segmentation methods, as introduced in Chapter 6, are based on

the assumption that the fuzzy boundary between the neighboring structures

is only one voxel thick. This assumption may not hold for 1-2 pixel thick

elongated structures, e.g., retinal vessels, hairs on the skin or microtubules

in the cytoskeleton. In Paper IV, we developed a 2D method for coverage

segmentation, especially suited for precise segmentation of thin structures.

In Paper V, we expanded this method to 3D. In addition, we suggested an

implementation that enables lower memory consumption and lower processing

time.

MethodNext, we discuss the algorithm for the 2D and 3D versions of the method.

2D version: The method starts with an original grayscale image and an

existing binary segmentation of an object. To achieve a coverage segmentation

it performs four steps:

1. A linear unmixing step is utilized the same way as in [71] to obtain

preliminary coverage values.

2. An upsampling and centre of gravity attraction step combines informa-

tion available from the linear unmixing with the information from the

original image and estimates a new binary segmentation at increased

spatial resolution. Coverage information indicates how large area of a

pixel is covered by the object. This information allows to estimate the

number of fully covered pixels in the new high-resolution image corre-

sponding to this coverage value. Information from the original grayscale

image is needed for estimating the position of these pixels by using a lo-

cal centre of gravity attraction.

3. A minipixel shifting step corrects holes and protrusions that may appear

in the high-resolution binary segmentation by locally re-arranging the

pixel positions guided by the original binary segmentation.

4. The final down-sampling step converts the high-resolution binary seg-

mentation into a coverage segmentation of the input image size.

3D version: A coverage segmentation of 3D thin structures is an expansion

of the 2D method and consists of analogous steps. The main difference is in the

upsampling step. An input volume upsampled by an integer factor of n (where

n = 3) would lead to very large volumes to store and to process. Therefore,

our algorithm proposes a modification, where a sliding cube of size 4×4×4

is used to scan through the volume and it up-samples only a couple of voxels

within the cube at a time. This modification radically descreases the execution

time, where the modified version is 12 times faster than the original one. The

two versions exhibit similar performance in terms of the average absolute error

of the coverage values.

58

(a) (b)Figure 8.6. Average absolute error of coverage values of object border pixels obtained

by (a) a 2D version of the method tested on a thin star, (b) a 3D version of the method

tested on thin conical spiral, at noise levels ranging 0%-40%. Examples of the tested

objects are shown in the upper left corner.

ResultsNext, we discuss the results for the 2D and 3D versions of the method.

2D version: The evaluation of the 2D version of the method was performed

qualitatively using retinal vessel images. See Figure 6.2 for an example. The

proposed method is confirmed to outperform previous versions, [41, 71], in

case of thin objects, i.e., the method does not produce unintuitively thick

fuzzy boundaries, does not shrink the vessels and is not sensitive to similar

intensities of noise and vessels. Quantitative evaluation regarding sensitiv-

ity to noise was performed observing two types of synthetic objects (thin star

and thin rounded square) randomly placed at 50 different positions within an

image at each observed noise level. We evaluated the average absolute error

of the coverage values over 50 repetitions at nine different levels of additive

uncorrelated Gaussian noise with standard deviation σ up to 40% of the fore-

ground/background contrast. The graph in Figure 8.6 (a) illustrates the results

and shows that the proposed method improved the performance, especially in

the presence of noise. All considered coverage segmentation methods, even

in the presence of noise, outperform crisp segmentation applied to noise-free

objects. The proposed method outperforms all other methods as soon as the

noise level exceeds 5%, and is better than the noise free crisp segmentation

even when the image is corrupted by as much as 25% of noise.

3D version: The evaluation of the 3D version of the method was performed

qualitatively on real 3D CTA images and quantitatively by measuring the av-

erage absolute error of coverage values at increasing noise levels using two

synthetic objects. The graph in Figure 8.6 (b) confirms very good perfor-

mance of the method in the 3D case as well. The accuracy of voxel coverage

59

values estimated by the proposed method is higher even in the presence of up

to 35% of additive uncorrelated Gaussian noise, than what is achieved by crisp

segmentation in a noise free-case.

The algorithm for coverage segmentation of thin structures was applied also

to thin artefact removal which lead to improved skin lession segmentation in

dermoscopic images, as presented in reference [42] which is listed as Related

Paper 3 in this thesis.

8.5 High resolution crisp reconstruction (Papers IV andV)

Papers IV and V provide a method for high resolution binary reconstruction

of a structure. High-resolution binary images are a by-product of the coverage

segmentation method and are obtained by skipping the final down-sampling

step of the method. To obtain a high-resolution volume image, the 3D version

of the method has to run without the sliding-cube modification.

ResultsThe 2D method was compared with the nearest neighbor up-sampling method

on 15 manually-segmented ground truth segmentations of retinal images. Pre-

cision, recall and f-score, as well as specific features describing vasculature

(i.e., connectivity, area and length), are computed based on pixel-wise com-

parisons. It is shown that the proposed method outperforms the basic upsam-

pling method. Same conclusion can be derived for the 3D method which was

compared with the nearest neighbor up-sampling using 20 synthetic 3D con-

ical spirals by measuring precision, recall and f-score. Figure 8.7 shows an

example of resulting images both for 2D and 3D case.

8.6 Boundary thinning in 3D (Paper V)

Coverage segmentation by local unmixing contains a boundary thinning step

where a set of coverage pixels is reduced to a one-pixel-thick fuzzy bound-

ary between the structures. In 2D, this step is performed by ordered thinning,

however, in 3D the topology-preserving thinning procedures are rather com-

plex. In Paper V we propose an alternative approach to boundary thinning

that can be efficiently implemented in 3D. The method uses a binary erosion

operation and a set of constraints for deciding whether a voxel assigned a par-

tial coverage should be excluded from the boundary and be fully part of the

object/background or not. This alternative approach prevents the undesired

shrinking of thin protrusions, which was observed with the previously pro-

posed method.

60

Figure 8.7. Comparison of the high-resolution crisp reconstruction methods, (a)

ground truth segmentation, (b) result from nearest neighbor up-sampling, (c) result

from the proposed method. For the 3D case the maximum intensity projection (MIP)

is shown.

8.7 Convolutional neural network classifier (Paper VI)

In Paper VI we propose an alternative method for vascular skeleton extraction,

where a convolutional neural networks (CNN) classifier was used to distin-

guish between true and false 2D artery cross-sections. The classifier replaced

the set of knowledge-based filters used in Papers I and II in order to reduce the

large number of false positive nodes.

MethodThe method is a modified version of the algorithm presented in Paper I and

focuses on improving the node classification step. We replaced the original

set of knowledge-based filters with a CNN classifier. The other steps of the

method remain unchanged.

The classifier receives a 2D image of a potential vascular cross-section of

size 31×31 pixels from an orthogonal slice on the input and based on the

trained CNN, it classifies the image into two categories: vessel and non-

vessel. The developed CNN classifier consists of four convolutional layers,

with a max-pooling layer placed after every second convolutional layer. The

illustration of the model is provided in Figure 8.8. The first convolutional

layer contains 32 kernels of size 3×3×1 and is padded with a two-pixels thick

frame of zeros in order to keep the spatial sizes of the patches unaltered after

the first convolutional layer. The second and third convolutional layers consist

of 32 and 64 kernels, respectively, of size 3×3×32. The last convolutional

layer contains 64 kernels of size 3×3×64. The max-pooling layer reduces the

61

Figure 8.8. An overview of the CNN classifier proposed in Paper VI, showing the

output of each convolution filter applied to an example patch of a vessel. Here, the

greyscale intensities are shown in false color for suitable visualization.

size of feature maps by selecting the maximum feature response in windows

of size 2×2 and stride of 2.

ResultsWe evaluated the proposed CNN classifier qualitatively in terms of preci-

sion, recall, and F-score on the model-evaluation subset containing labeled

2D patches not used in the model development process. The precision of the

proposed CNN classifier was 0.81 for medium-sized vessels compared to the

precision of only 0.28 yielded by the method utilizing knowledge-based filters.

These numbers confirm that the CNN classifier successfully reduced the num-

ber of false positives. As a consequence, the algorithm pipeline of the skeleton

extraction method that utilizes the CNN classifier was simplified compared to

the algorithm presented in Paper I.

Qualitative evaluation was performed by visual comparison of the result-

ing skeletons extracted from 21 CTA volumes of the lower limbs comparing

the algorithm proposed in Paper I with the proposed algorithm. Figure 8.9

shows the comparison for two representative volumes. The results after the

second step confirm that the classification by CNN leads to a fewer number of

false-positives than the other method. Final results of volume A (Figure 8.9a)

illustrate that the proposed method detects more vascular branches, and com-

pared to the method based on the knowledge-based filters, does not contain

spurious graph segments in the pelvic region. On the other hand, occasion-

ally CNN classifier discarded a larger amount of true vessel candidates, which

lead to missing a complete vessel branch. Volume B (Figure 8.9b) illustrates

such situation. The missed vessels were either very small or they were very

diseased and the CNN classifier would need larger training dataset to classify

them correctly. In volumes where the vessel appearance resembles the appear-

ance of vessels in the training dataset the proposed method performed very

well.

Regarding the computation time, the CNN classifier takes ca. 30 seconds to

classify a set of 130,000 images compared to ca. 22 seconds which are needed

to process the same amount of images by the set of knowledge-based filters.

62

(a) Volume A (b) Volume B

Figure 8.9. Results after each algorithm step shown for two volumes. The set of po-

tential nodes (black) is the same for both algorithms. In the second step classifiers

remove a portion of non-vessel nodes from this set. In the third step nodes that re-

mained are connected and create skeletons. In the final step anatomy based analysis

cleans the skeletons from spurious branches.

63

9. Conclusion and future perspective

We conclude this thesis by providing a short description of how our main

objective and specific aims were fulfilled and we suggest some directions for

the future work.

9.1 Summary of contributions

In this thesis, we proposed a fast and fully-automatic vascular skeleton ex-

traction algorithm. The algorithm detects approximate vascular skeletons in

healthy (Paper I), as well as in diseased (Paper II) arteries. This algorithm,

in combination with the segmentation method presented by our collaborators

Wang et al. [79], creates a complete arterial tree segmentation framework of-

fering a segmentation with subpixel precision. In Paper III, we suggested a

new segmentation method that benefits in speed from the input skeletons and

is based on a modified onion-kernel region-growing segmentation in order to

rapidly segment anatomical tree structures resulting in a binary segmentation.

To increase the precision of this segmentation we introduced two coverage

segmentation methods that work in 2D images and 3D volumes, Papers IV

and V, respectively. These coverage segmentation methods work by turning

the binary segmentation into a fuzzy segmentation and are particularly suited

for application in thin and elongated structures, such as blood vessels. In Pa-

per VI, a convolutional neural network classifier was designed to replace a set

of simple knowledge-based filters (used in Papers I, II and III) reducing the

false-positive rate in vascular cross-sectional classification.

These contributions are well in line with the main objective of the thesis: to

develop fast and automatic methods for approximate skeleton extraction and

segmentation of tubular structures from the volume CT datasets.

Conclusion of specific aims:I The proposed algorithm for vascular skeleton extraction provides

an approximate skeleton of large CTA scans of the lower limbs

in approximately 29 seconds per scan when running on four CPU

cores. This running time is compatible with the clinical needs. In

addition, the method adapts its internal parameters automatically

for each scan based on the volume’s intensity histogram analysis.

The method was tested on 25 CTA scans and achieved an average

overlap rate of 97% and an average detection rate of 71%.

65

II The skeleton extraction algorithm is extended by two new levels

into a cascading structure, where one level improves the skeleton

extraction of strongly calcified vessels and the other level detects a

larger number of small arteries with longer branches. The method

was tested on the same CTA datasets as the previous method and

achieved an average overlap rate of 89% and an average detection

rate of 82%.

III The method for airway-tree segmentation detects a larger num-

ber of small branches, especially in thoracic images acquired at

low-pressure conditions, when compared to the reference algorithm

based on region-growing that is initiated with a single seed. Anatom-

ical landmarks extracted from each branch bifurcation provide use-

ful additional information to enable the lung segmentation in im-

ages with poor contrast when performing a hybrid registration.

IV The presented coverage segmentation method is designed to pre-

serve thin and elongated protrusions of the objects. The qualitative

evaluation confirmed the preservation of these protrusions. When

comparing the sensitivity to noise the presented method showed

better performance than all other available methods for coverage

segmentation.

V The method was successfully extended to segment thin structures in

3D images. We propose a modification that radically decreases the

execution time thus allowing it to cope with large 3D data. In the

quantitative evaluation, the accuracy of estimated coverage values

of the voxels by the proposed method was higher even in presence

of up to 35% of noise, compared to what was achieved by crisp

segmentation in a noise free-case.

VI A classifier based on CNN greatly reduces the false positives in the

vascular cross-sectional classification. The CNN classifier yielded

an f-score of 0.82 for the medium size vessels which is two times

better than what was achieved with the knowledge-based filters.

The f-score for smaller and bigger vessels was 0.65 and 0.72, re-

spectively.

9.2 Future work

Several ideas for a continuation can be drawn from our work. One idea is

to expand the algorithm for vascular skeleton detection from CTA to MRA

as this imaging modality is widely used in clinical examination and produces

volume images. Since the grey-level intensity scale in MRA is not normalized

and standardized between the machines as it is in CTA, the intensity volume

histogram analysis will be a crucial part of the algorithm. We expect that after

66

the parameters are properly selected, the conceptual idea of the rest of the

algorithm should work without major modifications.

One of the main challenges in vascular skeleton detection and segmentation

is posed by tiny and diseased arteries and their large variability in appearance

depending on the type and severity of the disease. In Paper II we proposed

a cascading algorithm, where two levels of the algorithm were specialized in

detecting two specific diseases. One way to cope with this challenge is to add

more new levels which would be specialized in other diseases like aneurysms

or thrombosis. Another way is to use machine learning algorithms. We have

already shown in Paper VI that the CNN classifier trained only on 4 CTA vol-

umes yields acceptable precision and recall in classification of middle-sized

vessels. If more training samples were provided, containing an adequate rep-

resentation of both heathy and diseased arteries of several types and sizes the

results have a potential to be largely improved. The classifier can be futher

enhanced by taking multiple patches from different axes oriented planes or

to expand to 3D and be trained on 3D clusters of the candidates. The use of

machine learning does not need to be limited to application to classification,

the detection step, can also be replaced by machine learning-based methods.

However, one challenge will be to overcome the lack of proper training data

for very small vessels, since these vessels are hard to segment manually and

it is possible that the training data contains forgotten small vessels annotated

as non-vessels. In addition, tiny vessels having only 1-2 pixels in diameter do

not provide enough information for the learning process and additional infor-

mation about spatial position within the volume or relative to other voxels that

are potential vessel candidates would need to be taken into account.

In Paper III we proposed an airway-tree segmentation algorithm suited for

cases suffering from ARDS. In diseased lungs, bronchi may contain mucus,

which consists mainly of liquid and may cause bronchi to appear disconnected

on the CT scan. Currently the algorithm detects both of the disconnected parts

of the bronchus, however, for the future it could be beneficial to include a

deeper anatomical analysis based on the validation of angles at the bifurcation

to close these gaps and provide even more anatomical landmarks. This could

potentialy remove the length limitation in the connections between the nodes.

Finally, the principle of extracting the skeleton from a structure can be ex-

tended to extraction of the structure surface. Similarly designed knowledge-

based filters can be used to detect a point on the outer surface and cluster it

with other points that have similar intensity and significant gradient creating

a mesh. This may be useful for rapid segmentation of anatomical structures,

such as lungs or colon.

67

Summary in Swedish

Kärlsjukdomar är en av de vanligaste dödsorsakerna i utvecklade länder [54],

där tidig diagnos och korrekt behandling är vitalt för att minska antalet döds-

fall. Tack vara medicinska bildupptagningstekniker kan kärlens sjukdomstill-

stånd undersökas och utvärderas utan behov av öppen kirurgi, vilket mar-

kant reducerar risken för komplikationer och samtidigt ökar patientkomfor-

ten. Dock har de teknologiska framstegen i icke-invasiv vaskulär medicinsk

avbildning lett till en ständigt ökande mängd stora och komplexa dataset som

behöver visualiseras, analyseras samt tolkas. Genomgång av dessa stora da-

tamängder är ett monotont, felkänsligt och tidsödande arbete vilket lett till ett

stort intresse för automatiserade bildbehandlingsmetoder utvecklade för att as-

sistera röntgenläkaren till en snabb och korrekt diagnos och därigenom minska

arbetsbördan.

Kärlsegmentering har en nyckelroll bland dessa bildbehandlingstekniker

då de kan bidra till att erhålla precisa mått på sjukdomstillstånd, understödja

planering av behandling samt användas för övervakning. Det kliniska använ-

dandet av metoder för kärlsegmentering står dock inför ett antal utmaningar.

Segmenteringsalgoritmerna måste i hastighet vara jämförbara med interakti-

va undersökningar av bildvolymen. Algoritmerna ska inte kräva mer än ett

minimum av interaktion och samtidigt ge en segmentering av kärl av varie-

rande storlek i det kompletta kärlträdet. Dessutom behöver algoritmen vara

tillräckligt robust för att korrekt segmentera både friska och patologiska kärl,

de senare av stort kliniskt intresse. Dessa mål uppfylls inte av någon av de

algoritmer som tidigare presenterats.

I denna avhandling fokuserar vi på att lösa ovanstående problem. Segmen-

tering av rörformade strukturer såsom blodkärl underlättas om ett approxima-

tivt kärlskelett finns att tillgå innan segmenteringen påbörjas. Vi presenterar

en snabb och helautomatisk algoritm för skapandet av ett kärlträdskelett som

ger en startpunkt för en efterföljand ytgenerering vilket accelererar och under-

lättar segmenteringsprocessen. Denna algoritm för kärlträdskelettextrahering

är designad för att hitta det kompletta kärlträdet inklusive patologiska kärl

och kärl av varierande storlek. Vi introducerar en snabb segmenteringsalgo-

ritm som utnyttjar kärlträdskelettet för att skapa en binär segmentering. Vi har

utvecklat metoder som utgående från en initial binär segmentering skapar en

mer exakt täckande segmentering med subpixelprecision. Dessutom föreslår

vi en klassificerare baserad på ett så kallat faltat neuralt nätverk (convolution

neural networks classifier, CNN) för att ytterligare förbättra algoritmen för

extrahering av kärlträdskelett.

69

De i avhandlingen nämnda metoderna designades speciellt med tanke på

kärlsegmentering, men de kan även appliceras för segmentering av andra ana-

tomiska trädstrukturer, där segmentering av bronkträd i lunga har implemen-

terats och validerats i Artikel III. De metoder för kärlsegmentering som fö-

reslagits i denna avhandling har utvärderats på angiografiska datortomografi-

volymer (Computed Tomography Angiography, CTA) av nedre extremiteter,

utvalda utmanande fall ur den kliniska rutinverksamheten. De metoder som

presenterats för segmentering av bronkträdet har validerats på griskultingar

med inducerad Acute Respiratory Distress Syndrom (ARDS).

70

Acknowledgements

Focus on the journey,

not the destination.

Joy is found not in finishing

an activity but in doing it.

Greg Anderson

Walking the stony, winding path leading to the writing of this thesis would

not have been possible –nor would I have enjoyed the journey so much– with-

out a great and supportive company. I would like to express my great appreci-

ation to those who accompanied me on this journey. My special “Thank you”

belongs to:

• Hans Frimmel, my main supervisor, for always being very supportive,

for trusting me even when I did not. For guiding me through my doctoral

studies with ease and wit. For your help when I needed it and for your

knowledge.

• Ewert Bengtsson, my assistant supervisor, for keeping your door always

open for me. For your support, helpful discussions and for sharing your

huge knowledge and experience.

• Örjan Smedby, my assistant supervisor, for providing me with your

knowledge and insight into medicine and radiology. For always find-

ing time for me in your busy schedule.

• My collaborators, Chunliang Wang, Nataša Sladoje, Joakim Lindblad,

Tomáš Majtner, Duván Gómez, Marcela Hernández Hoyos, MaciejOrkisz, Anindya Gupta, and Ida-Maria Sintorn, for fruitful discus-

sions, for your hard work, good collaboration and valuable opinions and

comments. I learnt a lot by working with you!

• Marcela Hernández Hoyos, for your hospitality and supervision in Colom-

bia. Being a part of your team was a trully valuable experience.

• Carolina Wählby, Ingela Nyström, Nataša Sladoje, Joakim Lindblad,

Maciej Orkisz, Maxime Bombrun, Christophe Avenel, Nadia AssraouiAvenel, Sajith Sadanandan Kecheril, and Anindya Gupta, for proof-

reading and commenting on selected chapters of my thesis. A special

thanks to Carlos Pérez Penichet for constructive criticism and proof-

reading the whole thesis.

71

• Lena Nordström, for always being very kind and helpful and for taking

care of everything and everybody at Vi2. Your work is much appreciated.

• Astrid Raidl, for taking care of my computer and for carefully checking

the logs on the servers :). I will not scare you with my unfamiliar IP

address anymore.

• The late Olle Ericsson, for taking care of my computer and for being a

nice office mate.

• All my office mates, Azadeh Fakhrzadeh, Omer Ishaq, Fei Liu, FredrikWahlberg, Vlada Curic, Christophe Avenel, Ingrid Carlbom, ChristerKiselman, and Sajith Sadanandan Kecheril, for nice talks and making

my time in the office very enjoyable.

• All the past and present colleagues and friends at Vi2 for creating a great

and inspiring research environment.

• All my colleagues who became my very good friends, Alexandra Pacure-anu, Maxime Bombrun, Andre Liebscher, Damian Matuszewski, Elis-abeth Linnér, Pontus Olsson, Azadeh Fakhrzadeh, Vlada Curic, AmitSuveer, Anindya Gupta, Leslie Solorzano, Gabriele Partel, and EvaBreznik.

• Leslie Solorzano and Duván Gómez, for taking care of me in Colombia.

Leslie, I hope I have been able to repay, at least partially, your kindness

and great advice. Duván, thank you for showing me around Medellin

and Lyon.

• All my Uppsala friends. Special thanks to Carine, for sharing your

passion for knitting and scouting with me. Carine and Tobias, for all the

trips and nice times we had together. Anastasija, for all the adventures

in the nature we undertook together.

• Christophe Avenel, for being such a true friend, for helping me out

during the hard times and for sharing the joy during the good times.

Christophe and Nadia Avenel, for all the unforgettable moments. You

mean so much to me.

• Šamot, for everything you did for me and for your love. You taught me

to believe in myself. Without you, I would not achieve so much and I

am very grateful.

• Carlos, for being my close friend, for your love and support. I am learn-

ing every day with you.

• My family, for supporting me and believing in me. My mother, Andrea,

for being always here for me. Mirko, because I know that my mother is

in the best hands. My brother, Adrián, for taking care of everything at

home while I am away and for being the best brother I could have ever

wished for. Miška and Maya, for your positivity and contagious smile.

72

References

[1] R. Adams and L. Bischof. Seeded region growing. IEEE Transactions onPattern Analysis and Machine Intelligence, 16(6):641–647, 1994.

[2] A. Artigas, G. R. Bernard, J. Carlet, D. Dreyfuss, L. Gattinoni, L. Hudson,

M. Lamy, J. J. Marini, M. A. Matthay, M. R. Pinsky, et al. The

American–European consensus conference on ARDS, part 2: ventilatory,

pharmacologic, supportive therapy, study design strategies, and issues related to

recovery and remodeling. American Journal of Respiratory and Critical CareMedicine, 157(4):1332–1347, 1998.

[3] Ch. Bauer and H. Bischof. A novel approach for detection of tubular objects

and its application to medical image analysis. In Joint Pattern RecognitionSymposium, pages 163–172. Springer, 2008.

[4] H. Blum. A transformation for extracting new descriptions of shape. In Modelsfor the Perception of Speech and Visual Form, pages 362–380, MIT Press,

Cambridge, MA, 1967.

[5] G. Borgefors, I. Nyström, and G. Sanniti di Baja. Computing skeletons in three

dimensions. Pattern Recognition, 32(7):1225–1236, 1999.

[6] G. Borgefors, I. Nyström, G. Sanniti di Baja, and S. Svensson. Simplification of

3D skeletons using distance information. In International Symposium onOptical Science and Technology, pages 300–309. International Society for

Optics and Photonics, 2000.

[7] T. Boskamp, D. Rinck, F. Link, B. Kümmerlen, G. Stamm, and P. Mildenberger.

New vessel analysis tool for morphometric quantification and visualization of

vessels in CT and MR imaging data sets. Radiographics, 24(1):287–297, 2004.

[8] J. T. Bushberg and J. M. Boone. The Essential Physics of Medical Imaging.

Lippincott Williams & Wilkins, 2011.

[9] V. Caselles, F. Catté, T. Coll, and F. Dibos. A geometric model for active

contours in image processing. Numerische Mathematik, 66(1):1–31, 1993.

[10] N. D. Cornea, D. Silver, and P. Min. Curve-skeleton properties, applications,

and algorithms. IEEE Transactions on Visualization & Computer Graphics,

(3):530–548, 2007.

[11] L. Deng and D. Yu. Deep learning: Methods and applications. Foundations andTrends in Signal Processing, 7(3–4):197–387, 2014.

[12] S. Eiho, H. Sekiguchi, N. Sugimoto, T. Hanakawa, and S. Urayama.

Branch-based region growing method for blood vessel segmentation. In

Proceedings of International Society for Photogrammetry and Remote SensingCongress, pages 796–801, 2004.

[13] T. T. Elvins. A survey of algorithms for volume visualization. ACMSIGGRAPH Computer Graphics, 26(3):194–201, 1992.

[14] V. Fanelli and V. M. Ranieri. Mechanisms and clinical consequences of acute

lung injury. Annals of the American Thoracic Society, 12(Supplement

1):S3–S8, 2015.

73

[15] ARDS Definition Task Force. Acute respiratory distress syndrome. Jama,

307(23):2526–2533, 2012.

[16] A. F. Frangi, W. J. Niessen, R. M. Hoogeveen, T. van Walsum, and M. A.

Viergever. Model-based quantitation of 3-D magnetic resonance angiographic

images. IEEE Transactions on Medical Imaging, 18(10):946–956, 1999.

[17] A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale

vessel enhancement filtering. In Medical Image Computing andComputer-Assisted Intervention–MICCAI’98, pages 130–137. Springer, 1998.

[18] H. Frimmel, J. Näppi, and H. Yoshida. Centerline-based colon segmentation for

CT colonography. Medical Physics, 32(8):2665–2672, 2005.

[19] E. Galkina and K. Ley. Immune and inflammatory mechanisms of

atherosclerosis. Annual review of immunology, 27:165–197, 2009.

[20] D. Geiger, A. Gupta, L. A. Costa, and J. Vlontzos. Dynamic programming for

detecting, tracking, and matching deformable contours. IEEE Transactions onPattern Analysis and Machine Intelligence, 17(3):294–302, 1995.

[21] R. W. Gill. Pulsed doppler with B-mode imaging for quantitative blood flow

measurement. Ultrasound in Medicine & Biology, 5(3):223–235, 1979.

[22] O. Glasser. Wilhelm Conrad Röntgen and the early history of the Roentgen rays.

Number 1. Norman Publishing, 1993.

[23] R. Gonzalez and R. Woods. Digital image processing. Pearson Prentice Hall,

2008.

[24] I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. Book in preparation

for MIT Press, 2016.

[25] H. Gray. Anatomy of the human body. Lea & Febiger, 1918.

[26] M. A. Gülsün and H. Tek. Robust vessel tree modeling. In Medical ImageComputing and Computer-Assisted Intervention–MICCAI 2008, pages

602–611. Springer, 2008.

[27] J. C. P. Heggie, N. A. Liddell, and K. P. Maher. Applied imaging technology. St.

Vincent’s Hospital Melbourne, 4th ed. edition, 2001.

[28] S. J. Hessel, D. F. Adams, and H. L. Abrams. Complications of angiography.

Radiology, 138(2):273–281, 1981.

[29] W. R. Hiatt. Medical treatment of peripheral arterial disease and claudication.

The New England Journal of Medicine, 344(21):1608–21, 2001.

[30] E. A. Hoffman. Effect of body orientation on regional lung expansion: a

computed tomographic approach. Journal of Applied Physiology,

59(2):468–480, 1985.

[31] G. N. Hounsfield. Computerized transverse axial scanning (tomography): Part

1. Description of system. The British Journal of Radiology,

46(552):1016–1022, 1973.

[32] D. H. Hubel and T. N. Wiesel. Receptive fields of single neurones in the cat’s

striate cortex. The Journal of physiology, 148(3):574–591, 1959.

[33] D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and

functional architecture in the cat’s visual cortex. The Journal of physiology,

160(1):106–154, 1962.

[34] O. Ishaq. Image Analysis and Deep Learning for Applications in Microscopy.

PhD thesis, Acta Universitatis Upsaliensis, 2016.

[35] C. Kirbas and F. K. H. Quek. A review of vessel extraction techniques and

74

algorithms. ACM Computing Surveys, 36(2):81–121, 2004.

[36] P. A. Krieger. A Visual Analogy Guide to Human Anatomy and Physiology.

Morton Publishing Company, 2013.

[37] Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature,

521(7553):436–444, 2015.

[38] D. Lesage, E. D. Angelini, I. Bloch, and G. Funka-Lea. A review of 3D vessel

lumen segmentation techniques: Models, features and extraction schemes.

Medical Image Analysis, 13(6):819–845, 2009.

[39] F.-F. Li, A. Karpathy, and J. Johnson. Notes to Stanford CS class CS231n:

Convolutional Neural Networks for Visual Recognition, 2017.

[40] R. Li and S. Ourselin. Accurate curvilinear modelling for precise measurements

of tubular structures. In Digital Image Computing: Techniques andApplications; Proceedings of the 7th Biennial Australian Pattern RecognitionSociety Conference, DICTA 2003, page 243, 2003.

[41] J. Lindblad and N. Sladoje. Coverage segmentation based on linear unmixing

and minimization of perimeter and boundary thickness. Pattern RecognitionLetters, 33(6):728 – 738, 2012.

[42] T. Majtner, K. Lidayová, S. Yildirim-Yayilgan, and J. Y. Hardeberg. Improving

skin lesion segmentation in dermoscopic images by thin artefacts removal

methods. In Visual Information Processing (EUVIP), 2016 6th EuropeanWorkshop on, pages 1–6. IEEE, 2016.

[43] R. Malladi, J. A. Sethian, and B. C. Vemuri. Topology-independent shape

modeling scheme. In SPIE’s 1993 International Symposium on Optics,Imaging, and Instrumentation, pages 246–258. International Society for Optics

and Photonics, 1993.

[44] S. S. Mao, N. Ahmadi, B. Shah, D. Beckmann, A. Chen, L. Ngo, F. R. Flores,

Y. L. Gao, and M. J. Budoff. Normal thoracic aorta diameter on cardiac

computed tomography in healthy asymptomatic adults: Impact of age and

gender. Academic Radiology, 15(7):827–834, 2008.

[45] C. T. Metz, M. Schaap, A. C. Weustink, N. R. Mollet, T. van Walsum, and W. J.

Niessen. Coronary centerline extraction from CT coronary angiography images

using a minimum cost path approach. Medical Physics, 36(12):5568–5579,

2009.

[46] J. Mille and L. D. Cohen. Deformable tree models for 2D and 3D branching

structures extraction. In 2009 IEEE Computer Society Conference on ComputerVision and Pattern Recognition Workshops, pages 149–156, 2009.

[47] M. Miyazaki and V. S. Lee. Nonenhanced MR angiography. Radiology,

248(1):20–43, 2008.

[48] A. Morales Pinzón. Lung Segmentation and Airway Tree Matching: Applicationto Aeration Quantification in CT Images of Subjects with ARDS. PhD thesis,

Université de Lyon, 2016.

[49] K. Mori, J. Hasegawa, J. Toriwaki, H. Anno, and K. Katada. Recognition of

bronchus in three-dimensional X-ray CT images with applications to virtualized

bronchoscopy system. In Proceedings of the 13th International Conference onPattern Recognition, volume 3, pages 528–532. IEEE, 1996.

[50] G. Németh, P. Kardos, and K. Palágyi. Thinning combined with

iteration-by-iteration smoothing for 3D binary images. Graphical Models,

75

73(6):335–345, 2011.

[51] M. A. Nielsen. Neural networks and deep learning. URL:http://neuralnetworksanddeeplearning. com/.(visited: 30.03. 2017), 2015.

[52] I. Nyström. On Quantitative Shape Analysis of Digital Volume Images. PhD

thesis, Acta Universitatis Upsaliensis, 1997.

[53] J. W. Olin and B. A. Sealove. Peripheral artery disease: current insight into the

disease and its diagnosis and management. Mayo Clinic Proceedings,

85(7):678–92, 2010.

[54] World Health Organization. World Health Statistics 2016: Monitoring Healthfor the Sustainable Development Goals (SDGs). World Health Organization,

2016.

[55] S. Osher and J. A. Sethian. Fronts propagating with curvature-dependent speed:

algorithms based on Hamilton-Jacobi formulations. Journal of ComputationalPhysics, 79(1):12–49, 1988.

[56] K. Palágyi and A. Kuba. A parallel 3D 12-subiteration thinning algorithm.

Graphical Models and Image Processing, 61(4):199–221, 1999.

[57] P. Pignoli, E. Tremoli, A. Poli, P. Oreste, and R. Paoletti. Intimal plus medial

thickness of the arterial wall: a direct measurement with ultrasound imaging.

Circulation, 74(6):1399–406, 1986.

[58] B. Preim and D. Bartz. Visualization in medicine: theory, algorithms, andapplications. Morgan Kaufmann, 2007.

[59] B. Preim and Ch. P. Botha. Visual Computing for Medicine: Theory,Algorithms, and Applications. Newnes, 2013.

[60] M. R. Prince, E. K. Yucel, J. A. Kaufman, D. C. Harrison, and S. C. Geller.

Dynamic gadolinium-enhanced three-dimensional abdominal MR

arteriography. Journal of Magnetic Resonance Imaging, 3(6):877–881, 1993.

[61] J.-Ch. Richard, C. Pouzot, A. Morales Pinzón, J. S. T. González, M. Orkisz,

B. Neyran, M. Hernández Hoyos, F. Lavenne, and C. Guerin. Reliability of the

nitrogen washin-washout technique to assess end-expiratory lung volume at

variable PEEP and tidal volumes. Intensive care medicine experimental,2(1):1–14, 2014.

[62] G. D. Rubin, P. J. Walker, M. D. Dake, S. Napel, R. B. Jeffrey, C. H. McDonnell,

R. S. Mitchell, and D. C. Miller. Three-dimensional spiral computed

tomographic angiography: an alternative imaging modality for the abdominal

aorta and its branches. Journal of Vascular Surgery, 18(4):656–64, 1993.

[63] P. K. Saha, G. Borgefors, and G. Sanniti di Baja. A survey on skeletonization

algorithms and their applications. Pattern Recognition Letters, 76:3–12, 2016.

[64] P. K. Saha, B. B. Chaudhuri, and D. D. Majumder. A new shape preserving

parallel thinning algorithm for 3D digital images. Pattern Recognition,

30(12):1939–1955, 1997.

[65] P. K. Sahoo, S. Soltani, and A. K. C. Wong. A survey of thresholding

techniques. Computer Vision, Graphics, and Image Processing, 41(2):233–260,

1988.

[66] G. Sapiro and A. Tannenbaum. Affine invariant scale-space. InternationalJournal of Computer Vision, 11(1):25–44, 1993.

[67] M. Schaap, L. Neefjes, C. Metz, A. van der Giessen, A. Weustink, N. Mollet,

J. Wentzel, T. van Walsum, and W. Niessen. Coronary lumen segmentation

76

using graph cuts and robust kernel regression. In International Conference onInformation Processing in Medical Imaging, pages 528–539. Springer, 2009.

[68] J. Schmidhuber. Deep learning in neural networks: An overview. NeuralNetworks, 61:85–117, 2015.

[69] K. Siddiqi and S. Pizer. Medial representations: mathematics, algorithms andapplications, volume 37. Springer Science & Business Media, 2008.

[70] N. Sladoje and J. Lindblad. High-precision boundary length estimation by

utilizing gray-level information. IEEE Transactions on Pattern Analysis andMachine Intelligence, 31(2):357–363, 2009.

[71] N. Sladoje and J. Lindblad. Pixel coverage segmentation for improved feature

estimation. In Proceedings of the 15th International Conference on ImageAnalysis and Processing, ICIAP ’09, pages 929–938. Springer-Verlag, 2009.

[72] N. Sladoje and J. Lindblad. The coverage model and its use in image

processing. In Selected Topics on Image Processing and Cryptology, ZbornikRadova (Collection of Papers), volume 15, pages 39–117. Mathematical

Institute of the Serbian Academy of Sciences and Arts, Belgrade, 2012.

[73] M. Sonka, V. Hlavac, and R. Boyle. Image processing, analysis, and machinevision. Cengage Learning, 2014.

[74] H. C. Stary. Natural history and histological classification of atherosclerotic

lesions: an update. Arteriosclerosis, Thrombosis, and Vascular Biology,

20(5):1177–8, 2000.

[75] J. S. Suri, K. Liu, L. Reden, and S. Laxminarayan. A review on MR vascular

image processing: skeleton versus nonskeleton approaches: Part II. IEEETransactions on Information Technology in Biomedicine: a publication of theIEEE Engineering in Medicine and Biology Society, 6(4):338–350, 2002.

[76] G. J. Tortora and B. Derrickson. Introduction to the human body: the essentialsof anatomy and physiology. Wiley, 2011.

[77] C. M. van Bemmel, O. Wink, B. Verdonck, M. A. Viergever, and W. J. Niessen.

Blood pool contrast-enhanced MRA: improved arterial visualization in the

steady state. IEEE Transactions on Medical Imaging, 22(5):645–652, 2003.

[78] W. J. van Rooij, M. E. Sprengers, A. N. de Gast, J. P. Peluso, and M. Sluzewski.

3D rotational angiography: the new gold standard in the detection of additional

intracranial aneurysms. American Journal of Neuroradiology, 29(5):976–9,

2008.

[79] Ch. Wang, R. Moreno, and Ö. Smedby. Vessel segmentation using implicit

model-guided level sets. In Proceedings of MICCAI Workshop 3DCardiovascular Imaging: a MICCAI segmentation Challenge – Medical ImageComputing and Computer-Assisted Intervention. 2012.

[80] Ch. Wang and Ö. Smedby. Integrating automatic and interactive methods for

coronary artery segmentation: let the PACS workstation think ahead.

International Journal of Computer Assisted Radiology and Surgery,

5(3):275–85, 2010.

[81] Ch. Xu and J. L. Prince. Snakes, shapes, and gradient vector flow. IEEETransactions on Image Processing, 7(3):359–369, 1998.

[82] L. A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, 1965.

77

Errata

Paper IV

Coverage Segmentation of Thin Structures by Linear Unmixing and LocalCentre of Gravity Attraction

II. Background• Original text “where Sk ⊂ R

2 is” with character S in calligraphic

font should be replaced by “where Sk ⊂ R2 is” with character S in

ordinary font.

• Original text “The continuous sets Sk are” with character S in cal-

ligraphic font should be replaced “The continuous sets Sk are” with

character S in ordinary font.

Paper V

Coverage Segmentation of 3D Thin Structures

II. Background• Original text “where Sk ⊂ R

3 is” with character S in calligraphic

font should be replaced by “where Sk ⊂ R3 is” with character S in

ordinary font.

• Original text The continuous sets Sk are” with character S in cal-

ligraphic font should be replaced by “The continuous sets Sk are”

with character S in ordinary font.

79

Acta Universitatis UpsaliensisDigital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1496

Editor: The Dean of the Faculty of Science and Technology

A doctoral dissertation from the Faculty of Science andTechnology, Uppsala University, is usually a summary of anumber of papers. A few copies of the complete dissertationare kept at major Swedish research libraries, while thesummary alone is distributed internationally throughthe series Digital Comprehensive Summaries of UppsalaDissertations from the Faculty of Science and Technology.(Prior to January, 2005, the series was published under thetitle “Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology”.)

Distribution: publications.uu.seurn:nbn:se:uu:diva-318796

ACTAUNIVERSITATIS

UPSALIENSISUPPSALA

2017