automatic detection of pulmonary nodules in lung ct images

114
School of Information and Mechatronics Signal and Image Processing Laboratory Wook-Jin Choi

Upload: wookjin-choi

Post on 02-Jun-2015

269 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: automatic detection of pulmonary nodules in lung ct images

School of Information and Mechatronics

Signal and Image Processing Laboratory

Wook-Jin Choi

Page 2: automatic detection of pulmonary nodules in lung ct images

• Introduction

• Lung Volume Segmentation

• Genetic Programming based Classifier

• Hierarchical Block-image Analysis

• Shape-based Feature Descriptor

• Experimental Results

• Conclusions

2

Page 3: automatic detection of pulmonary nodules in lung ct images

3

Page 4: automatic detection of pulmonary nodules in lung ct images

• Lung cancer is the leading cause of cancer deaths.

• Most patients diagnosed with lung cancer already have advanced disease

– 40% are stage IV and 30% are III

– The current five-year survival rate is only 16%

• Defective nodules are detected at an early stage

– The survival rate can be increased

4

Page 5: automatic detection of pulmonary nodules in lung ct images

5

(a) male (b) female

Trends in death rates for selected cancers, United States, 1930-2008

Page 6: automatic detection of pulmonary nodules in lung ct images

• Early detection of lung nodules is extremely important for the diagnosis and clinical management of lung cancer

• Lung cancer had been commonly detected and diagnosed on chest radiography

• Since the early 1990s CT has been reported to improve detection and characterization of pulmonary nodules

6

Page 7: automatic detection of pulmonary nodules in lung ct images

• CT was introduced in 1971 – Sir Godfrey Hounsfield, United Kingdom

• CT utilize computer-processed X-rays – to produce tomographic images or 'slices' of specific

areas of the body

• The Hounsfield unit (HU) scale is a linear transformation of the original linear attenuation coefficient measurement into one in which the radio density of distilled water

7

water

waterx1000

HU

Page 8: automatic detection of pulmonary nodules in lung ct images

8

The HU of common substances

Substance HU

Air −1000

Lung −500

Fat −84

Water 0

Cerebrospinal Fluid 15

Blood +30 to +45

Muscle +40

Soft Tissue, Contrast Agent +100 to +300

Bone +700(cancellous bone)to +3000 (dense bone)

Nodule

Page 9: automatic detection of pulmonary nodules in lung ct images

• Lung cancer screening is currently implemented using low-dose CT examinations

• Advanced in CT technology

– Rapid image acquisition with thinner image sections

– Reduced motion artifacts and improved spatial resolution

• The typical examination generates large-volume data sets

• These large data sets must be evaluated by a radiologist

– A fatiguing process

9

Page 10: automatic detection of pulmonary nodules in lung ct images

• The use of pulmonary nodule detection CAD system can provide an effective solution

• CAD system can assist radiologists by increasing efficiency and potentially improving nodule detection

10

General structure of pulmonary nodule detection system

Page 11: automatic detection of pulmonary nodules in lung ct images

CAD systems Lung segmentation Nodule Candidate Detection False Positive Reduction

Suzuki et al.(2003)[26] Thresholding Multiple thresholding MTANN

Rubin et al.(2005)[27] Thresholding Surface normal overlap Lantern transform and rule-ba

sed classifier

Dehmeshki et al.(2007)[28] Adaptive thresholding Shape-based GATM Rule-based filtering

Suarez-Cuenca et al.(2009)[29] Thresholding and 3-D connec

ted component labeling 3-D iris filtering

Multiple rule-based LDA classi

fier

Golosio et al.(2009)[30] Isosurface-triangulation Multiple thresholding Neural network

Ye et al.(2009)[31] 3-D adaptive fuzzy segmenta

tion Shape based detection

Rule-based filtering and weig

hted SVM classifier

Sousa et al.(2010)[32] Region growing Structure extraction SVM classifier

Messay et al.(2010)[33] Thresholding and 3-D connec

ted component labeling

Multiple thresholding and mo

rphological opening

Fisher linear discriminant and

quadratic classifier

Riccardi et al.(2011)[34] Iterative thresholding 3-D fast radial filtering and sc

ale space analysis

Zernike MIP classification bas

ed on SVM

Cascio et al.(2012)[35] Region growing Mass-spring model Double-threshold cut and neu

ral network

11

Page 12: automatic detection of pulmonary nodules in lung ct images

• To evaluate the performance of the proposed method, Lung Image

Database Consortium (LIDC) database is applied

• LIDC database, National Cancer Institute (NCI), United States

– The LIDC is developing a publicly available database of thoracic

computed tomography (CT) scans as a medical imaging research

resource to promote the development of computer-aided

detection or characterization of pulmonary nodules

• The database consists of 84 CT scans

– 100-400 Digital Imaging and Communication (DICOM) images

– An XML data file containing the physician annotations of nodules

– 148 nodules

– The pixel size in the database ranged from 0.5 to 0.76 mm

– The reconstruction interval ranged from 1 to 3mm

12

Page 13: automatic detection of pulmonary nodules in lung ct images

13

Page 14: automatic detection of pulmonary nodules in lung ct images

• Thresholding – Fixed threshold

– Optimal threshold

– 3-D adaptive fuzzy thresholding

• Lung region extraction – 3-D connectivity with seed point

– 3-D connected component labeling

• Contour correction – Morphological dilation

– Rolling ball algorithm

– Chain code representation

14

Page 15: automatic detection of pulmonary nodules in lung ct images

15

• Air has an attenuation of -1000 HU

• Most lung tissue is in the range of -910 HU to -500 HU

• The chest wall, blood vessel, and bone are above -500 HU

• The low and high intensities are differentiable around the intensity -500 HU

( , , ) ( , , ) 500i x y z I x y z HUS

Page 16: automatic detection of pulmonary nodules in lung ct images

16

Input CT images, their intensity histograms, and thresholded images

Page 17: automatic detection of pulmonary nodules in lung ct images

• A fixed threshold is applicable to segment lung area – The intensity ranges of images are varied by different

acquisition protocols

• To obtain optimal threshold – Iterative approach continues until the threshold

converges

– The initial threshold :

– is i th threshold and new threshold as

17

(0) 500T HU

( 1)

2

i o bT

( )iT

Page 18: automatic detection of pulmonary nodules in lung ct images

18

Input CT images, their intensity histograms, and thresholded images

Page 19: automatic detection of pulmonary nodules in lung ct images

• White areas – non-body voxels

– including lung cavity

• Black areas – body voxels

– excluding lung region

• Lung regions are extracted from the non-body voxels by using 3-D connected component labeling

19

18-connectivity voxels

Page 20: automatic detection of pulmonary nodules in lung ct images

20

Labeled images after applying 3-D connected component labeling

Page 21: automatic detection of pulmonary nodules in lung ct images

• To extract lung volume

– Remove rim attached to boundaries of image

– The first and the second largest volumes are

selected as the lung region

• The lung region contains small holes

– To remove these holes

– Morphological hole filling operations are applied

21

|lung first secondS l l

Page 22: automatic detection of pulmonary nodules in lung ct images

22

Binary images of the selected lung region

Lung mask images after hole filling

Page 23: automatic detection of pulmonary nodules in lung ct images

• The contour of the lung volume is needed to

correct

– To include wall side nodule (juxta-pleural nodule)

23

Extracted lung region using 3D connected component labeling and contour

corrected lung region (containing wall side nodule)

Page 24: automatic detection of pulmonary nodules in lung ct images

24

Contour correction using chain-code representation

Page 25: automatic detection of pulmonary nodules in lung ct images

25

Page 26: automatic detection of pulmonary nodules in lung ct images

26

Page 27: automatic detection of pulmonary nodules in lung ct images

27

Page 28: automatic detection of pulmonary nodules in lung ct images

28

Page 29: automatic detection of pulmonary nodules in lung ct images

29

Page 30: automatic detection of pulmonary nodules in lung ct images

• Detection of nodule candidates is important

• The performance of nodule detection system relies on the accuracy of candidate detection

• ROI extraction – Optimal multi-thresholding

• Nodule candidates detection and segmentation – Rule-based pruning

30

Page 31: automatic detection of pulmonary nodules in lung ct images

• The traditional multi-thresholding method

needs many steps of grey levels

• An iterative approach is applied to select

the threshold value

• The optimal threshold value is calculated

on median slice of lung CT scan

31

( 1)

2

i o bT

Page 32: automatic detection of pulmonary nodules in lung ct images

• The optimal threshold value

– A base threshold for multi-thresholding

• Additional six threshold values are obtained

– Base threshold + 400,+ 300,+ 200,+ 100, - 100,

and - 200

32

Page 33: automatic detection of pulmonary nodules in lung ct images

• Rule based classifier removes vessels and noise

• Vessel removing

– Volume is extremely bigger than nodule

– Elongated object

• Noise removing

– Radius of ROI is smaller than 3mm

– Bigger than 30mm

• Remaining ROIs are nodule candidates

33

Page 34: automatic detection of pulmonary nodules in lung ct images

34

Rule Description

R1 Small noise

R2 Vessel

R3 Large noise

R4 Nodule

Pruning rules for nodule candidate detection

Page 35: automatic detection of pulmonary nodules in lung ct images

35

Page 36: automatic detection of pulmonary nodules in lung ct images

36

(d) (e) (f)

The results of nodule candidate detection: (a,d) ROIs, (b,e) vessel, and (c,f) nodule

candidates after rule-based pruning

(a) (b) (c)

Page 37: automatic detection of pulmonary nodules in lung ct images

• The features are useful information that describe characteristics of the nodule candidates

• In the proposed CAD system, these features will be used to train the GPC

• The proposed feature extraction process consists of two stages – The variety types of features are extracted from

the nodule candidates

– Subsets of features are selected and combined into sub-groups

37

Page 38: automatic detection of pulmonary nodules in lung ct images

38

Index Feature Index Feature

2-D geometric features Mean inside

Area Mean outside

Diameter Variance inside

Perimeter Skewness inside

Circularity Kurtosis inside

3-D geometric features Eigenvalues

Volume 3-D intensity based statistical features

Compactness Minimum value inside

Bounding Box Dimensions Mean inside

Principal Axis Length Mean outside

Elongation Variance inside

2-D intensity based statistical features Skewness inside

Minimum value inside Kurtosis inside

1f

2f

3f

4f

5f

6f

97 ~ ff

1210 ~ ff

13f

14f

15f

16f

17f

18f

19f

2720 ~ ff

28f

29f

30f

31f

32f

33f

Features for nodule detection

Page 39: automatic detection of pulmonary nodules in lung ct images

Feature vector Description

2-D geometric features

3-D geometric features

2-D intensity-based statistical features

3-D intensity-based statistical features

2-D features

3-D features

Geometric features

Intensity-based statistical features

All features

39

1 1 4{ ,..., }f ff

2 5 13{ ,..., }f ff

43 71 2{ ,..., }f ff

4 28 33{ ,..., }f ff

5 1 3f f f

6 2 4f f f

7 1 2f f f

8 3 4f f f

1 2 3 4f f f f f

Eight different groups of feature vectors

Page 40: automatic detection of pulmonary nodules in lung ct images

• Genetic Programming (GP) – An evolutionary

optimization technique

• The basic structure of GP is very similar to Genetic Algorithm(GA)

• The chromosome – GA : variable (binary digit

or string)

– GP : program (tree or graph)

40

A function represented as a tree structure

Page 41: automatic detection of pulmonary nodules in lung ct images

• GP chromosome – The terminal set

• The elements of feature vector extracted from nodule candidate images

• Randomly generated constants with in the range 0,1

– The function set • Four standard arithmetic operator namely plus, minus,

multiply and division

• Additional mathematical operators log, exp, abs, sin and cos

• All operators in the function set are protected to avoid exception

• GP evolves combination of the terminal set and function set

41

Page 42: automatic detection of pulmonary nodules in lung ct images

• Fitness Function – evaluate every individuals in GP generation

• True positive rate (TPR)

• Specificity (SPC) – SPC is the value subtracted from 1 to FPR and also called true negative

rate (TNR)

• Area under the ROC curve (AUC) – ROC curve is plotted between TP and FP for different threshold values

– AUC is area under the ROC curve and a good measure of classifier performance in different condition

42

TPTPR

TP FN

1 1TN FP

SPC FPRTN FP FP TN

f TPR FPR AUC

Page 43: automatic detection of pulmonary nodules in lung ct images

43

Objective To evolve a optimum classifier with a maximum TPR, SPC and AUC

Function Set +,-,*,protected division, log, exp, abs, sin and cos

Terminal Set Elements of a feature vector and randomly generated constants

Fitness Fit(B)=TPR×SPC×AUC

Selection Generational

Wrapper Positive if , else negative

Population Size 300

Generation Size 80

Initial Tree Depth Limit 6

Initial population Ramped half and half

GP Operators prob. Variable ratio of crossover mutation is used

Sampling Tournament

Survival mechanism Keep the best individuals

Real max. tree level 30

Page 44: automatic detection of pulmonary nodules in lung ct images

44

Flow chart for training the proposed GPC

Page 45: automatic detection of pulmonary nodules in lung ct images

45

Feature spaces for four types of features

2-D geometric feature 3-D geometric feature

2-D intensity-based statistical feature 3-D intensity-based statistical feature

Page 46: automatic detection of pulmonary nodules in lung ct images

• Examples of GPC expression

– log(log(log(times(log(f_{20}),times(abs(log(log(times( times(f_{5},l

og(f_{31})),log(abs(log(log(log(times(log(f_{9}),log(f_{31}))))))))))),lo

g(times(times(log(f_{5}),log(log( times(times(f_{5},log(log(f_{5}))),ti

mes(times(f_{5}, log(f_{9})),log(f_{9})))))),log(f_{31}))))))))

– plus(plus(plus(plus(plus(plus(f_{4},log(times(f_{11},plus(log(plus(f_

{9},plus(log(f_{11}),f_{4}))),f_{4})))),f_{4}),plus(log(plus(sin(log(abs(ti

mes(f_{11},plus(log(f_{4}),f_{4}))))),f_{4})),f_{4})),log(log(log(times(f_

{4},abs(f_{2})))))),log(plus(log(f_{10}),times(f_{1},abs(log(log(times(f

_{10},abs(f_{9}))))))))),log(log(times(log(log(times(f_{11},plus(log(ti

mes(log(log(times(f_{11},plus(log(f_{4}),

f_{4})))),f_{1})),f_{4})))),f_{1}))))

46

Page 47: automatic detection of pulmonary nodules in lung ct images

47

Tree representation of the GPC expression

Page 48: automatic detection of pulmonary nodules in lung ct images

48

Transformed features and classification threshold generated using a GPC

* Nodule

+ Non-nodule

Page 49: automatic detection of pulmonary nodules in lung ct images

49

Training Performance on the training set Performance on testing set

Feature set Fitness Accuracy Sensitivity Specificity Accuracy Sensitivity SPC

0.979 99.1% 98.1% 100.0% 78.0% 70.0% 86.0%

0.954 97.8% 95.6% 100.0% 80.5% 86.5% 74.5%

0.859 94.4% 90.6% 98.1% 74.3% 73.0% 75.7%

0.741 90.9% 91.9% 90.0% 61.3% 64.2% 58.3%

0.972 98.8% 98.1% 99.4% 82.3% 82.3% 82.3%

0.951 98.1% 98.1% 98.1% 84.0% 90.7% 77.3%

0.986 99.4% 98.8% 100.0% 86.3% 87.3% 85.3%

0.858 94.7% 93.1% 96.3% 74.3% 74.2% 74.3%

0.988 99.4% 100.0% 98.8% 83.8% 89.2% 78.5%

0.026 1.3% 0.0% 2.6% 4.8% 6.0% 7.9%

Min 0.938 96.9% 100.0% 93.8% 76.7% 76.7% 66.7%

Max 1.000 100.0% 100.0% 100.0% 89.2% 98.3% 90.0%

1f

2f

3f

4f

5f

6f

7f

8ff

GPC results for different feature vectors using a 20–80 dataset

Page 50: automatic detection of pulmonary nodules in lung ct images

50

Training Performance on the training set Performance on the testing set

Feature set Fitness Accuracy Sensitivity Specificity Accuracy Sensitivity SPC

0.876 94.9% 95.3% 94.5% 80.8% 73.7% 87.9%

0.865 94.5% 90.0% 98.9% 86.1% 85.8% 86.3%

0.764 90.9% 88.4% 93.4% 78.9% 75.5% 82.4%

0.628 85.5% 87.1% 83.9% 70.0% 74.5% 65.5%

0.925 96.8% 96.1% 97.6% 88.9% 89.7% 88.2%

0.907 96.2% 93.9% 98.4% 85.7% 85.5% 85.8%

0.940 97.5% 96.8% 98.2% 85.5% 88.7% 82.4%

0.751 90.1% 88.7% 91.6% 80.8% 81.3% 80.3%

0.919 96.7% 95.1% 98.4% 92.3% 94.0% 90.7%

0.028 1.0% 2.0% 1.1% 5.2% 8.0% 5.6%

Min 0.855 94.3% 90.2% 96.7% 83.3% 80.0% 80.0%

Max 0.943 97.5% 96.7% 100.0% 96.7% 100.0% 100.0%

1f

2f

3f

4f

5f

6f

7f

8ff

GPC results for different feature vectors using a 50–50 dataset.

Page 51: automatic detection of pulmonary nodules in lung ct images

Training Performance on the training set Performance on the testing set

Feature set Fitness Accuracy Sensitivity Specificity Accuracy Sensitivity SPC

0.874 95.0% 93.3% 96.7% 88.3% 88.0% 88.7%

0.890 95.4% 93.3% 97.5% 87.3% 86.0% 88.7%

0.709 89.1% 85.2% 93.0% 81.0% 81.3% 80.7%

0.557 82.0% 87.7% 76.4% 69.3% 78.7% 60.0%

0.872 94.9% 93.3% 96.6% 90.0% 92.0% 88.0%

0.855 94.2% 92.0% 96.4% 88.7% 87.3% 90.0%

0.923 96.8% 96.1% 97.5% 89.3% 89.3% 89.3%

0.723 89.4% 86.9% 92.0% 83.0% 78.7% 87.3%

0.889 95.5% 93.6% 97.4% 89.0% 96.0% 82.0%

0.049 1.8% 3.7% 2.1% 5.2% 4.7% 11.4%

Min 0.829 93.4% 88.5% 91.8% 80.0% 86.7% 60.0%

Max 0.945 97.5% 98.4% 98.4% 96.7% 100.0% 100.0%

51

1f

2f

3f

4f

5f

6f

7f

8ff

GPC results for different feature vectors using a 80–20 dataset.

Page 52: automatic detection of pulmonary nodules in lung ct images

52

Page 53: automatic detection of pulmonary nodules in lung ct images

53

Page 54: automatic detection of pulmonary nodules in lung ct images

• Coarse to fine hierarchical

block-image analysis

– Block size : 32, 24, 16, 12, 8

• 3-D CT scan is split into 3-

D block-images

• The non-informative

block-images are filtered

out by using entropy

analysis

54

Page 55: automatic detection of pulmonary nodules in lung ct images

55

Result images after block splitting with respect to various block sizes

Page 56: automatic detection of pulmonary nodules in lung ct images

• Calculate the entropy H(x) on block image

• Select informative blocks by using entropy

56

1

2 2

1

1( ) ( ) log ( ) log ( )

( )

n n

i i

H x p i p i p ip i

Page 57: automatic detection of pulmonary nodules in lung ct images

57

The entropy histograms of block-images for five different block sizes

(x-axis : entropy value, y-axis : number of blocks, (a) 32, (b) 24, (c) 16, (d) 12, and (e) 8)

Page 58: automatic detection of pulmonary nodules in lung ct images

• The selected block-

image is enhanced

• The object in the

selected block-image

is segmented

• The location of block

image is adjusted

58

Page 59: automatic detection of pulmonary nodules in lung ct images

59

• Block-image enhancement is presented for more accurate analysis

• 3-D coherence-enhancing diffusion (CED) filter – Hessian matrix based

– Preserve small spherical structure (nodule)

– Enhance tubular structure (vessel)

(a) Input image and (b) the result image after enhancement

Page 60: automatic detection of pulmonary nodules in lung ct images

• Optimal threshold

– Iterative approach

– Initial threshold : -500HU

– Threshold converges, and optimal threshold

obtained

60

Page 61: automatic detection of pulmonary nodules in lung ct images

• The location of block-image should be adjusted – The segmented object is not located in the center

of the block

• Block location is iteratively updated by using centroid of the segmented object

• The iteration of the adjustment continues until the center position converges

• Or distance between the adjusted location and the original location is larger than half of the block size

61

Page 62: automatic detection of pulmonary nodules in lung ct images

62

Iterations of automatic block location adjustment, upper: 3-D shapes, lower: the

median slices of 3-D block; (a) the first; (b) the fifth; and (c) the last iterations of

adjustment

Page 63: automatic detection of pulmonary nodules in lung ct images

63

• Three different types of features are

extracted from nodule candidate block-

images

• Nodule has their own shapes

– Important characteristics to distinguish

• 2-D and 3-D geometric features describe

the shape of nodule candidates

Page 64: automatic detection of pulmonary nodules in lung ct images

64

Features for nodule detection

Page 65: automatic detection of pulmonary nodules in lung ct images

• Support vector machine (SVM)

– SVM is a useful technique for data

classification

– Supervised learning models with associated

learning algorithms

– SVM analyze data and recognize patterns

– Classification and regression analysis

65

Page 66: automatic detection of pulmonary nodules in lung ct images

• The basic SVM takes a set of input data and predicts two possible classes for each given input

• Training dataset

• The SVM requires the solution of the following optimization problem

66

Page 67: automatic detection of pulmonary nodules in lung ct images

• SVM can efficiently perform non-linear classification using the kernel trick

• Kernel function

– Polynomial function

– Radial basis function

– Minkowski distance function

67

Page 68: automatic detection of pulmonary nodules in lung ct images

• k-fold cross-validation is applied to

evaluated the proposed classifier

• Performance validation measure

– The number of true positives (TPs) and false

positives (FPs)

– Accuracy, sensitivity, specificity, and area

under the ROC curve.

68

Page 69: automatic detection of pulmonary nodules in lung ct images

69

k p AUC Accuracy Sensitivity Specificity

5 0.25 0.9738 91.52% 87.16% 95.88%

7 0.25 0.9784 93.97% 91.02% 96.92%

10 0.25 0.9736 92.43% 88.97% 95.88%

The k-fold cross validation results of SVM classifiers with radial basis

function kernel for different k values

Page 70: automatic detection of pulmonary nodules in lung ct images

70

p AUC Accuracy Sensitivity Specificity

SVM-r 0.1 0.9727 84.72% 69.44% 100.00%

0.125 0.9746 88.96% 78.70% 99.23%

0.25 0.9784 93.97% 91.02% 96.92%

0.5 0.9754 92.82% 91.54% 94.10%

1 0.9712 91.79% 91.53% 92.05%

2 0.9673 92.30% 93.08% 91.53%

SVM-p 0.1 0.4660 47.40% 0.00% 94.81%

0.125 0.4632 44.81% 0.26% 89.35%

0.25 0.6876 68.26% 86.13% 50.39%

0.5 0.9462 89.85% 91.52% 88.18%

1 0.9463 90.74% 92.78% 88.69%

2 0.9646 92.29% 91.25% 93.32%

SVM-m 0.1 0.8706 82.55% 86.91% 78.19%

0.125 0.7051 69.71% 78.46% 60.95%

0.25 0.5706 60.68% 68.69% 52.68%

0.5 0.5469 59.02% 66.63% 51.41%

1 0.5420 58.11% 66.11% 50.12%

2 0.5527 57.60% 65.85% 49.36%

The 7-fold cross validation results of SVM classifiers with three different kernel functions, SVM-r: radial basis function,

SVM-p: polynomial function, and SVM-m: Minkowski distance function

Page 71: automatic detection of pulmonary nodules in lung ct images

71

ROC curves of the SVM classifiers with respect to three different kernel functions,

SVM-r: radial basis function, SVM-p: polynomial function, and SVM-m: Minkowski

distance function; (a) p = 0:25 and (b) p = 1.

Page 72: automatic detection of pulmonary nodules in lung ct images

72

Page 73: automatic detection of pulmonary nodules in lung ct images

73

Page 74: automatic detection of pulmonary nodules in lung ct images

• Eigenvalue decomposition

of Hessian Matrix

– Dot enhancement filter

– Feature extraction

• Multi-scale dot

enhancement filter

– Enhance the nodules

– The shape of nodules is like

dot or ball

74

Page 75: automatic detection of pulmonary nodules in lung ct images

75

• Multi-scale dot enhancement filter is

based on eigenvalue of Hessian matrix

• Hessian Matrix

• Local structure information is obtained by

Hessian matrix

Page 76: automatic detection of pulmonary nodules in lung ct images

• Eigenvalue decomposition of Hessian Matrix

– Structure information : surface-ness, curve-ness, and point-

ness

– This information is expressed in the three singular tensors (stick, plate, and ball)

• Tensor based representation

76

Page 77: automatic detection of pulmonary nodules in lung ct images

• Stick tensor

• Plate tensor

• Ball tensor

• Surface-ness : saliency , orientation

• Curve-ness : saliency , orientation

• Point-ness : saliency , no orientation

77

Page 78: automatic detection of pulmonary nodules in lung ct images

78

Page 79: automatic detection of pulmonary nodules in lung ct images

• The dot enhancement filter is applied to enhance the spherical object, such as nodule

• For each voxel, the dot value is defined as

• are three eigenvalues from the Hessian matrix

• Gaussian image smoothing with a variety scales is performed prior to the calculation of the gradient for different size of nodules and reducing noise

79

Page 80: automatic detection of pulmonary nodules in lung ct images

• Assuming that the diameter of nodule to be detected are in a range the N discrete smoothing scales ___ in the range of can be calculated as

where and each scale has corresponding nodule diameter

• The maximum dot value calculated among the different smoothing scales

• Five steps smoothing scales are used in the range of nodule diameter [3mm, 30mm]

80

(1/( 1))

1 0( / ) Nr d d

Page 81: automatic detection of pulmonary nodules in lung ct images

• The image block is extracted as a potential nodule candidate

– The dot values are larger than predefined threshold

• The dimension of the image block is

• It is noted that the size of the image block is considered at the relation to the corresponding smoothing scale as follows:

where the braces indicate the ceiling function

81

Page 82: automatic detection of pulmonary nodules in lung ct images

• A novel shape-based feature extraction method is proposed

• Angular Histogram of Surface Normal Feature

• The feature extraction has important role in the pulmonary nodule CAD system

• The detected nodule candidates are considered as nodules or non-nodules using the extracted feature information

82

Page 83: automatic detection of pulmonary nodules in lung ct images

• Popular approach in the last decade for 2-D images

• The scale invariant feature transform (SIFT) – It can extract salient points and feature descriptors in the

most invariant way with respect to scaling, translation, orientation, affine changes and illumination within images

– The SIFT is designed and tested on 2-D images of 3-D object.

– Allaire et al. proposed fully orientation invariant 3-D SIFT

• The histograms of oriented gradients (HOG) – Describing salient points on 2-D images of 3-D objects

– Scherer et al. proposed the 3-D extension of HOG is proposed for 3-D object retrieval

83

Page 84: automatic detection of pulmonary nodules in lung ct images

• The shape-based feature descriptor is extracted for small 3-D object in image patch

• The AHSN feature extraction method is proposed to analyze the shape of the target object

• The eigenvalue decomposition of the Hessian matrix is applied to every voxels for target image

• The histograms are obtained on surface-ness information

– surface saliency :

– surface normal vector :

84

Page 85: automatic detection of pulmonary nodules in lung ct images

• The orientation of surface normal vector is obtained prior to calculate AHSN feature based on the eigenvalue decomposition of the Hessian matrix

• The orientation of surface normal vector is represented as two kinds of orientation in spherical coordination

85

Page 86: automatic detection of pulmonary nodules in lung ct images

• Two angular histograms are constructed – The orientation θ histogram with n bins is formed

• Each bin covering 180/n degrees

• Each sample in the image block added to a histogram bin is weighted by its surface-ness saliency and normalized by total sum of surface-ness saliency

– The orientation φ is quantized into n bins • Each bin covering 360/n degrees

• Each sample in the image block added to a histogram bin is weighted and normalized

– The dimension of feature descriptor is 2n

– The extracted AHSN feature is scale-invariant

86

Page 87: automatic detection of pulmonary nodules in lung ct images

87

The extracted AHSN feature for a sphere (nodule model), left –

reconstructed 3-D shape, center - orientation θ histogram, right -

orientation φ histogram

Page 88: automatic detection of pulmonary nodules in lung ct images

88

The extracted AHSN feature for a cylinder (vessel model), left –

reconstructed 3-D shape, center - orientation θ histogram, right -

orientation φ histogram

Page 89: automatic detection of pulmonary nodules in lung ct images

89

The extracted AHSN feature for a curved surface (wall model), left –

reconstructed 3-D shape, center - orientation θ histogram, right -

orientation φ histogram

Page 90: automatic detection of pulmonary nodules in lung ct images

90

The extracted AHSN feature for a pulmonary nodule, left – reconstructed

3-D shape, center - orientation θ histogram, right - orientation φ

histogram

Page 91: automatic detection of pulmonary nodules in lung ct images

91

The extracted AHSN feature for a pulmonary vessel, left – reconstructed

3-D shape, center - orientation θ histogram, right - orientation φ

histogram

Page 92: automatic detection of pulmonary nodules in lung ct images

• Lung wall influence the detection accuracy

• For more accurate nodule detection, walls

are eliminated from image blocks of

nodule candidates

92

Page 93: automatic detection of pulmonary nodules in lung ct images

93

Page 94: automatic detection of pulmonary nodules in lung ct images

94

Comparison of AHSN feature for a juxta-pleural nodule at before (1st

row) and after (2nd row) wall elimination, left - reconstructed 3-D shape,

center - orientation θ histogram, right - orientation φ histogram

Page 95: automatic detection of pulmonary nodules in lung ct images

95

Comparison of AHSN feature for a solid nodule at before (1st row) and

after (2nd row) wall elimination, left - reconstructed 3-D shape, center -

orientation θ histogram, right - orientation φ histogram

Page 96: automatic detection of pulmonary nodules in lung ct images

• The extracted AHSN feature vectors are

analyzed by SVM classifier

• SVM is a useful technique for data

classification

• k-fold cross-validation is applied to

evaluated the proposed classifier (k = 10)

96

Page 97: automatic detection of pulmonary nodules in lung ct images

Classfier Accuracy Sensitivity Specificity

Before LWE SVM-p 96.4% 98.4% 94.3%

SVM-r 97.8% 98.7% 96.9%

SVM-m 93.9% 95.9% 92.0%

After LWE SVM-p 97.0% 97.9% 96.1%

SVM-r 97.8% 97.4% 98.2%

SVM-m 94.5% 94.6% 94.3%

97

The results of 10-fold cross validation on different kernel functions using

SVM as a classier before and after wall elimination (LWE)

Page 98: automatic detection of pulmonary nodules in lung ct images

Descriptor Accuracy Sensitivity Specificity

SVM-p Gradient 95.1% 96.4% 93.8%

Hessian Matrix 97.0% 97.9% 96.1%

SVM-r Gradient 96.1% 96.4% 95.9%

Hessian Matrix 97.8% 97.4% 98.2%

SVM-m Gradient 92.8% 93.0% 92.6%

Hessian Matrix 94.5% 94.6% 94.3%

98

The results of 10-fold cross validation on with four different kernel

functions based SVMs for the descriptors using gradient and Hessian

matrix

Page 99: automatic detection of pulmonary nodules in lung ct images

Descriptor Accuracy Sensitivity Specificity

SVM-p AHSN 180 97.0% 97.9% 96.1%

AHSN 90 96.9% 97.4% 96.4%

AHSN 72 96.9% 98.5% 95.3%

AHSN 36 96.0% 97.4% 94.6%

3-D SIFT 128 92.9% 93.3% 92.5%

3-D HOG 468 95.2% 96.7% 93.8%

3-D HOG 216 94.2% 95.1% 93.3%

SVM-r AHSN 180 97.8% 97.4% 98.2%

AHSN 90 97.5% 97.2% 97.9%

AHSN 72 97.6% 97.4% 97.7%

AHSN 36 96.5% 96.9% 96.1%

3-D SIFT 128 36.2% 8.4% 100.0%

3-D HOG 468 77.2% 36.5% 100.0%

3-D HOG 216 89.3% 78.9% 99.7%

SVM-m AHSN 180 94.5% 94.6% 94.3%

AHSN 90 95.2% 95.3% 95.1%

AHSN 72 95.8% 96.2% 95.4%

AHSN 36 94.9% 95.4% 94.3%

3-D SIFT 128 88.9% 87.9% 89.9%

3-D HOG 468 94.0% 90.9% 94.0%

3-D HOG 216 94.7% 94.3% 95.1%

99

The results of 10-fold cross validation for the different descriptors on various kernel functions

of SVM classifier

Page 100: automatic detection of pulmonary nodules in lung ct images

100

ROC curves of the SVM classifiers with respect to three different kernel

functions, SVM-r: radial basis function, SVM-p: polynomial function, and SVM-m:

Minkowski distance function; (a) p = 0:25 and (b) p = 1.

Page 101: automatic detection of pulmonary nodules in lung ct images

101

Page 102: automatic detection of pulmonary nodules in lung ct images

102

Nodules Non-nodules

Page 103: automatic detection of pulmonary nodules in lung ct images

103

(a) (b)

The result of pulmonary nodule detection: (a) 43rd slice, (b) 3-D

representation, the detected nodules are indicated by a red color and the

non-nodules are indicated by a white color

Page 104: automatic detection of pulmonary nodules in lung ct images

104

AUC Accuracy Specificity Sensitivity FPs/scan

Nodule Candidates Detection 96.6% 51.25

20-80 0.921 76.6% 75.9% 88.3% 12.32

50-50 0.960 86.7% 86.4% 91.7% 6.99

80-20 0.967 89.6% 89.3% 90.9% 5.45

The results of CAD system using GP based classifier

Page 105: automatic detection of pulmonary nodules in lung ct images

105

FROC curves of the GPC with respect to three training and testing

datasets

Page 106: automatic detection of pulmonary nodules in lung ct images

106

AUC Accuracy Specificity Sensitivity FPs/scan

Nodule Candidates Detection 97.3% 60.21

0.1 0.9931 95.89% 99.62% 92.67% 0.23

0.125 0.9934 96.92% 99.11% 93.95% 0.54

0.25 0.9929 97.61% 96.23% 95.28% 2.27

0.5 0.9835 95.15% 93.93% 92.85% 3.65

1 0.9727 92.98% 92.33% 90.63% 4.62

2 0.9584 92.41% 89.74% 90.45% 6.18

The results of CAD system using Hierarchical Block-image Analysis

Page 107: automatic detection of pulmonary nodules in lung ct images

107

FROC curves of the proposed CAD system with respect to three different

kernel parameters of SVM-r classifiers

Page 108: automatic detection of pulmonary nodules in lung ct images

108

The overall performance of CAD system for different parameters p of SVM-r

classifiers

AUC Accuracy Specificity Sensitivity FPs/scan

Nodule Candidates Detection 97.9% 135.39

AHSN 180 0.9945 97.8% 98.2% 95.4% 2.43

AHSN 90 0.9923 97.5% 97.9% 95.2% 2.84

AHSN 72 0.9895 97.6% 97.7% 95.4% 3.11

Page 109: automatic detection of pulmonary nodules in lung ct images

109

FROC curves of the proposed CAD system with respect to three different

dimensions of AHSN features

Page 110: automatic detection of pulmonary nodules in lung ct images

CAD systems Nodule size FPs per case Sensitivity

Suzuki et al.(2003)[26] 8 - 20 mm 16.1 80.3%

Rubin et al.(2005)[27] >3 mm 3 76%

Dehmeshki et al.(2007)[28] 3 - 20 mm 14.6 90%

Suarez-Cuenca et al.(2009)[29] 4 - 27 mm 7.7 80%

Golosio et al.(2009)[30] 3 - 30 mm 4.0 79%

Ye et al.(2009)[31] 3 - 20 mm 8.2 90.2%

Sousa et al.(2010)[32] 3 - 40.93 mm - 84.84%

Messay et al.(2010)[33] 3-30 mm 3 82.66%

Riccardi et al.(2011)[34] >3 mm 6.5 71.%

Cascio et al.(2012)[35] 3-30 mm 6.1 97.66%

Genetic Programming 3-30 mm 5.45 90.9%

Hierarchical Block Analysis 3-30 mm 2.27 95.2%

Shape-based Feature 3-30 mm 2.43 95.4%

110

Page 111: automatic detection of pulmonary nodules in lung ct images

• Automated pulmonary nodule detection system is studied

• Pulmonary nodule detection CAD system is an effective solution for early detection of lung cancer

• The proposed systems are based on

– Genetic programming based classifier

– Hierarchical block-image analysis

– 3-D shape-based feature descriptor

111

Page 112: automatic detection of pulmonary nodules in lung ct images

• The performance of the proposed CAD systems is evaluated on the LIDC database of NCI

• The GPC based system was shown to significantly reduce the false positives while maintaining a high sensitivity – 5.45 FPs/scan, 90.9% sensitivity

• The hierarchical block-image analysis based system has shown more accurate result with improved local object segmentation – 2.27 FPs/scan, 95.28% sensitivity

• Shape-based feature descriptor was applied the nodule detection CAD system that has shown higher accuracy and robustness than conventional descriptor – 2.43 FPs/scan, 95.4% sensitivity

• The proposed methods have significantly reduced the false positives in nodule candidates

112

Page 113: automatic detection of pulmonary nodules in lung ct images

113

Page 114: automatic detection of pulmonary nodules in lung ct images

114