image processing in lung cancer screening and treatment

Image Processing in Lung Cancer

Screening and Treatment

Wookjin Choi, PhD

Introduction

Lung cancer is the leading cause of cancer deaths.

Most patients diagnosed with lung cancer already have advanced disease

40% are stage IV and 30% are III

The current five-year survival rate is only 16%

(a) male (b) female

Trends in death rates for selected cancers, United States, 1930-2008 2

Lung cancer screening and treatment

Lung Cancer Screening

• Nodule detection and diagnosis

• Biopsy

• …

Lung Cancer Treatment

• Surgery

• Chemotherapy

• Radiotherapy

• Tumor Response

• …

Computer-Aided De

tection (CADe) and

Diagnosis (CADx)

Image-guided Radiot

herapy and Quantitat

ive Assessment

3

Image processing in Lung cancer screening and

treatment

Feature Extraction and Analysis

Nodule (tumor) segmentation

Lung segmentation

Image Registration

4

Lung Segmentation

• Thresholding– Fixed threshold

– Optimal threshold

– 3-D adaptive fuzzy thresholding

• Lung region extraction– 3-D connectivity with seed point

– 3-D connected component labeling

• Contour correction– Morphological dilation

– Rolling ball algorithm

– Chain code representation

5

Lung Segmentation

6

Lung CAD

COMPUTER-AIDED

DETECTION AND DIAGNOSIS

7

Detecting Lung Cancer is hard

8

Where is t

he nodule

?

9

Where is t

he lung ca

ncer?

10

Where is t

he lung ca

ncer?

11

Computer Aided Detection

12

Pulmonary Nodule Detection CAD system

CAD systems Lung segmentation Nodule Candidate Detection False Positive Reduction

Suzuki et al.(2003)[3] Thresholding Multiple thresholding MTANN

Rubin et al.(2005)[4] Thresholding Surface normal overlapLantern transform and rule-

based classifier

Dehmeshki et al.(2007)[5] Adaptive thresholding Shape-based GATM Rule-based filtering

Suarez-Cuenca et al.(2009)[6]

Thresholding and 3-D

connected component

labeling

3-D iris filteringMultiple rule-based LDA

classifier

Golosio et al.(2009)[7] Isosurface-triangulation Multiple thresholding Neural network

Ye et al.(2009)[8]3-D adaptive fuzzy

segmentationShape based detection

Rule-based filtering and

weighted SVM classifier

Sousa et al.(2010)[9] Region growing Structure extraction SVM classifier

Messay et al.(2010)[10]

Thresholding and 3-D

connected component

labeling

Multiple thresholding and

morphological opening

Fisher linear discriminant and

quadratic classifier

Riccardi et al.(2011)[11] Iterative thresholding3-D fast radial filtering and

scale space analysis

Zernike MIP classification

based on SVM

Cascio et al.(2012)[12] Region growing Mass-spring modelDouble-threshold cut and

neural network 13

Genetic Programming based Classifier

Wook-Jin Choi, Tae-Sun Choi, “Genetic programming-based feature transform and classification for the automatic detection of pulmonary nodules on

computed tomography images”, Information Sciences, Vol. 212, pp. 57-78, December 2012, doi: http://dx.doi.org/10.1016/j.ins.2012.05.008

Feature spaces for four types of features

2-D geometric feature 3-D geometric feature

2-D intensity-based statistical feature 3-D intensity-based statistical feature

Genetic programming classifier learning

Classification space

GP based classification expression in tree shape

Optimal multi-thresholding based Nodule candidates de

tection.

14

Hierarchical Block-image Analysis

Wook-Jin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection System in Computed Tomography Images: A Hierarchical Block

Classification Approach”, Entropy, Vol. 15, No. 2, pp. 507-523, February 2013, doi:http://dx.doi.org/10.3390/e15020507

ROC curves of the SVM classifiers with respect to three different kernel functions,

SVM-r: radial basis function, SVM-p: polynomial function, and SVM-m:

Minkowski distance function; (a) p = 0:25 and (b) p = 1.Result images after block splitting with respect to various block sizes

The entropy histograms of block-images for five different block sizes

(x-axis : entropy value, y-axis : number of blocks, (a) 32, (b) 24, (c) 16, (d) 12, and (e) 8)

15

θ φ

θ φ

Three-dimensional Shape-based Feature Descriptor

Wook-Jin Choi, Tae-Sun Choi, “Automated Pulmonary Nodule Detection based on Three-dimensional Shape-based Feature Descriptor”, Computer

Methods and Programs in Biomedicine, Vol. 113, No. 1, January 2014, pp. 37–54, doi: http://dx.doi.org/10.1016/j.cmpb.2013.08.015

Surface saliency weighted surface

normal vectors

Two angular histograms of the

surface normal vectors

θ φ

ROC curves of the SVM classifiers with respect to three different kernel

functions, SVM-r: radial basis function, SVM-p: polynomial function,

and SVM-m: Minkowski distance function; (a) p = 0:25 and (b) p = 1.

FROC curves of the proposed CAD system with

respect to three different dimensions of AHSN

features

θ φ

θ φ

Feature optimization with wall detection a

nd elimination algorithm

3D shape-based feature descriptor

High surface saliency

value

Low surface saliency

value

16

Comparative Analysis

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16

Ov

era

ll s

ensi

tivit

y

False positives per scan

Suzuki et al. (2003)

Dehmeshki et al. (2007)

Suarez-Cuenca et al.

(2009)

Golosio et al. (2009)

Ye et al. (2009)

Messay et al. (2010)

Riccardi et al. (2011)

Cascio et al. (2012)

Choi et al. (2012)

Choi et al. (2013)

Choi et al. (2014)

17

Computer Aided Diagnosis

Once the lung nodules are detected and segmented from the

corresponding chest images

The next task is to determine whether the detected nodules

are malignant or benign

The malignancy of lung nodules correlates highly with

Geometrical size

Shape

Appearance descriptors

Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Metho

dologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353 18

Computer Aided Diagnosis

Typical computer-aided diagnosis (CAD) system for lung cancer. The input of a CAD system i

s the medical images obtained using an appropriate modality. A lung segmentation step is used

to reduce the search space for lung nodules. Nodule detection is used to identify the locations

of lung nodules. The detected nodules are segmented. Then, a candidate set of features, such a

s volume, shape, and/or appearance features, are extracted and used for diagnosis.

Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Meth

odologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353 19

Computer Aided Diagnosis

Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Meth

odologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353

Study Purpose Method Database Observations

Jennings

et al.

To retrospectively

determine the distribution

of stage I lung cancer

growth rates with serial

volumetric CT

Volumetric

measurement

149

patients

At serial volumetric CT measurements, there was wide

variability in growth rates. Some biopsy-proved cancers

decreased in volume between examinations

Zheng et

al.

To simultaneously segment

and register lung and

tumor in serial CT data

Nonrigid

transformation

on lung

deformation and

rigid structure

on the tumor

6 volumes

of 3

patients,

83 nodules

The mean error of boundary distances between automatic

segmented boundaries of lung tumors and manual

segmentation is 3.50 pixels. The mean and variance of

percentages of the nodule volume variations caused by

errors in segmentation are 0.8 and 0.6

Marchian

ò et al.

To assess in vivo

volumetric repeatability of

an automated algorithm for

volume estimation.

Semiautomatic

volumetric

measurement

101

subjects,

233

nodules

The 95% confidence interval for difference in measured

volumes was in the range of ±27%. About 70% of

measurements had a relative difference in nodule volume

of less than 10%

El-Baz et

al.

To monitor the

development of lung

nodules

Global and local

registration, GR

volumetric

measurement

135 LDCT

from 27

subjects,

27 nodules

All the 27 nodules are correctly classified based on GR

measurements over 12 months

Growth-rate-based methodologies for following up pulmonary nodules.

20

Computer Aided Diagnosis

Ayman El-Baz, Garth M. Beache, Georgy Gimel'farb, et al., “Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Meth

odologies,” International Journal of Biomedical Imaging, vol. 2013, Article ID 942353, 46 pages, 2013. doi:10.1155/2013/942353

Study Purpose Method Database Observations

Suzuki et al.To classify nodules into

Benign or Malignant

Multiple MTANNs using

pixel values in

a subregion

Thick-slice (10 mm) screening

LDCT scans of

76 M and 413 Bnodules

= 0.88 in a leave-one-

out test

Iwano et al.To classify nodules into

benign or malignant

LDA based on nodule's

circularity and second

moment features

HRCT (0.5–1 mm slice) scans of

52 Mand 55 B nodules

Sensitivity of 76.9% and

a specificity of 80%

Way et al.To classify nodules into

benign or malignant

LDA or SVM with

stepwise feature selection

CT scans of 124 Mand

132 B nodules in 152 patients

= 0.857 in a leave-one-

out test

Chen et al.To classify nodules into

benign or malignantANN ensemble

CT scans (slice thickness of 2.5

or 5 mm) of 19 M and

13B nodules

= 0.915 in a leave-one-

out test

Lee et al.To classify nodules into

benign or malignant

GA-based feature

selection and a random

subspace method

Thick-slice (5 mm) CT scans of

62 M and 63 B nodules

= 0.889 in a leave-one-

out test

El-Baz et al.To classify nodules into

benign or malignant

Analysis of the spatial

distribution of the nodule

Hounsfield values

CT scans (2 mm slice) of

51 M and 58 B nodules

Sensitivity of 92.3% and

a specificity of 96.6%

Classification between malignant (M) and benign (B) nodules based on shape and appearance features.

21

Lung CAD

IMAGE-GUIDED RADIATION

THERAPY

22

Image-Guided Radiotherapy

Gupta T, Narayan C A, Image-guided radiation therapy: Physician's perspectives, J Med Phys

Tumor Segmen

tationRegistration

Feature Extracti

on and Analysis

Registration

Registration

Tumor Segmen

tation

23

Radiotherapy and lung cancer

The efficacy and safety of RT reflect the interplay

between

• Total dose delivered to the malignant tumor

• The rate of dose delivery (daily fractionation)

• The volume (and type) of tumor-bearing organ irradiated.

• The intrinsic tolerance of the tissue irradiated

24

4D CT

Excellent for taking into account

respiratory motion

Takes a set of CT images and

sorts them to represent each

phase of the breathing cycle

Box with infrared reflectors on

abdomen, set up infrared camera

to capture movement

Why is 4D CT important?

Same slice in different

phases of the breathing

cycle showing tumor

movement

26

Is 4DCT Worthwhile?

Underberg, R.W.M., Lagerwaard, F.J., Cuijpers, J.P., Slotman, B.J., Van Sornsen de Koste, J.R.,Senan, S. (2004). Four-Dimensional CT Scan

s for Treatment Planning in Stereotactic Radiotherapy for Stage 1 Lung Cancer, International Journal of Radiation Oncology Biology Physics

, 60: 4.

27

Gating

• Utilize 4DCT scan to get brea

thing pattern

• Determine a phase of the bre

athing cycle to treat during, p

lan on that scan Only

• Monitor treatment with respi

ratory motion

• when patient’s breathing

enters the selected part o

f the breathing cycle, tre

atment is delivered

Varian RPM system

28

Stereotactic body radiotherapy (SBRT)

Modeled after brain radiosurgery principles

• Multiple convergent beams

• Rigid patient immobilization

• Precise localization via stereotactic coordinate system

• Single fraction treatment

• Size-restriction for target

29

Anatomic Tumor Response Assessment in CT or MRI

Imaging as Surrogate for

Survival or progression-free survival

Response rate, time to tumor progression

RECIST criteria based on longest diameter

Complete response (CR): disappear

Partial response (PR): ≥ 50% decrease

Stable disease (SD): others

Progressive disease (PD): ≥ 25% increase or new

Tumor size change does not occur or does not occur early in

some effective treatments

30

Metabolic Tumor Response Assessment in FDG-PET

Strong correlation between FDG uptake and cancer cell

number

Metabolic (functional) change may occur earlier and more

markedly than tumor size change

Qualitative evaluation plus semi-quantitative assessment

with SUV or SUL

31

Wahl, J Nucl Med. 50(Suppl 1): 122S–150S.

Large decline in SUL (-41%) despite stable pancreatic mass anatomically (a

rrows) Partial metabolic response.

PET/CT for Tumor Response: An Example in Pancre

atic Tumor

32

Qualitative (Visual) PET Response Evaluation

Distribution and intensity of FDG uptake in tumor are

compared with uptake in normal tissues

Changes are visually evaluated

Requires clinical experience, disease patterns

Performs well in conversion of markedly positive PET scan to totally

negative scan

Moderate inter-observer variation

Difficult to detect small changes

33

Semi-Quantitative PET Response Assessment

SUV is most widely used

SUL is more consistent across patients

ROI selection

Maximal pixel: SUVmax, not as reproducible

Manual contour

Small fixed region ~1 cm3: SUVpeak

Fixed percentage isocontour: 40%, 50%

Fixed threshold: SUV = 2.5

3×SD above background (typically liver)

34

PERCIST Criteria

SULpeak of the hottest tumor

PERCIST criteria

CMR: normalize to background level

PMR: ≥ 30% decrease and ≥ 0.8 unit in SUL

SMR: others

PMD: ≥ 30% increase and ≥ 0.8 unit in SUL or visible increase in extent

of uptake (75% in TLG) with no decline in SUL, or new FDG-avid lesion

35

FDG Uptake Shows Spatial Variation

Belhassen and Zaidi 2010. Med Phys

Zhao, et al. 2005. J Nucl Med

36

Quantitative PET/CT analysis framework

Extract spatial-temporal image features:

Intensity distribution (histogram)

Spatial variations (texture)

Geometric properties (shape, structure)

Temporal changes due to therapy

Construct response models using machine learning

techniques with multiple features

Feature selection

Support vector machine

Cross-validation

37

• Region growing

• Morphology filter

• Multi-modality im

age segmentation

• ITK

• Intensity distribution

• Spatial variations

• Geometric properties

• > 100 features for each

tumor

• ITK

• ROC analyses

• Tumor response

• Survival

• Matlab

Tumor

Segmentation

Image

Registration

Feature

ExtractionPredicting

Ability

• Multi-level rigid

• Pre/Post-CT

• ITK

Extracting Spatial-Temporal FDG-PET Features for

Tumor Response Evaluation

38

Registration

Article Type of Registration Abnormality Treatme

nt Scanning Time

Aristophanous et al. Rigid NSCLC RT Before and after

treatment

Necib et al. Rigid Metastatic colorectal cancer CTx Before and after

treatment

Tan et al. Rigid Esophageal Cancer CRT Before and after treatment

Vera et al. Rigid Esophageal cancer CRT Before and during treatment

Cannon et al. Deformable(Demons)

Head and neck cancer

RT or CRT

Before and after treatment

Roels et al. Deformable (B-Spline) Rectal Cancer CRT Before, during and

after treatment

van Velden et al. DeformableAdvanced colorectal carcinoma

CTx Before and after treatment

PET/CT based tumor response assessment studies using rigid registration, deformable registration or the combinatio

n of rigid and deformable registration algorithms. 39

Tumor Segmentation

Tumor segmentation can be performed either manually by

physicians or (semi-)automatically using image analysis

tools.

The accuracy of a tumor segmentation method is often hard

to evaluate in patients due to the lack of ground truth.

In response evaluation that involves two or more serial image

studies

The reproducibility of a segmentation method is as important as its

accuracy

40

Multi-modality adaptive region-growing (MARG)

A sharp volume increase occurred at an f where the region

just grows into the background (normal tissue)

f was identified by fitting the curve and calculating curvature

Tumor A

rea

Background Area

f

41

MARG: Results on a NSCLC Dataset from AAPM

TG211

Pathologic tumor v

olume

MARG results 50% threshold res

ult

• For 10 patients, MARG (Dice = 0.69), slightly higher accuracy than thresh

olding methods (Dice = 0.67)

• Accuracy limited by the reliability of 3D pathologic tumor volume reconstr

uction and its alignment with PET/CT images 42

Spatial-Temporal FDG-PET Features for Predicting

Pathologic Tumor Response

A new SUV intensity feature - Skewness

pre-CRT

• Top: responder, more skewed (fewer hig

her SUVs)

• Bottom: non-responder, less skewed (mo

re higher SUVs)

Three texture features post-CRT – Inertia, Co

rrelation, and Cluster Prominence

• Top: responder, homogeneous FDG uptake p

ost-CRT

• Bottom: non-responder, heterogeneous FDG

uptake post-CRT

43

Accuracy of Individual Spatial-Temporal FDG-PET

Features

44

FDG-PET Histogram Distances for Predicting

Pathologic Tumor Response

• A responder shows larger histogram dista

nce from pre-CRT to post-CRT

• A non-responder shows smaller histogra

m distance

45

Accuracy of Individual Histogram Distances

7 bin-to-bin and 7 cross-bin histogram distances have high

er AUCs than conventional PET/CT response measures

46

Modeling Tumor Response – Model Construction

and Evaluation with Cross-Validation

47

Results

20 patients with esophageal cancer. Model predicts

pathologic response to chemoradiotherapy (CRT)

SVM model with 17 selected features from all feature groups:

AUC = 1.0, sensitivity = 100%, specificity = 100%

Models with conventional PET/CT response measures or

clinical parameters: AUCs < 0.75

48

Conclusions

Image processing in Lung cancer screening and

treatment

Computer aided detection and diagnosis

• Lung segmentation

• Nodule detection and segmentation

• Feature extraction and analysis

Image-guided radiotherapy

• Registration

–CT/CT, PET/CT

• Tumor segmentation

• Feature extraction and analysis49

Future works

Validate the accuracy of image registration and tumor segmentation methods

The usefulness of image features, and the generalizability of response models

often developed on small retrospective datasets in large retrospective and prospective datasets.

Clinic and biologic interpretation of the advanced PET/CT image features

For physicians and biologists

Challenges for implementing the quantitative PET/CT image analysis for tumor response evaluation

Delineating the tumor volume in multi-modality (PET/CT) images

Identifying a few features that truly capture biological changes correlated with tumor response for a specific disease and therapy

Validating the results in large, multi-center patient datasets, vendor implementation and ultimately clinic acceptance

Thank you