classification of benign and malignant vertebral...

90
Lucas Frighetto-Pereira, Guilherme Augusto Metzner, Paulo Mazzoncini de Azevedo-Marques, Rangaraj Mandayam Rangayyan, Marcello Henrique Nogueira-Barbosa Classification of Benign and Malignant Vertebral Compression Fractures in Magnetic Resonance Images

Upload: others

Post on 23-Mar-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Lucas Frighetto-Pereira, Guilherme Augusto Metzner,

Paulo Mazzoncini de Azevedo-Marques,

Rangaraj Mandayam Rangayyan,

Marcello Henrique Nogueira-Barbosa

Classification of Benign and Malignant

Vertebral Compression Fractures in Magnetic Resonance Images

Page 2: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Anatomy of the spine

โ– MRI sagittal slice

VertebralBody

IntervertebralDisc

VertebralArch

Vertebra

LumbarSpine

UNIVERSIDADE DE

SรƒO PAULO

Page 3: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Vertebral compressionfractures (VCFs)

โ– Partial collapse of vertebral bodies

โ– Traumatic VCFs raise no doubt about

their etiology

โ–But a recent vertebral collapse withouthistory of significant trauma createsdifficulty in defining the cause of the VCF

UNIVERSIDADE DE

SรƒO PAULO

Page 4: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Medical diagnosis

โ€ข Young patient with a VCF

โ€ข History of significant acute trauma

โ€ข Usually easy diagnosis

UNIVERSIDADE DE

SรƒO PAULO

Page 5: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Medical diagnosis

โ€ข Elderly patient with VCF

โ€ข No history of significant acute trauma

โ€ข Diagnosis ?

UNIVERSIDADE DE

SรƒO PAULO

Page 6: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

VCFs without history ofsignificant trauma

โ–VCFs are the most common type ofosteoporotic fractures

โ–The elderly have a high incidence of VCFsrelated to metastatic cancer affecting bone

โ–MRI is the most commonly used imagingmethod for spinal diseases and earlydetection of fractures

UNIVERSIDADE DE

SรƒO PAULO

Page 7: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

OsteoporoticVCF

MetastaticVCF

T1-WeightedMRI

Page 8: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Clinical classificationof VCFs

โ–Osteoporotic VCFs

โžข classified as Benign VCFs

โ–Metastatic VCFs

โžข classified as Malignant VCFs

UNIVERSIDADE DE

SรƒO PAULO

Page 9: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Benign VCFs in T1-weighted MRI

โ– Partial preservation of normal fatty bone-marrow signal in the vertebral body

โ–Degeneration of normally rectangularshapes of vertebrae into concave and roughshapes with indentations

โ–Rougher contours than malignant VCFs andnormal vertebrae

UNIVERSIDADE DE

SรƒO PAULO

Page 10: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Malignant VCFs in T1-weighted MRI

โ–Global reduction of signal intensity or nodular abnormality in the affected vertebral body

โ–Could result in a posterior convexity without substantial concavities

โ–May also cause the contours of vertebrae to be relatively smoothened due to convexity

UNIVERSIDADE DE

SรƒO PAULO

Page 11: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Normal Benign VCFs Malignant VCFs

Page 12: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Benign vs MalignantVCFs

โ–Both tend to create concavities in the vertebral plateaus

โ–Could cause doubt in the diagnosis

โ–Correct classification is critical for planning treatment

UNIVERSIDADE DE

SรƒO PAULO

Page 13: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Malignant VCF

Benign VCF

Which image has the malignant VCF and which one has the benign VCF?

Page 14: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Objectives

โ– Study the characteristics of VCFs in MRI

โ– Develop image processing techniques to extractfeatures

โ– Classify VCFsNormal

Fractured

Benign

Malignant

UNIVERSIDADE DE

SรƒO PAULO

Page 15: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Study steps

โ– Selection of cases and images

โ–Manual segmentation of vertebral bodies

โ– Extraction of features of vertebral bodies

โ–Classification, validation, and statisticalanalysis

UNIVERSIDADE DE

SรƒO PAULO

Page 16: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Database

โ–University Hospital of Ribeirรฃo Preto Medical School โ€“ University of Sรฃo Paulo

โ–Cases and images collected from theRadiology Information System (RIS)

โ–Cases from September 2010 to March 2014

โ– Philips 1.5T MRI System โ€“ T1-weighted MRI

UNIVERSIDADE DE

SรƒO PAULO

Page 17: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Database

โ– Lumbar vertebral bodies (L1 to L5)

โ–Median sagittal slice

โ– TIFF images with 8-bits/pixel

โ– 153 exams analyzed, 63 selected

โ– 38 women, 25 men

โ–Mean age: 62 years

UNIVERSIDADE DE

SรƒO PAULO

Page 18: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Database

โ– 63 selected exams:

โžข At least one VCF per patient

โžข The nonfractured vertebral bodies of patientswithout malignant fractures are considered tobe normal

UNIVERSIDADE DE

SรƒO PAULO

Page 19: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Excluded cases

โ–Vertebral fractures secondary to trauma

โ– Infection and avascular necrosis

โ– Severe degenerative scoliosis

โ– Previous surgeries, radiotherapy, andchemotherapy

UNIVERSIDADE DE

SรƒO PAULO

Page 20: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Database

L5 L4 L3 L2 L1 Total

Benign VCFs 6 7 9 10 21 53

Malignant VCFs 9 11 10 10 9 49

Normal 26 24 23 22 11 106

Total 41 42 42 42 41 208

UNIVERSIDADE DE

SรƒO PAULO

Page 21: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Examples ofvertebral bodies

Normal

Benign VCFs

Malignant VCFs

UNIVERSIDADE DE

SรƒO PAULO

Page 22: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Manual segmentation

MRI exam Vertebral body masks

UNIVERSIDADE DE

SรƒO PAULO

Page 23: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Software flow chart UNIVERSIDADE DE

SรƒO PAULO

Page 24: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

MRI exam and its maskUNIVERSIDADE DE

SรƒO PAULO

Page 25: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Detection of the coordinatesof the vertebral bodies

L5

L4

L3

L2

L1

UNIVERSIDADE DE

SรƒO PAULO

Page 26: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Normalizationof the MR images

. 5x5 disc block

Extraction of blocks ofintervertebral discs

using the mask ROIs as reference

UNIVERSIDADE DE

SรƒO PAULO

DiscsMean

Page 27: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Normalizationof the MR images

๐‘›๐‘’๐‘ค๐ผ๐‘š๐‘” ๐‘–, ๐‘— =๐‘–๐‘š๐‘”๐‘‚๐‘Ÿ๐‘–๐‘”๐‘–๐‘›๐‘Ž๐‘™(๐‘–, ๐‘—)

๐‘‘๐‘–๐‘ ๐‘๐‘ ๐‘€๐‘’๐‘Ž๐‘›

๐’Š๐’Ž๐’ˆ๐‘ต๐’๐’“๐’Ž ๐’Š, ๐’‹ = 255 ร—๐‘›๐‘’๐‘ค๐ผ๐‘š๐‘” ๐‘–, ๐‘— โˆ’ min ๐‘›๐‘’๐‘ค๐ผ๐‘š๐‘”

max ๐‘›๐‘’๐‘ค๐ผ๐‘š๐‘” โˆ’min ๐‘›๐‘’๐‘ค๐ผ๐‘š๐‘”

UNIVERSIDADE DE

SรƒO PAULO

Page 28: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

MRI exam โˆฉ Mask

โˆฉ

Processing new image...

UNIVERSIDADE DE

SรƒO PAULO

Page 29: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Detection of theROIs

UNIVERSIDADE DE

SรƒO PAULO

Page 30: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

ROIs of thevertebral bodies UNIVERSIDADE DE

SรƒO PAULO

Normal

Benign VCFs

Malignant VCFs

Page 31: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Computation of thefeatures

โ– 3 Statistical gray-level features

โ– 14 Texture features

โ– 10 Shape features

27 Features

UNIVERSIDADE DE

SรƒO PAULO

Page 32: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Statistical gray-levelfeatures

Coefficient ofvariation

Skewness

Kurtosis

UNIVERSIDADE DE

SรƒO PAULO

Page 33: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Statistical gray-levelfeatures

โ–Coefficient of variation (CV )

๐ถ๐‘‰ =๐œŽ

๐œ‡

๐œ‡ =1

256

๐‘–=1

256

๐‘ฅ๐‘–

๐œŽ =1

256

๐‘–=1

256

๐‘ฅ๐‘– โˆ’ ๐œ‡ 2

UNIVERSIDADE DE

SรƒO PAULO

Page 34: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Statistical gray-levelfeatures

โ– Skewness

๐‘ ๐‘˜๐‘’๐‘ค๐‘›๐‘’๐‘ ๐‘  =1

256 ร— ๐œŽ3

๐‘–=1

256

(๐‘ฅ๐‘– โˆ’ ๐œ‡)3

UNIVERSIDADE DE

SรƒO PAULO

Page 35: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Statistical gray-level features

โ–Kurtosis

๐‘˜๐‘ข๐‘Ÿ๐‘ก๐‘œ๐‘ ๐‘–๐‘  =1

256 ร— ๐œŽ4

๐‘–=1

256

(๐‘ฅ๐‘– โˆ’ ๐œ‡)4

UNIVERSIDADE DE

SรƒO PAULO

Page 36: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Differences in texturebetween normal and VCFs UNIVERSIDADE DE

SรƒO PAULO

Malignant VCF

Malignant VCF

Malignant VCF

Malignant VCF

Normal

Benign VCF

Page 37: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Texture features

Gray-level

cooccurrence matrix

14 texture features of

Haralick et al.

UNIVERSIDADE DE

SรƒO PAULO

Page 38: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Cooccurrence matrix

0 0 1 0 0

0 1 2 1 0

1 2 2 2 1

0 1 2 1 0

0 0 1 0 0

Ex: Image 5x5 pixels,3 gray levels

0 1 2

2 2 4

2 4 2

4 2 2

Distance = 1 pixel

Angle = ยฑ45ยฐ

0 1 2

0

1

2

Number of pixels of intensity 0 thatare at ยฑ45 degrees and distance 1

of pixels of intensity 2

UNIVERSIDADE DE

SรƒO PAULO

Cooccurrence matrix ๐‘ ๐‘–, ๐‘— for

๐‘– = 0, ๐‘— = 2

Page 39: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

14 texture featuresof Haralick et al.

โ–Angular second moment (Energy)

๐‘“1 =

๐‘–

๐‘—

๐‘(๐‘–, ๐‘—) 2

โ–Contrast

๐‘“2 =

๐‘›=0

๐‘๐‘”โˆ’1

๐‘›2

๐‘–=1

๐‘๐‘”

๐‘—=1

๐‘๐‘”

๐‘ ๐‘–, ๐‘—

UNIVERSIDADE DE

SรƒO PAULO

๐‘๐‘”: number of

distinct gray levelsin the quantized image

Page 40: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ–Correlation

๐‘“3 =ฯƒ๐‘–ฯƒ๐‘— ๐‘–๐‘— ๐‘ ๐‘–, ๐‘— โˆ’ ๐œ‡๐‘ฅ ๐œ‡๐‘ฆ

๐œŽ๐‘ฅ ๐œŽ๐‘ฆ

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 41: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

14 texture featuresof Haralick et al.

โ– ๐œ‡๐‘ฅ , ๐œ‡๐‘ฆ means

โ–๐œŽ๐‘ฅ , ๐œŽ๐‘ฆ standard deviations

UNIVERSIDADE DE

SรƒO PAULO

๐‘๐‘ฆ ๐‘— =

๐‘–=1

๐‘๐‘”

๐‘ ๐‘–, ๐‘—๐‘๐‘ฅ ๐‘– =

๐‘—=1

๐‘๐‘”

๐‘ ๐‘–, ๐‘—

Page 42: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Sum of squares: Variance

๐‘“4 =

๐‘–

๐‘—

๐‘– โˆ’ ๐œ‡ 2 ๐‘(๐‘–, ๐‘—)

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 43: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Inverse difference moment

๐‘“5 =

๐‘–

๐‘—

1

1 + ๐‘– โˆ’ ๐‘— 2๐‘(๐‘–, ๐‘—)

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 44: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Sum average

โ– Sum variance

๐‘“6 =

๐‘–=2

2๐‘๐‘”

๐‘– ๐‘๐‘ฅ+๐‘ฆ (๐‘–)

๐‘“7 =

๐‘–=2

2๐‘๐‘”

๐‘– โˆ’ ๐‘“82 ๐‘๐‘ฅ+๐‘ฆ(๐‘–)

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 45: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

๐‘๐‘ฅ+๐‘ฆ ๐‘˜ =

๐‘–=1

๐‘๐‘”

๐‘—=1

๐‘๐‘”

๐‘(๐‘–, ๐‘—) ๐‘˜ = 2,3, โ€ฆ , 2๐‘๐‘”

๐‘˜ = ๐‘– + ๐‘—

Page 46: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Sum entropy

โ– Entropy

๐‘“8 = โˆ’

๐‘–=2

2๐‘๐‘”

๐‘๐‘ฅ+๐‘ฆ(๐‘–) log ๐‘๐‘ฅ+๐‘ฆ ๐‘–

๐‘“9 = โˆ’

๐‘–

๐‘—

๐‘ ๐‘–, ๐‘— log ๐‘ ๐‘–, ๐‘—

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 47: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ–Difference variance

โ–Difference entropy

๐‘“10 = variance of ๐‘๐‘ฅโˆ’๐‘ฆ

๐‘“11 = โˆ’

๐‘–=0

๐‘๐‘”โˆ’1

๐‘๐‘ฅโˆ’๐‘ฆ(๐‘–) log ๐‘๐‘ฅโˆ’๐‘ฆ(๐‘–)

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 48: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

๐‘๐‘ฅโˆ’๐‘ฆ ๐‘˜ =

๐‘–=1

๐‘๐‘”

๐‘—=1

๐‘๐‘”

๐‘(๐‘–, ๐‘—) ๐‘˜ = 0,1, โ€ฆ , ๐‘๐‘” โˆ’ 1

๐‘˜ = ๐‘– โˆ’ ๐‘—

Page 49: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Information measures of correlation 1

โ– Information measures of correlation 2

๐‘“12 =๐ป๐‘‹๐‘Œ โˆ’ ๐ป๐‘‹๐‘Œ1

max ๐ป๐‘‹,๐ป๐‘Œ

๐‘“13 = 1 โˆ’ exp โˆ’2 ๐ป๐‘‹๐‘Œ2 โˆ’ ๐ป๐‘‹๐‘Œ 1/2

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 50: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

๐ป๐‘‹๐‘Œ = โˆ’

๐‘–

๐‘—

๐‘ ๐‘–, ๐‘— log ๐‘ ๐‘–, ๐‘—

๐ป๐‘‹ and ๐ป๐‘Œ are entropy of ๐‘๐‘ฅ and ๐‘๐‘ฆ

๐ป๐‘‹๐‘Œ1 = โˆ’

๐‘–

๐‘—

๐‘ ๐‘–, ๐‘— log ๐‘๐‘ฅ ๐‘– ๐‘๐‘ฆ ๐‘—

๐ป๐‘‹๐‘Œ2 = โˆ’

๐‘–

๐‘—

๐‘๐‘ฅ ๐‘– ๐‘๐‘ฆ ๐‘— log ๐‘๐‘ฅ ๐‘– ๐‘๐‘ฆ ๐‘—

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 51: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ–Maximal correlation coefficient

๐‘“14 = (second largest eigenvalue of ๐‘„)1/2

where ๐‘„ ๐‘–, ๐‘— = ฯƒ๐‘˜๐‘ ๐‘–,๐‘˜ ๐‘(๐‘—,๐‘˜)

๐‘๐‘ฅ ๐‘– ๐‘๐‘ฆ(๐‘˜)

UNIVERSIDADE DE

SรƒO PAULO

14 texture featuresof Haralick et al.

Page 52: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Shape features

โ–Compactness ๐ถ๐‘œ

Perimeter PVertebral area A

UNIVERSIDADE DE

SรƒO PAULO

,4

12P

ACo

โˆ’=

Page 53: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Fourier-descriptor-based feature FDF

Shape features

โˆ’

=

โˆ’=

1

0

2exp)(

1)(

N

n

nkN

jnzN

kZ

k = -N/2+1, โ€ฆ, -1, 0, 1, 2, โ€ฆ, N/2

z(n) = x(n) + j y(n)

n = 0, 1, ..., N-1

UNIVERSIDADE DE

SรƒO PAULO

Page 54: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Fourier-descriptor-based feature FDF

Shape features

+โˆ’=

=

โˆ’=

+โˆ’+

=2

12

2

2

1

1

12

22

|)(|

|)(||)(|

N

Nk

N

kk

kk

N

kZ

kZkZFDF

UNIVERSIDADE DE

SรƒO PAULO

Page 55: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Shape features

โ–Convex deficiency CD

Vertebral area VA

๐ถ๐ท =๐ถ๐ป โˆ’ ๐‘‰๐ด

๐‘‰๐ด

UNIVERSIDADE DE

SรƒO PAULO

Convex hull CH

Page 56: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Shape features

โ– 7 Central invariant moments (Hu)

๐‘€1 = ยต20 + ยต02

๐‘€2 = (ยต20 โˆ’ ยต02)2+4ยต11

2

๐‘€3 = (ยต30 โˆ’ 3ยต12)2+(3ยต21 โˆ’ ยต03)

2

๐‘€4 = (ยต30 + ยต12)2+(ยต21 + ยต03)

2

UNIVERSIDADE DE

SรƒO PAULO

Page 57: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Shape features

๐‘€5

= ยต30 โˆ’ 3ยต12 ยต30 + ยต12 [ ยต30 + ยต122โˆ’3 ยต21 + ยต03

2]+ (3ยต21 โˆ’ ยต03)(ยต21 + ยต03)[3(ยต30 + ยต12)

2โˆ’(ยต21 + ยต03)2]

๐‘€6

= ยต20 โˆ’ ยต02 (ยต30 + ยต12)2 โˆ’ (ยต21 + ยต03)

2

+ 4ยต11 ยต30 + ยต12 ยต21 + ยต03

๐‘€7

= (3ยต21 โˆ’ ยต03)(ยต30 + ยต12)[(ยต30 + ยต12)2โˆ’3(ยต21 + ยต03)

2] โˆ’ (ยต303ยต )( + )[3( + )2 ( + )2]

UNIVERSIDADE DE

SรƒO PAULO

Page 58: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Shape features

ยต00 = ๐‘š00 = ยต

ยต10 = ยต01 = 0

ยต20 = ๐‘š20 โˆ’ ยต๐‘ฅยฒ

ยต11 = ๐‘š11 โˆ’ ยต๐‘ฅ๐‘ฆ

ยต02 = ๐‘š02 โˆ’ ยต๐‘ฆยฒ

ยต30 = ๐‘š30 โˆ’ 3๐‘š20๐‘ฅ + 2ยต๐‘ฅยณ

ยต21 = ๐‘š21 โˆ’๐‘š20๐‘ฆ โˆ’ 2๐‘š11๐‘ฅ + 2ยต๐‘ฅยฒ๐‘ฆ

ยต12 = ๐‘š12 โˆ’๐‘š02๐‘ฅ โˆ’ 2๐‘š11๐‘ฆ + 2ยต๐‘ฅ ๐‘ฆยฒ

ยต03 = ๐‘š03 โˆ’ 3๐‘š02๐‘ฆ + 2ยต๐‘ฆยณ

UNIVERSIDADE DE

SรƒO PAULO

Page 59: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Shape features

๐‘š๐‘๐‘ž

=

๐‘–

๐‘—

๐‘–๐‘๐‘—๐‘ž๐‘–๐‘š๐‘” ๐‘–, ๐‘— , ๐‘, ๐‘ž = 0,1,2, โ€ฆ

๐‘ฅ =๐‘š10

๐‘š00๐‘ฆ =

๐‘š01

๐‘š00

UNIVERSIDADE DE

SรƒO PAULO

Page 60: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Organization of thefeature vector

Coefficientof variation

Skewness Kurtosis ... M7

1 2 3 ... 27

UNIVERSIDADE DE

SรƒO PAULO

Page 61: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Files of features

L1

L2

L3

L4

L5

txt files

UNIVERSIDADE DE

SรƒO PAULO

Page 62: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Inserting thereference classification

โ–Manual addition of the class

โ–Classification according to radiologist andbiopsy

ClassCoefficientof variation

Skewness Kurtosis ... M7

1 2 3 ...

27

NormalVCF

Benign Malignant

UNIVERSIDADE DE

SรƒO PAULO

Page 63: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Feature selection

โ– Software WEKA

โ–Wrapper method for feature selection

โžข kNN with k = 1, 3, ..., 13

โžข Naรฏve Bayes

โžข RBF network

โ–Best first as search method

โžข Greedy search for the best subset of features

UNIVERSIDADE DE

SรƒO PAULO

Page 64: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Classification

โ– Software WEKA

โ–Classifiers:

โžข k-nearest neighbor: k = 1, 3, 5, 7, 9, 11, 13

โžข Naรฏve Bayes

โžข RBF network

โ– Stratified 10-fold cross-validation

โžข 9 folds for training, 1 fold for test

UNIVERSIDADE DE

SรƒO PAULO

Page 65: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Clinical Classes

โ–VCF vs Normal

โ–Benign VCF vs Malignant VCF

โ–Malignant VCF, Benign VCF, and Normal

UNIVERSIDADE DE

SรƒO PAULO

Page 66: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Validation

โ–Confusion Matrix

โžข Sensitivity

โžข Specificity

โžข AUROC

โžข% of correct classification

UNIVERSIDADE DE

SรƒO PAULO

Page 67: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

๐ด๐‘ง and p-values UNIVERSIDADE DE

SรƒO PAULO

โ– * for 0.01 โ‰ค p < 0.05

โ– ** for 0.001 โ‰ค p < 0.01

โ– *** for p < 0.001

โ– p-values obtained using Wilcoxon rank-sum test

โ– NS indicates no significant difference

โ– NA indicates that ๐ด๐‘ง could not be obtained

Page 68: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

๐ด๐‘ง and p-values UNIVERSIDADE DE

SรƒO PAULO

Benign VCF versus Malignant VCF

All VCFs together versus Normal

Feature Significance ๐‘จ๐’› Significance ๐‘จ๐’›๐ถ๐‘‰ NS 0.580 *** 0.751

๐‘†๐‘˜๐‘’๐‘ค *** 0.861 * 0.549

๐พ๐‘ข๐‘Ÿ๐‘ก *** 0.824 NS 0.532

๐ป1 *** 0.849 NS 0.625

๐ป2 *** 0.866 * 0.661

๐ป3 NS 0.480 NS 0.629

๐ป4 *** 0.874 NS 0.642

๐ป5 *** 0.844 * 0.577

๐ป6 *** 0.829 *** 0.731

๐ป7 *** 0.871 NS 0.640

๐ป8 *** 0.854 ** 0.620

๐ป9 *** 0.858 *** 0.647

๐ป10 *** 0.871 ** 0.674

Page 69: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

๐ด๐‘ง and p-values UNIVERSIDADE DE

SรƒO PAULO

Benign VCF versus Malignant VCF

All VCFs together versus Normal

Feature Significance ๐‘จ๐’› Significance ๐‘จ๐’›

H11 *** 0.868 ** 0.632

H12 *** 0.731 NS 0.524

H13 *** 0.854 *** 0.614

H14 NS 0.566 NS 0.462

Co *** 0.722 *** 0.864

FDF *** 0.837 NS 0.449

CD *** 0.700 *** 0.881

M1 NS 0.567 *** 0.964

M2 NS 0.518 *** 0.932

M3 ** 0.655 * 0.887

M4 * 0.617 NS 0.936

M5 NS 0.389 NS NA

M6 NS 0.480 NS 0.498

M7 NS 0.538 NS NA

Page 70: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

๐ด๐‘ง and p-values UNIVERSIDADE DE

SรƒO PAULO

Page 71: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Mean and standard deviation of features UNIVERSIDADE DE

SรƒO PAULO

Page 72: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Mean and standard deviation of features UNIVERSIDADE DE

SรƒO PAULO

โ–Mean skewness of malignant VCFs is higherthan that for benign VCFs

โžข T1 signals are distributed more on the lowerside of the histogram for malignant VCFs

โ–๐ป6 and ๐ป7 show large differences in their mean values for malignant VCFs versus benign VCFs

Page 73: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Mean and standard deviation of features UNIVERSIDADE DE

SรƒO PAULO

Page 74: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Feature selection UNIVERSIDADE DE

SรƒO PAULO

Page 75: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Feature selection UNIVERSIDADE DE

SรƒO PAULO

โ– k-NN did not select the gray-level featuresfor benign vs malignant VCFs

โžข ๐น๐ท๐น, ๐‘€5, ๐ป10, and ๐ป13were selected at least three times

โ–CV is statistically significant for all VCFs vsnormal vertebral bodies and was selectedfor all classifiers

Page 76: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Feature selection UNIVERSIDADE DE

SรƒO PAULO

โ–Various texture features were selected for both types of classification

โ–Naรฏve Bayes selected the highest number offeatures for both types of classification

Page 77: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Classification UNIVERSIDADE DE

SรƒO PAULO

Classifier ACC rate % AUROC

k-NN

k = 7 82.4 0.84

k = 9 81.4 0.90

k = 11 84.3 0.90

k = 13 84.3 0.90

Naรฏve Bayes

RBF network

85.3 0.92

78.4 0.86

Classifier ACC rate % AUROC

k-NN

k = 7 90.1 0.95

k = 9 89.0 0.92

k = 11 89.0 0.92

k = 13 89.5 0.94

Naรฏve Bayes

RBF network

90.6 0.97

91.1 0.94

โ– Benign vs malignantVCFs

โ– All VCFs vs normal vertebral bodies

Page 78: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Classification UNIVERSIDADE DE

SรƒO PAULO

โ–RBF network classifier for benign vsmalignant VCFs

โžข ACC rate was the lowest obtained

โžข AUROC is only better than that of 7-NN

โ–RBF network classifier for all VCFs vs normal vertebral bodies

โžข ACC rate is the highest obtained

Page 79: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Classification UNIVERSIDADE DE

SรƒO PAULO

โ–AUROC for classification of all VCFs together vs normal vertebral bodies is at least 0.92

โ–AUROC of the naรฏve Bayes classifier is 0.97 for this purpose

โžข Better than the previous study using only shapefeatures in which AUROC was 0.945

โ– This shows the importance of texture andgray-level features for this purpose

Page 80: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Classification UNIVERSIDADE DE

SรƒO PAULO

โ–AUROC for classification of benign vs malignant VCFs is 0.92 for naรฏve Bayes

โžข Better than the previous study in which thehighest AUROC was 0.91 for 3-NN

โ– In a previous study using only shapefeatures the highest AUROC was 0.78

โžข This shows the importance of texture and gray-level features for this purpose

Page 81: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Benign VCFs, malignant VCFs, and normal vertebral bodies UNIVERSIDADE DE

SรƒO PAULO

Predicted classification True classification

Malignant VCFs Benign VCFs Normal vertebral bodies

39 5 5 Malignant VCFs

13 35 5 Benign VCFs

4 1 84 Normal vertebral bodies

โ€ข Features selected: โ€ข CV, Skew, H2, H3, H5, H6, H8, H9, H11, H12,

H13,H14,Co, FDF, CD, M1, M3, and M7

โ€ข Weighted average AUROC of 0.94

โ€ข ACC rate of 82.7%

Page 82: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ–Manual segmentation of the vertebral bodies

โžข Automatic segmentation methods could lead to the realization of a clinically useful CAD system

โ– Individual and separate analysis of thevertebral bodies ignores importantinformation outside their regions

Limitations of the study UNIVERSIDADE DE

SรƒO PAULO

Page 83: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– The use of only the median sagittal slice

โ– Some lateral VCFs may be misclassified

โ– Extension of segmentation and feature extraction methods to 3D is desirable

Limitations of the study UNIVERSIDADE DE

SรƒO PAULO

Page 84: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ–Analysis of only T1-weighted MRI

โžข Benign VCFs

โ€ข isointense vertebra in T2-weighted and T1-weighted MRI after gadolinium contrast

โžขMalignant VCFs

โ€ข heterogeneous or high signal in T2-weighted and in

T1-weighted MRI after gadolinium contrast

Limitations of the study UNIVERSIDADE DE

SรƒO PAULO

Page 85: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ–Most of the features presented are important for both types of VCF classification

โ– For benign vs malignant VCFs

โžข AZ values of texture and gray-level features are higher than those shape features

โ– For all VCFs vs normal vertebral bodies

โžข AZ values of shape features are higher than those of texture and gray-level features

Conclusion UNIVERSIDADE DE

SรƒO PAULO

Page 86: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– The features FDF and CV follow the

opposite trend

โ– The naรฏve Bayes method was the best classifier in both types of classification

โ– The proposed methods are promising

for CAD of VCFs

Conclusion UNIVERSIDADE DE

SรƒO PAULO

Page 87: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Future works:

โžข Evaluate our methods with the inclusion of anautomatic segmentation method

โžข Extend the methods to 3D analysis of vertebral bodies

Conclusion UNIVERSIDADE DE

SรƒO PAULO

Page 88: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

โ– Sรฃo Paulo Research Foundation (FAPESP)

โžข 2014/12135-0 and 2015/08778-6

โ– National Council of Technological and ScientificDevelopment (CNPq)

โ– Natural Sciences and Engineering Research Council of Canada

โ– Ph.D students

โžข Rafael de Menezes-Reis

โžข Faraz Oloumi

AcknowledgmentUNIVERSIDADE DE

SรƒO PAULO

Page 89: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Feature selection: benign vs malignant

VCFs

UNIVERSIDADE DE

SรƒO PAULO

Featurek-NN Naรฏve

Bayes

RBF

Networkk = 7 k = 9 k = 11 k = 13

X X

X X X

X X X X

X X X

X X X

X

X

X X X X X

X X

X X

X X X X X X

X

X

Page 90: Classification of Benign and Malignant Vertebral ...people.ucalgary.ca/~ranga/enel697/VCF_CAD.pdfClassification of Benign and Malignant Vertebral Compression Fractures in Magnetic

Feature selection: all VCFs

vs normal vertebral bodies

UNIVERSIDADE DE

SรƒO PAULO

Featurek-NN Naรฏve

Bayes

RBF

Networkk = 7 k = 9 k = 11 k = 13

X X X X X X

X

X

X

X X

X

X

X

X

X

X X X X

X

X X X X X X

X X X

X X X X

X X X X