medical imaging projects daniela s. raicu, phd assistant professor email: [email protected] lab...

Medical Imaging Projects

Daniela S. Raicu, PhDAssistant Professor

Email: [email protected] URL: http://facweb.cs.depaul.edu/research/vc/

http://facweb.cs.depaul.edu/research/vc/

2MedIX REU Program, Summer 2007

IMP & MediX Labs @ DePaul

Faculty: GM. Besana, L. Dettori, J. Furst, G. Gordon, S. Jost, D. Raicu, N. Tomuro

CTI Students: W. Horsthemke, C. Philips, R. Susomboon, J. Zhang E. Varutbangkul, S.G. Valencia

IMP Collaborators & Funding Agencies• National Science Foundation (NSF) - Research Experience for Undergraduates (REU) • Northwestern University - Department of Radiology, Imaging Informatics Section• University of Chicago – Medical Physics Department• Argonne National Laboratory - Biochip Technology Center• MacArthur Foundation


Outline

Medical Imaging and Computed Tomography

Soft Tissue Segmentation in Computed Tomography Project 1: Region-based classification Project 2: Texture-based snake approach

Content-based Image Retrieval and Annotation Project 3: Lung Nodule Retrieval based on image content and

radiologists’ feedback Project 4: Associations discovery between image content and

radiologists’ assessment


The study of medical imaging is concerned with the interaction of all forms of radiation with tissue and the development of appropriate technology to extract clinically useful information from observation of this technology.

What is Medical Imaging (MI)?

X-Ray fMRICT


_______________________________________________

Computed Tomography (CT)

• G. Hounsfield (computer expert) and A.M. Cormack (physicist) (Nobel Prize in Medicine in 1979)

• CT overcomes limitations of plain radiography

• CT doesn’t superimpose structures (like X-ray)

• CT is an imaging based on a mathematical formalism that states that if an object is viewed from a number of different angles than a cross-sectional image of it can be computed (reconstructed)


Stages of construction of a voxel dataset from CT data(a) CT data capture works by taking many one dimensional projections through a slice (scanning)(b) CT reconstruction pipeline

CT Data


_______________________________________________

CT – Data Acquisition

Slice-by-slice acquisition• X-ray tube is rotating around patient to acquire a slice• patient is moved to acquire the next sliceVolume acquisition• X-ray tube is moving continuously along a spiral (helical) path and the data is acquired continuously


(a) slice-by-slice scanning

(b) Spiral (volume) scanning

CT – Data Acquisition


CT – SPIRAL SCANNING

• a patient is moved 10mm/s (24cm / single scan)• slice thickness: 1mm-1cm• faster than slice-by-slice CT• no shifting of anatomical structures• slice can be reconstructed with an arbitrary orientation with (a single breath) volume

CT multi-slice systems:• parallel system of detectors • 4/8/16 slices at a time• generates a large data of thin slices• better spatial resolution ( better reconstruction)


Understanding Visual Information: Technical, Cognitive and Social Factors

CT - DATA PROCESSING

CT numbers (Hounsfield units) HU:• computed via reconstruction algorithm (~tissue density/ X-ray absorption)• most attenuation (bone)• least attenuation (air)• blood/calcium increases tissue density


Understanding Visual Information: Technical, Cognitive and Social Factors

Relationship between CT numbers and brightness level

CT - DATA PROCESSING


CT - IMAGE DISPLAY

Thoracic image:a) width 400HU/level 40HU (no lung detail is seen)

b) width 1000HU/level –700HU (lung detail is well seen; bone and soft tissue detail is lost)

Human eye can perceive only a limitedrange gray-scale values


CT Medical Imaging (MI)@ CTI

Filtering

Correction

Registration

Segmentation

Analysis

Visualization Classification Retrieval

Projects 1&2: Texture-based soft-tissue segmentation

Projects 3&4: Content-based medical image retrieval and annotation


Outline


Soft Tissue Segmentation in Computed Tomography Project 1: Region-based classification approach Project 2: Texture-based snake approach





Goal: context-sensitive tools for radiology reportingApproach: pixel-based texture classification

Soft-tissue Segmentation in Computed Tomography

Pixel Level Texture Extraction

Pixel Level Classification Organ

Segmentation

1 2, , kd d d _tissue label


Pixel-based texture extraction:

Soft-tissue Segmentation in Computed Tomography

Pixel Level Texture Extraction

1 2, , kd d d

Challenges: Storage:

Input: 0.5+ terabyte of raw data dispersed over about 100K+ images Output: 90+ terabytes of low-level features in a 180 dimensional feature space

Compute: 24 hours of compute time = 180 features for a single image on a modern 3GHz workstation

Input Patient Data Characteristics: hundreds of images per patient image spatial resolution: 512 x512 image gray-level resolution: 212

Output Data Characteristics: low-level image features (numerical descriptors) k=180 Haralick texture features per pixel (9 descriptors x4 directions x5 displacements)


Project 1: Challenges and opportunities

Calculate image features at region-level instead of pixel-level Include Gabor features in the feature extraction phase in addition to the co-occurrence texture features Explore different approaches for region classification in addition to the decision tree approach

Current Implementation: Matlab

Stack of CT slices Image Partitioning

kfeature

feature

feature

2

1

Feature Extraction

labeltissue __

Region Classification


Liver Segmentation ExampleJ.D. Furst, R. Susomboon, and D.S. Raicu, "Single Organ Segmentation Filters for Multiple Organ Segmentation", IEEE 2006 International Conference of the Engineering in Medicine and Biology Society (EMBS'06)

Region growing at 70% Region growing at 60% Segmentation Result

Original Image Initial Seed at 90% Split & Merge at 85% Split & Merge at 80%


Snake Application Demo

Next figures are demonstrated how to automatically classify the CT images of heart and liver.

Soft-tissue Segmentation in

Computed Tomography


Demo For HEART

There are 4 main menu to operate this application.

OPEN:To open a new Image.

SEGMENT:To automatically segment the region of interest organTEXTURE:

To calculate the texture models: co-occurrence/run-length

CLASSIFICATION:To automatically classify the segmented organ


HEART: Segmentation

The application allows users

to changeSnake/

Active contouralgorithm

parameters


HEART: Segmentation (cont.)

Button is clicked

User selects points

around the region of interest


HEART: Segmentation (result)

Show segmented

organ

If the user likes the result of the segmentation,then the user will go to the classification step


HEART: Classification

Selection of texture models:Co-occurrence,

Run-length,Or Combine both models

Texture features corresponding to the selected texture model are calculated and shown here


HEART: Classification Result

Results are shown as follows.

Predicted organ: Heart

Probability:0.86And also rule which is usedto predict that

this segmentedorgan is HEART


Demo For LIVER

Start application by open and load the image.


LIVER: Segmentation

The application allows users

to changeSnake/

Active contouralgorithm

parameters


LIVER: Segmentation (cont.)

Segmentation Button is clicked

User selects pointsaround the region of

interest


LIVER: Segmentation Result

Show segmented

organ

If user is satisfied with the result, then it will go to the classification step


LIVER: Classification

Select texture models:

Co-occurrence,Run-length,

Or Combine both models

Texture features is calculated for the selected model


LIVER: Classification Result

Results are shown as follows.

Predicted organ: Liver

Probability:1.00And also rule which is usedto predict that

this segmentedorgan is LIVER



Calculate texture image features at the pixel level instead of using the gray-levels Apply snake on the texture features Investigate different ways to objectively compare two segmentation algorithms, in particular the snake and the classification-based approach

Current Implementation: Matlab


Outline







-

Definition of Content-based Image Retrieval:Content-based image retrieval is a technique for retrieving images on the basis of automatically derived image features such as texture and shape.

Content-based medical image retrieval (CBMS)

systems

Applications of Content-based Image Retrieval: Teaching Case-base reasoning Evidence-based medicine


Feature Extraction

Similarity Retrieval

Image Features

[D1, D2,…Dn]Image Database

Query Image

Query Results

Feedback Algorithm

User Evaluation

Diagram of a CBIR


CBIR as a tool for lookup and reference

• Case Study: lung nodules retrieval– Lung Imaging Database Resource for Imaging Research

http://imaging.cancer.gov/programsandresources/InformationSystems/LIDC/page7

– 29 cases, 5,756 DICOM images/slices, 1,143 nodule images

– 4 radiologists annotated the images using 9 nodule characteristics: calcification, internal structure, lobulation, malignancy, margin, sphericity, spiculation, subtlety, and texture

• Goals:– Retrieve nodules based on image features:

• Texture, Shape, and Size – Find the correlations between the image features and the

radiologists’ annotations


Examples of nodule images


CBIR as a tool for lung nodule lookup and reference

Low-level feature extraction:


Nodule Characteristics

– Calcification• (1. Popcorn, 2. Laminated, 3. Solid,

4. Non-Central, 5. Central, 6. Absent)– Internal Structure

• (1. soft tissue, 2. fluid, 3. fat, 4. air)– Subtlety

• (1. extremely subtle,..................., 5. obvious)– Sphericity

• (1. Linear, 2. ......, 3. Ovoid, 4. ....., 5. Round)– Texture

• (1. Non-Solid, 2. ....., 3. Part Solid, 4. ......., 5. Solid)


Nodule Characteristics

– Margin• (1. Poorly, ......................., 5. Sharp)

– Lobulation• (1. Marked, ....................., 5. No Lobulation)

– Spiculation• (1. Marked, ....................., 5. No Spiculation)

– Malignancy• (1. Highly Unlikely for Cancer, ..............., 5. Highly Suspicious for Cancer)


Choose a nodule


Choose an image feature& a similarity measure

M. Lam, T. Disney, M. Pham, D. Raicu, J. Furst, “Content-Based Image Retrieval for Pulmonary Computed Tomography Nodule Images”, SPIE Medical Imaging Conference, San Diego, CA, February 2007

44MedIX REU Program, Summer 2007Retrieved Images



Calculate co-occurrence texture features at the local level instead of global level Incorporate shape and size features in the retrieval process in addition to texture features Integrate radiologists’ assessments/feedback into the retrieval process Investigate different approaches for retrieval in addition to similarity measures Report the retrieval results with a certain confidence level (probability) instead of just a binary output (similar/not similar)

Current implementation: C#Available Open Source at: http://brisc.sourceforge.net/


Outline







Calcification

Lobulation

InternalStructure

Malignancy

Margin

Spiculation

Sphericity

Texture

Subtlety

Characteristics

(Constant) 5.377275 1.64E-54gabormean_1_2 -0.02069 7.80E-07MinIntensityBG 0.003819 3.30E-82Energy -28.5314 3.31E-12gabormean_0_1 -0.00315 5.80E-14IntesityDifference 0.000272 0.003609inverseVariance 6.317133 3.41E-05gabormean_1_1 0.009743 0.000259gabormean_2_1 -0.00667 5.79E-05Correlation -0.39183 5.67E-05clusterTendency 5.16E-06 0.000131ConvexPerimeter -0.00291 0.023032

Adj-R2 = 0.990

Regression Coefficients p-value

Estimated Malignancy = 5.377275 - 0.02069 gabormean_1_2 + 0.003819 MinIntensityBG - 28.5314 energy - 0.00315 gabormean_0_1 + 0.000272 IntesityDifference + 6.317133 inverseVariance + 0.009743 gabormean_1_1 - 0.00667 gabormean_2_1 - 0.39183 correlation + 5.16E-06 clusterTendency - 0.00291 ConvexPerimeter

F-value = 963.560p-value = 0.000

Associations between image content and semantics



Investigate other approaches for finding the associations between image features and radiologists’ assessment in addition to logistic regression and decision trees

from image content to semantics from semantics to semantics from image features and semantics to semantics

Create GUIs to display examples of images for each semantic concept Investigate how the current associations discovery approaches apply to mammography assessment (Northwestern project)

Current implementation: Matlab, Weka, SPSS


Questions?

Thank you!

medical imaging projects daniela s. raicu, phd assistant professor email: [email protected] lab...

Documents

slice ct

xray ct

eduresearchvcmedix reu

slice patient

ct data acquisitionslice

computed tomography

ct spiral scanning

ct datact data capture