An Interactive Segmentation Approach

Using Color Pre-processing

Marisol Martinez EscobarPh.D Candidate

Major Professor: Eliot WinerDepartment of Mechanical Engineering &

Human-Computer Interaction

December 9, 2009

Outline• Introduction • Background

– Segmentation methods– Colorization methods

• Methodology– DICOM colorization method– Segmentation approach

• Results– Statistical analysis of results– Comparison between grayscale & colorization

• Conclusions• Future Work

Introduction

MRI Hand Scan*University of Exter http://centres.exeter.ac.uk/pmrrc/gallery/hand/hand.html

First X-ray*Wikipedia X-rayhttp://en.wikipedia.org/wiki/X-ray

Introduction

• Medical Images– Diagnosis, planning, treatment

and education

• Medical Scan– Computed Tomography (CT)

and Magnetic Resonance Imaging (MRI)

– Non-invasive

Medical Data

• Stored as Hounsfield Units (HU)– Tissue density relative to water– Usually ranges -1000 HU (air) – +1000 HU (bone)

• Windowing Process – Reduces HU values to a 0-255 range

Tissue Value (HU)

Fat -90

Water 0

Muscle +44

Bone +1005

255

0

-1000 +1000

Width

Center

HU

Inte

ns

ity

Segmentation• Delineation of regions of interest from an image • Complex process since tumors have different

shapes, sizes, tissue densities, and locations

Segmentation Approches• Classical Methods (Hojjatoleslami et al 1998, Pole

et al, Zhang et al 2001)

• Advanced Methods (Vincken et al 1997, Xu et al 2000, Kaus et al 2004)

• Hybrid Methods (Gibou et al 2005, Atkins et al 1998).

Limitation in Segmentation Approaches

Color Segmentation

• Classical techniques (Lin et al), advanced techniques (Chent et al, Verikas et al) Hybrid approaches (Cremers et al)

• Limitations– Not applied for internal tumor segmentation– RGB source files– Mostly applied to non-medical segmentation

Colorization• Process of adding color to a grayscale image by the use of a computer

– Add color channels to the image from 1 channel to 3 channels– Possible number of colors from 256 to 16 million.– No unique solution

• Adding information can improve segmentation

Examples of Colorization• User initial paint (Levin et al, Tzeng et al )• Initial color source (Welsh et al 2002)• Color seed (Takahiko et al)

Research Issues

• Improve the accuracy of tumor segmentation from medical image data using color pre-processing and interactive user inputs.

• To provide an easy to use tool that will aid in the Medical field

Methodology Development

Region of interest selection and colorization

Seed selection for first slice and segmentation

Post-processing and interactive adjustements

Colorization

• User selects region of interest• The region of interest determines the HU

range

minmax HUHUHUrange HU Min

HU Max

Midpoint

Red Green Blue255

0 0

0

255

255

0 0

0

Colorization

rangeHU

ueHUpixelValP

0

2255

25520.1Re

Blue

PGreen

Pd

25.0255

25.01255

0Re

PBlue

PGreen

d

Segmentation• User selects a seed• Segmentation is based on

distance and color

– Tp = pixel threshold,– C = Color component,– D = Distance component– R = search radius

R

DCTp

Segmentation

• Color Component

• Distance Component

255

2/1222bbggrr APAPAP

APC

2/122yyxx SPSP

SPD

Segmentation

pRCR

6

123 321 CCCC

ROI

Seed

RMAX

Post-processing

• Morphological Operations

Interactive Adjustements

• 2D Textures– Array of 512x512 sent to the GPU

• Allows for real time visualization of the results

• Allows tweaking of parameters

Interface Framework

• Open source libraries– DCMTK– OpenGL– VTK– VRJuggler– wxWidgets

Medical Desktop

Visualization Segmentation Collaboration

Transverse, Sagittal, and Coronal 2D Views

Volume Rendering

Pseudo-coloring

Windowing

Connection to Virtual Reality Environment

Segmentation tab

• Sliders• Apply all• Plenty of screenshots

Other features

Test Cases Description

• 10 different test cases with different levels of difficulty

• Several runs of each test cases

Results

• Gold Standard– Two radiologists manually segmented the

results• False positive and false negative were

calculated

%100

)(

)(x

RV

RAVAVFP

Results

Results – Colorization

• Easy cases have low FN and FP because of different tissue densities

• 10 out of the 20 test cases gave false positives of 25% or less, and 10 out of the 20 test runs gave false negatives of 25% or less.

Results- Cases A

• Low FN and FP because of difference between tumor and healthy tissues

Results Cases B & C

• Low FN in calcified cases because algorithm selects tumor tissues correctly

• High FN because tumor tissues that vary are not selected

Comparison Grayscale vs. Color• Same test cases • FP of up to 52% on the easy cases up to 284% on the difficult

cases• FN of up to 14% on the easy cases and up to 99% on the

difficult cases.• Colorization prior to segmentation yields better results

Grayscale Color

Test Case#

Level

FP FN FP FN

1A

21.8807 14.255 11.0837 14.0453

5B

23.6672 93.077 40.3981 31.7252

6B

224.641 99.545 18.8161 30.1461

7C

19.2508 92.218 5.9099 57.6397

Summary Results

• Adding color to the original HU values improves segmentation– Half of the test cases show less than 25% FP

and FN for a simple thresholding technique – Same grayscale methods show up to 284% FP

and 99% FN

Future Work

• Different and more complex segmentation algorithms using color information

• Different colorization methods • Shaders to increase the speed of the results• Improve the user interface.

Thank You