feature-preserving artifact removal from dermoscopy images
DESCRIPTION
Feature-preserving Artifact Removal from Dermoscopy Images. Howard Zhou 1 , Mei Chen 2 , Richard Gass 2 , James M. Rehg 1 , Laura Ferris 3 , Jonhan Ho 3 , Laura Drogowski 3. 1 School of Interactive Computing, Georgia Tech 2 Intel Research Pittsburgh - PowerPoint PPT PresentationTRANSCRIPT
Feature-preserving Artifact Removal from Dermoscopy Images
Howard Zhou1, Mei Chen2,
Richard Gass2, James M. Rehg1,
Laura Ferris3, Jonhan Ho3, Laura Drogowski3
1School of Interactive Computing, Georgia Tech2Intel Research Pittsburgh
3Department of Dermatology, University of Pittsburgh
Skin cancer and melanoma Skin cancer : most common of all cancers
Skin cancer and melanoma Skin cancer : most common of all cancers
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Basal Cell Carcinoma
Hemangioma
Compound nevus Seborrheic keratosis
Skin cancer and melanoma Skin cancer : most common of all cancers Melanoma : leading cause of mortality
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Basal Cell Carcinoma
Hemangioma
Compound nevus Seborrheic keratosis
Melanoma
Melanoma
Skin cancer and melanoma Skin cancer : most common of all cancers Melanoma : leading cause of mortality Early detection significantly reduces mortality
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Basal Cell Carcinoma
Hemangioma
Compound nevus Seborrheic keratosis
Melanoma
Melanoma
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Clinical ViewDermoscopy view
Dermoscopy Skin surface microscopy Improve diagnostic accuracy by 30% for trained,
experienced physicians Requires 5 or more years of experience Computer-aided diagnosis (CAD) to assist less
experienced physicians
Clinical view Dermoscopy view
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Artifacts in dermoscopy images
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Hair lesion boundary
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Hair lesion boundary
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Hair lesion boundary Hair pigmented network
Previous work Hair detection and tracing
Fleming et al. 1998 Thresholding and averaging
“DullRazor”, Tim K. Lee et al. 1997 Schmid et al. 2003
Thresholding and inpainting Paul Wighton et al. 2008 (right here in the
conference)
Detection: thresholding Removal: morphological
operations
Schmid et al.
Schmid et al. Thresholding false
detection Accidental removal of
diagnostic features
Schmid et al. 2003
ThresholdingThresholding
Schmid et al. Morphological operation
(neighbors’ average) blurring
Morphological operationMorphological operation
Schmid et al. 2003
Feature-preserving Feature-preserving artifact removal artifact removal (FAR)(FAR)
Detection: Explicit curve modeling
Removal: Exemplar-based inpainting
Our method (FAR)Schmid et al. 2003
Our method (FAR)
FAR Curve modeling
more accurate hair detection
ThresholdingThresholding Curve modelingCurve modeling
Schmid et al. 2003
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
ThresholdingThresholding Curve modelingCurve modelingMorphological operationMorphological operation Exemplar-based inpaintingExemplar-based inpainting
Schmid et al. 2003
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
ThresholdingThresholding Curve modelingCurve modelingMorphological operationMorphological operation Exemplar-based inpaintingExemplar-based inpainting
Schmid et al. 2003
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
System overview
Threholding
Curve fitting & intersection analysis
Exemplar patches
Exemplar-based inpainting
Dermoscopy image
Hair removed
Luminance difference dark thin structure
Line points
Line segments
Parameterized curves
Mask
Line points linking
Line points Detection
Input dermoscopy image
Enhancing dark-thin structure Luminosity channel in CIE L*u*v* Difference b/a morphological closing
[ Schmid-Saugeona et al. 2003, “Towards a computer-aided diagnosis system for pigmented skin lesions” ]
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
Curve B(t)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Curve B(t)Cross section
n(t)
f(x)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Cross section
n(t)
f(x)
Curve B(t)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Cross section
n(t)
f(x)
f’ = 0|f’’| large
Curve B(t)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Cross section
n(t)
f(x)
f’ = 0|f’’| large
Curve B(t)
n(t) : direction ┴ curve B(t)
eigenvector corresponding to the maximum absolute eigenvalue of the local Hessian
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
Linking line points
Link the neighboring points to get line segments (sets of ordered line points)
Fitting polynomial curves
A set of ordered points Pi s
P
Fitting polynomial curves
A set of ordered points Pi s Parametric curveP
Fitting polynomial curves
A set of ordered points Pi s Parametric curve
B(t)
P
Fitting polynomial curves
B(t)
P A set of ordered points Pi s Parametric curve
Minimize sum of squared distance
Fitting polynomial curves
A set of ordered points Pi s Parametric curve
Minimize sum of squared distance
Linear system (can be solved by Gaussian elimination)
B(t)
P
Handling hair intersection
Configurations:
Hair intersection Line segments Intersection analysisLink Line segment
……
Before curve fitting and linking
Line segments
After curve fitting and linking
Parameterized curves
After curve fitting and linking
Parameterized curves
After curve fitting and linking
Hair mask
After curve fitting and linking
Hair mask
Exemplar-based inpainting
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
[ Image courtesy of Criminisi et al. 2003 ]
Fill in with patches from the image itself Patch ordering structure propagation.
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
Before FAR
After FAR
More results Explicit curve
modeling Exemplar-based
inpainting
Our method (FAR)Schmid et al. 2003
More results Explicit curve
modeling Exemplar-based
inpainting
Our method (FAR)Schmid et al. 2003
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
When is FAR not suitable ?
Oops, too much hair!
When is FAR not suitable ?
Too much hair Makes explicit
modeling difficult
Schemid et al. 2003 (DullRazor) Our method (FAR)
Conclusion
Automatic system that detects and removes curvilinear artifacts
Feature-preserving artifact removal: Explicit curve modeling Exemplar-based
inpainting
Future work
Speed up exemplar-based inpainting
Future work
Speed up exemplar-based inpainting Handle hair with arbitrary intensity
Future work
Speed up exemplar-based inpainting Handle hair with arbitrary intensity Extend to removing air bubbles
Questions ?
Additional results
Our method (FAR)Original Dermoscopy image