midag@unc medial object shape representations for image analysis & object synthesis stephen m....

60
MIDAG@UNC MIDAG@UNC Medial Object Shape Medial Object Shape Representations Representations for Image Analysis & Object for Image Analysis & Object Synthesis Synthesis Stephen M. Pizer Stephen M. Pizer Kenan Professor Kenan Professor Medical Image Display & Analysis Group Medical Image Display & Analysis Group University of North Carolina, USA University of North Carolina, USA Credits: Many on MIDAG, especially Credits: Many on MIDAG, especially Daniel Fritsch, Andrew Thall, George Daniel Fritsch, Andrew Thall, George Stetten, Stetten, Paul Yushkevich Paul Yushkevich

Post on 20-Dec-2015

225 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Medial Object Shape Medial Object Shape RepresentationsRepresentations

for Image Analysis & Object Synthesisfor Image Analysis & Object Synthesis

Medial Object Shape Medial Object Shape RepresentationsRepresentations

for Image Analysis & Object Synthesisfor Image Analysis & Object Synthesis

Stephen M. PizerStephen M. Pizer

Kenan ProfessorKenan Professor

Medical Image Display & Analysis GroupMedical Image Display & Analysis Group

University of North Carolina, USAUniversity of North Carolina, USA

Credits: Many on MIDAG, especiallyCredits: Many on MIDAG, especially

Daniel Fritsch, Andrew Thall, George Stetten, Daniel Fritsch, Andrew Thall, George Stetten,

Paul YushkevichPaul Yushkevich

Stephen M. PizerStephen M. Pizer

Kenan ProfessorKenan Professor

Medical Image Display & Analysis GroupMedical Image Display & Analysis Group

University of North Carolina, USAUniversity of North Carolina, USA

Credits: Many on MIDAG, especiallyCredits: Many on MIDAG, especially

Daniel Fritsch, Andrew Thall, George Stetten, Daniel Fritsch, Andrew Thall, George Stetten,

Paul YushkevichPaul Yushkevich

Page 2: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Medial Object Shape Medial Object Shape RepresentationsRepresentations

for Image Analysis & Object Synthesisfor Image Analysis & Object Synthesis

Medial Object Shape Medial Object Shape RepresentationsRepresentations

for Image Analysis & Object Synthesisfor Image Analysis & Object Synthesis

Page 3: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

WhatWhat shape shape representation is forrepresentation is for

WhatWhat shape shape representation is forrepresentation is for

Analysis from imagesAnalysis from images Extract the kidney-shaped objectExtract the kidney-shaped object Register based on the pelvic bone shapesRegister based on the pelvic bone shapes Extract shape information w/o modelExtract shape information w/o model

SynthesisSynthesis Design the objectDesign the object Deform the object, with physical realismDeform the object, with physical realism

Shape science Shape science Shape and biologyShape and biology Shape-based diagnosisShape-based diagnosis

Analysis from imagesAnalysis from images Extract the kidney-shaped objectExtract the kidney-shaped object Register based on the pelvic bone shapesRegister based on the pelvic bone shapes Extract shape information w/o modelExtract shape information w/o model

SynthesisSynthesis Design the objectDesign the object Deform the object, with physical realismDeform the object, with physical realism

Shape science Shape science Shape and biologyShape and biology Shape-based diagnosisShape-based diagnosis

Page 4: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

WhatWhat shape shape representation is forrepresentation is for

WhatWhat shape shape representation is forrepresentation is for

Analysis from imagesAnalysis from images Extract the kidney-shaped objectExtract the kidney-shaped object

Register based on the pelvic bone shapesRegister based on the pelvic bone shapes

Analysis from imagesAnalysis from images Extract the kidney-shaped objectExtract the kidney-shaped object

Register based on the pelvic bone shapesRegister based on the pelvic bone shapes

Page 5: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

WhatWhat shape shape representation is forrepresentation is for

WhatWhat shape shape representation is forrepresentation is for

SynthesisSynthesis Design the objectDesign the object Deform the object, with physical realismDeform the object, with physical realism

SynthesisSynthesis Design the objectDesign the object Deform the object, with physical realismDeform the object, with physical realism

Page 6: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

WhatWhat shape shape representation is forrepresentation is for

WhatWhat shape shape representation is forrepresentation is for

Shape science Shape science Shape and biologyShape and biology Shape-based diagnosisShape-based diagnosis

Shape science Shape science Shape and biologyShape and biology Shape-based diagnosisShape-based diagnosis

Brain structures (Gerig)

Page 7: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape SciencesShape SciencesShape SciencesShape Sciences

GeometryGeometry The spatial layout: via primitivesThe spatial layout: via primitives

LandmarksLandmarks Boundary places and orientationsBoundary places and orientations Medial places, figural sizes and orientationsMedial places, figural sizes and orientations Space itselfSpace itself

StatisticsStatistics The average shapeThe average shape Modes of variation from the averageModes of variation from the average

Computer GraphicsComputer Graphics Image AnalysisImage Analysis

GeometryGeometry The spatial layout: via primitivesThe spatial layout: via primitives

LandmarksLandmarks Boundary places and orientationsBoundary places and orientations Medial places, figural sizes and orientationsMedial places, figural sizes and orientations Space itselfSpace itself

StatisticsStatistics The average shapeThe average shape Modes of variation from the averageModes of variation from the average

Computer GraphicsComputer Graphics Image AnalysisImage Analysis

Page 8: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Options for Primitives Options for Primitives Options for Primitives Options for Primitives

Space: Space: xxii for grid elements for grid elements

Landmarks: Landmarks: xxii described by local geometry described by local geometry

Boundary: (Boundary: (xxii ,normal ,normalii) spaced along boundary) spaced along boundary

Figural: nets of diatoms sampling figuresFigural: nets of diatoms sampling figures

Space: Space: xxii for grid elements for grid elements

Landmarks: Landmarks: xxii described by local geometry described by local geometry

Boundary: (Boundary: (xxii ,normal ,normalii) spaced along boundary) spaced along boundary

Figural: nets of diatoms sampling figuresFigural: nets of diatoms sampling figures

Page 9: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Primitives for shape Primitives for shape representation: Landmarksrepresentation: Landmarks

Primitives for shape Primitives for shape representation: Landmarksrepresentation: Landmarks

Sets of points of special Sets of points of special geometrygeometry

Sets of points of special Sets of points of special geometrygeometry

Page 10: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Primitives for shape Primitives for shape representation: Boundariesrepresentation: Boundaries

Primitives for shape Primitives for shape representation: Boundariesrepresentation: Boundaries

Boundary points with normalsBoundary points with normals Boundary points with normalsBoundary points with normals

Page 11: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Object Representation by M-RepsObject Representation by M-Reps

Page 12: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Each M-figure Represented by Each M-figure Represented by Net of Medial PrimitivesNet of Medial Primitives

o

o

ooo

o

o

o

oo

Page 13: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Each M-figure Represented by Each M-figure Represented by Net of Medial PrimitivesNet of Medial Primitives

Page 14: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Figural ModelsFigural ModelsFigural ModelsFigural Models

Figures: successive medial involutionFigures: successive medial involution Main figureMain figure ProtrusionsProtrusions IndentationsIndentations Separate figuresSeparate figures

Hierarchy of figuresHierarchy of figures Relative positionRelative position Relative widthRelative width Relative orientationRelative orientation

Figures: successive medial involutionFigures: successive medial involution Main figureMain figure ProtrusionsProtrusions IndentationsIndentations Separate figuresSeparate figures

Hierarchy of figuresHierarchy of figures Relative positionRelative position Relative widthRelative width Relative orientationRelative orientation

o

o

ooo

o

o

o

oo

Page 15: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Primitives’ Desired Primitives’ Desired PropertiesProperties

Primitives’ Desired Primitives’ Desired PropertiesProperties

GeometryGeometry Intuitive: simple, global + localIntuitive: simple, global + local Efficiently deformableEfficiently deformable Easily extracted or createdEasily extracted or created Spatial tolerance inherentSpatial tolerance inherent

StatisticsStatistics Unimodality: normally distributedUnimodality: normally distributed Via geometrical, tolerance-sensitive metricVia geometrical, tolerance-sensitive metric

GeometryGeometry Intuitive: simple, global + localIntuitive: simple, global + local Efficiently deformableEfficiently deformable Easily extracted or createdEasily extracted or created Spatial tolerance inherentSpatial tolerance inherent

StatisticsStatistics Unimodality: normally distributedUnimodality: normally distributed Via geometrical, tolerance-sensitive metricVia geometrical, tolerance-sensitive metric

Page 16: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Figural Models Figural Models with Boundary Deviationswith Boundary Deviations

Figural Models Figural Models with Boundary Deviationswith Boundary Deviations

HypothesisHypothesisAt a global level, a figural At a global level, a figural

model is the most intuitivemodel is the most intuitive

At a local level, boundary At a local level, boundary deviations are most intuitivedeviations are most intuitive

HypothesisHypothesisAt a global level, a figural At a global level, a figural

model is the most intuitivemodel is the most intuitive

At a local level, boundary At a local level, boundary deviations are most intuitivedeviations are most intuitive

o

o

ooo

o

o

o

oo

Page 17: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Union and Difference of M-figuresUnion and Difference of M-figures

Page 18: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Medial PrimitivesMedial PrimitivesMedial PrimitivesMedial Primitives

xx, (, (bb,,nn) frame, r, ) frame, r, (object angle) (object angle) Imply boundary segmentsImply boundary segments

with tolerancewith tolerance

Similarity transform equivariantSimilarity transform equivariant Zoom invariance implies width-proportionality ofZoom invariance implies width-proportionality of

tolerance of implied boundarytolerance of implied boundary boundary curvature distributionboundary curvature distribution spacing along netspacing along net interrogation aperture for imageinterrogation aperture for image

xx, (, (bb,,nn) frame, r, ) frame, r, (object angle) (object angle) Imply boundary segmentsImply boundary segments

with tolerancewith tolerance

Similarity transform equivariantSimilarity transform equivariant Zoom invariance implies width-proportionality ofZoom invariance implies width-proportionality of

tolerance of implied boundarytolerance of implied boundary boundary curvature distributionboundary curvature distribution spacing along netspacing along net interrogation aperture for imageinterrogation aperture for image

n

Page 19: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

3D kidney model extracted from CT3D kidney model extracted from CT3D kidney model extracted from CT3D kidney model extracted from CT

Four figure model of the kidneysFour figure model of the kidneys

Red represents indentation figuresRed represents indentation figures

Four figure model of the kidneysFour figure model of the kidneys

Red represents indentation figuresRed represents indentation figures

Page 20: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Need for Special End PrimitivesNeed for Special End Primitives

Represent Represent non-blobby objectsnon-blobby objects

angulated edges, corners, creasesangulated edges, corners, creases still allow rounded edges , corners, creasesstill allow rounded edges , corners, creases allow bent edgesallow bent edges

ButBut Avoid infinitely fine medial samplingAvoid infinitely fine medial sampling Maintain tangency, symmetry principlesMaintain tangency, symmetry principles

Page 21: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

End PrimitivesEnd Primitives

Rounded end primitive Rounded end primitive in cross-sectionin cross-section

Corner primitive Corner primitive in cross-sectionin cross-section

Page 22: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Displacements from Displacements from Figurally Implied BoundaryFigurally Implied Boundary

Displacements from Displacements from Figurally Implied BoundaryFigurally Implied Boundary

Boundary implied by figural model Boundary after displacements

Page 23: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Coarse-to-fine representationCoarse-to-fine representationCoarse-to-fine representationCoarse-to-fine representation

For each of three levelsFigural hierarchyFor each figure,

net chain, successively smaller tolerance

For each net tile, boundary displacement

chain

For each of three levelsFigural hierarchyFor each figure,

net chain, successively smaller tolerance

For each net tile, boundary displacement

chain

Page 24: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Multiscale Medial ModelMultiscale Medial ModelMultiscale Medial ModelMultiscale Medial Model

From larger scale medial net Coarsely sampled Smooother figurally implied boundary Larger tolerance

Interpolate smaller scale medial net Finer sampled More detail in figurally implied boundary Smaller tolerance

Represent medial displacements

From larger scale medial net Coarsely sampled Smooother figurally implied boundary Larger tolerance

Interpolate smaller scale medial net Finer sampled More detail in figurally implied boundary Smaller tolerance

Represent medial displacements

Page 25: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Multiscale Medial ModelMultiscale Medial ModelMultiscale Medial ModelMultiscale Medial Model

From larger scale medial net, interpolate smaller scale medial net

and represent medial displacements

From larger scale medial net, interpolate smaller scale medial net

and represent medial displacementsb.

Page 26: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Multiscale Medial/Boundary ModelMultiscale Medial/Boundary ModelMultiscale Medial/Boundary ModelMultiscale Medial/Boundary Model

From medial net Coarsely sampled, smoother implied

boundary Larger tolerance

Represent boundary displacements along implied normals Finer sampled, more detail in boundary Smaller tolerance

From medial net Coarsely sampled, smoother implied

boundary Larger tolerance

Represent boundary displacements along implied normals Finer sampled, more detail in boundary Smaller tolerance

Page 27: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image Analysis

SegmentationSegmentation Extract an object from imageExtract an object from image

RegistrationRegistrationFind geometric transformation that brings Find geometric transformation that brings

two images into alignmenttwo images into alignment 3D/3D3D/3D 3D/2D3D/2D

Shape MeasurementShape MeasurementFind how probable a shape isFind how probable a shape is

SegmentationSegmentation Extract an object from imageExtract an object from image

RegistrationRegistrationFind geometric transformation that brings Find geometric transformation that brings

two images into alignmenttwo images into alignment 3D/3D3D/3D 3D/2D3D/2D

Shape MeasurementShape MeasurementFind how probable a shape isFind how probable a shape is

Page 28: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Repres’n in Image AnalysisShape Repres’n in Image AnalysisShape Repres’n in Image AnalysisShape Repres’n in Image Analysis

SegmentationSegmentation

Find the most probable deformed mean Find the most probable deformed mean model, given the imagemodel, given the image

Probability involvesProbability involvesProbability of the deformed model (prior)Probability of the deformed model (prior)Probability of the image, given the Probability of the image, given the

deformed model (likelihood)deformed model (likelihood)

SegmentationSegmentation

Find the most probable deformed mean Find the most probable deformed mean model, given the imagemodel, given the image

Probability involvesProbability involvesProbability of the deformed model (prior)Probability of the deformed model (prior)Probability of the image, given the Probability of the image, given the

deformed model (likelihood)deformed model (likelihood)

Page 29: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Probability of a deformed modelProbability of a deformed modelProbability of a deformed modelProbability of a deformed model

From training setFrom training set via principal components analysis, coarse-to-finevia principal components analysis, coarse-to-fine

-C * Geometric difference from typical shape-C * Geometric difference from typical shape

From training setFrom training set via principal components analysis, coarse-to-finevia principal components analysis, coarse-to-fine

-C * Geometric difference from typical shape-C * Geometric difference from typical shape

Page 30: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Medialness: medial strength of Medialness: medial strength of a medial primitive in an imagea medial primitive in an imageMedialness: medial strength of Medialness: medial strength of a medial primitive in an imagea medial primitive in an image

Probability of image | deformed modelProbability of image | deformed model Sum of boundariness values Sum of boundariness values

at implied boundary positionsat implied boundary positions in implied normal directionsin implied normal directions with apertures proportional to with apertures proportional to

tolerancetolerance

Boundariness value Boundariness value Intensity profile distance from mean (at scale)Intensity profile distance from mean (at scale)

statistical, based on training setstatistical, based on training set Intensity differencesIntensity differences

via Gaussian derivativesvia Gaussian derivatives

Probability of image | deformed modelProbability of image | deformed model Sum of boundariness values Sum of boundariness values

at implied boundary positionsat implied boundary positions in implied normal directionsin implied normal directions with apertures proportional to with apertures proportional to

tolerancetolerance

Boundariness value Boundariness value Intensity profile distance from mean (at scale)Intensity profile distance from mean (at scale)

statistical, based on training setstatistical, based on training set Intensity differencesIntensity differences

via Gaussian derivativesvia Gaussian derivatives

b

rR(- )bx

rR()b

Page 31: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Figurally implied boundaries Figurally implied boundaries and rendering, via 4-figure modeland rendering, via 4-figure model

Figurally implied boundaries Figurally implied boundaries and rendering, via 4-figure modeland rendering, via 4-figure model

Page 32: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

3D DSL Model Deformation3D DSL Model Deformation3D DSL Model Deformation3D DSL Model Deformation

Initial Position of Model in Target ImageInitial Position of Model in Target Image

Page 33: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

3D DSL Model Deformation3D DSL Model Deformation3D DSL Model Deformation3D DSL Model Deformation

Figural DeformationFigural DeformationIteration 3Iteration 3

Page 34: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

3D DSL Model Deformation3D DSL Model Deformationwith interfigural penaltieswith interfigural penalties

3D DSL Model Deformation3D DSL Model Deformationwith interfigural penaltieswith interfigural penalties

Initial positionInitial position After optimizationAfter optimization

Page 35: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Repres’n in Image AnalysisShape Repres’n in Image AnalysisShape Repres’n in Image AnalysisShape Repres’n in Image Analysis

RegistrationRegistration

Find the most probable deformation, Find the most probable deformation, given the imagegiven the image

RegistrationRegistration

Find the most probable deformation, Find the most probable deformation, given the imagegiven the image

Page 36: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image Analysis

Prior-free medial shape analysisPrior-free medial shape analysis

Cores: height ridges of medialness (Pizer, Cores: height ridges of medialness (Pizer, Fritsch, Morse, Furst)Fritsch, Morse, Furst)

Statistical analysis of medial diatoms Statistical analysis of medial diatoms (Stetten)(Stetten)

Prior-free medial shape analysisPrior-free medial shape analysis

Cores: height ridges of medialness (Pizer, Cores: height ridges of medialness (Pizer, Fritsch, Morse, Furst)Fritsch, Morse, Furst)

Statistical analysis of medial diatoms Statistical analysis of medial diatoms (Stetten)(Stetten)

Page 37: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image Analysis

Cores: height ridges of medialnessCores: height ridges of medialness Cores: height ridges of medialnessCores: height ridges of medialness

Page 38: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

M I P @ U N CM I P @ U N C

Page 39: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image AnalysisShape Rep’n in Image Analysis

Statistical analysis of medial diatomsStatistical analysis of medial diatoms Statistical analysis of medial diatomsStatistical analysis of medial diatoms

a

sphere cylinder slab

Page 40: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

sphere

slab cylinder

Page 41: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

sphere

slab cylinder

Page 42: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

sphere

slab cylinder

Page 43: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

a

myocardium

left ventricle

left atrium

mitral valve

epicardium

slab

cylinder

cap

Page 44: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

sphere

slab cylinder

Page 45: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

sphere

slab cylinder

Page 46: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Page 47: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Rep’n in CAD/CAMShape Rep’n in CAD/CAM Shape Rep’n in CAD/CAMShape Rep’n in CAD/CAM

Stock figural modelsStock figural models Deformation tools: large scaleDeformation tools: large scale Coarse-to-fine specificationCoarse-to-fine specification Figural connection toolsFigural connection tools Direct rendering, according to display Direct rendering, according to display

needsneeds

Stock figural modelsStock figural models Deformation tools: large scaleDeformation tools: large scale Coarse-to-fine specificationCoarse-to-fine specification Figural connection toolsFigural connection tools Direct rendering, according to display Direct rendering, according to display

needsneeds

Page 48: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Deformation in CAD/CAMDeformation in CAD/CAM Deformation in CAD/CAMDeformation in CAD/CAM

Page 49: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Shape Rep’n in CAD/CAMShape Rep’n in CAD/CAM Shape Rep’n in CAD/CAMShape Rep’n in CAD/CAM

Design models for image analysisDesign models for image analysis Design models for image analysisDesign models for image analysis

Page 50: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Medial Object Shape Medial Object Shape RepresentationsRepresentations

for Image Analysis & Object Synthesisfor Image Analysis & Object Synthesis

Medial Object Shape Medial Object Shape RepresentationsRepresentations

for Image Analysis & Object Synthesisfor Image Analysis & Object Synthesis Figural models, at successive levels of Figural models, at successive levels of

tolerancetolerance Boundary displacementsBoundary displacements

Work in progressWork in progress Segmentation and registration toolsSegmentation and registration tools Statistical analysis of object populationsStatistical analysis of object populations CAD tools, incl. direct renderingCAD tools, incl. direct rendering Connection relative critical manifoldsConnection relative critical manifolds ……

Figural models, at successive levels of Figural models, at successive levels of tolerancetolerance

Boundary displacementsBoundary displacements

Work in progressWork in progress Segmentation and registration toolsSegmentation and registration tools Statistical analysis of object populationsStatistical analysis of object populations CAD tools, incl. direct renderingCAD tools, incl. direct rendering Connection relative critical manifoldsConnection relative critical manifolds ……

Page 51: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Application: Image guided Application: Image guided planning & delivery of radiotherapyplanning & delivery of radiotherapy

Application: Image guided Application: Image guided planning & delivery of radiotherapyplanning & delivery of radiotherapy

Planning in 3DPlanning in 3D Extracting normal anatomyExtracting normal anatomy Extracting tumorExtracting tumor Planning beam posesPlanning beam poses

Patient placementPatient placement Verification of plan via Verification of plan via

portal imageportal image

Planning in 3DPlanning in 3D Extracting normal anatomyExtracting normal anatomy Extracting tumorExtracting tumor Planning beam posesPlanning beam poses

Patient placementPatient placement Verification of plan via Verification of plan via

portal imageportal image

Page 52: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Finding Treatment Pose from Finding Treatment Pose from Portal Radiograph and Planning DRRPortal Radiograph and Planning DRR

Finding Treatment Pose from Finding Treatment Pose from Portal Radiograph and Planning DRRPortal Radiograph and Planning DRR

Planning CT Scan

Patient Setup

Planning DRR

Candidate DRRs Portal Image

Planning pose Candidate poses Treatment pose

Page 53: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Medial Net Shape ModelsMedial Net Shape ModelsMedial Net Shape ModelsMedial Net Shape Models

Medial nets, positions onlyMedial net

Page 54: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Integrated Medialness vs. Pose OffsetIntegrated Medialness vs. Pose OffsetIntegrated Medialness vs. Pose OffsetIntegrated Medialness vs. Pose Offset

Image Match Behavior Near Patient Pose Along Three Rotation Axis

-70

-65

-60

-55

-50

-45

-40

-35

-30

-5.00 -4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Distance to truth (deg)

Ima

ge

ma

tch

Gantry

Table

Collimator

Image Match Behavior Near Patient Pose Along Three Translation Axis

-80

-60

-40

-20

0

20

40

60

-5.00 -4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Distance to truth (cm)Im

ag

e m

atc

h

X

Y(zoom)

Z

Page 55: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Page 56: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Representing Boundary DisplacementsRepresenting Boundary DisplacementsRepresenting Boundary DisplacementsRepresenting Boundary Displacements

Along figurally implied boundary normalsAlong figurally implied boundary normals Coarse-to-fineCoarse-to-fine Captures along-boundary covarianceCaptures along-boundary covariance

Useful for renderingUseful for rendering

Along figurally implied boundary normalsAlong figurally implied boundary normals Coarse-to-fineCoarse-to-fine Captures along-boundary covarianceCaptures along-boundary covariance

Useful for renderingUseful for rendering

Page 57: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Summing Medialness on Medial NetSumming Medialness on Medial Netvia Medial Weighting Functionvia Medial Weighting Function

Summing Medialness on Medial NetSumming Medialness on Medial Netvia Medial Weighting Functionvia Medial Weighting Function

Page 58: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

CT Slice of Kidneys in AbdomenCT Slice of Kidneys in AbdomenCT Slice of Kidneys in AbdomenCT Slice of Kidneys in Abdomen

Page 59: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Object Shape Object Shape

Brain structures (Gerig)Brain structures (Gerig)Brain structures (Gerig)Brain structures (Gerig)

Page 60: MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group

MIDAG@UNCMIDAG@UNC

Geometric aspects : Geometric aspects : TransformationsTransformations

Euclidean: Euclidean: translation and rotationtranslation and rotation

Similarity:Similarity: translation, rotation, zoomtranslation, rotation, zoom

AffineAffine