landmark detection using statistical shape modelling and template matching (miccai 2014 cbm...

Automatic Landmark Detection using Statistical Shape Modelling

and Template Matching

Authors Habib Baluwala, Duane Malcolm, Jess Jor,

Poul Nielsen, Martyn Nash

Introduction

● Research Problem: Develop biomechanical models of the human breast

● Construction of biomechanical model of the torso skin surface

● Objective : – Align the mean mesh of the skin surface with new image

data– Match the mesh to the edges of the skin surface of the new

image

● Wide intensity range ● Variability of breast shapes ● Variability of torso shapes

Challenges in Mesh Alignment

Statistical Shape Modelling (SSM)

● Most structures of clinical interest have a characteristic shape and anatomical location relative to other structures

● Across the normal population the shape varies statistically

● Provides prior knowledge of shape

Training Data

● Thirty 2D MRI training images ● The outline of the torso skin surface is represented by 24

manually labelled points● We add another 6 anatomical landmarks:

– Sternum centre

– Aorta centre

– Spinal cord centre

– Vertebra centre – Left and right nipple

● Total : 30 landmarks

Modelling Shape using PCA

● Compute the mean of the data ● Compute the covariance of the data ● Compute the eigenvectors and eigenvalues of the

covariance matrix, sorted in decreasing order of eigenvalue size

● Remove the small eigenvalues, retaining most of the variation

● is the mean shape, is a set of orthogonal modes of variation and defines a set of components of deformable model.

Torso Shape Model

+

++

Template Matching● Move a template over an image and calculate

the similarity between the template and image patch

● Similarity measures– Cross Correlation (CC)

– Normalised Cross correlation (NCC)

– Sum of Squared Differences (SSD)

– Normalised Sum of Squared Differences (NSSD)

Average Template

+ + +

=

…..(30 images)

Template Matching

Vertebra centre template

Test image

Template matching result (correlation maps)

Template Matching

Aorta centre template

Test image

Template matching result (correlation maps)

Combining SSM and Template matching

● Vary the mode weights for the first three shape components

● Calculate the new shape and landmark positions

● Move the correlation map to its respective landmark location in the SSM shape model

● Multiply the correlation maps

SSM + Template matching

..

.27 more landmark template matching results

Shape Model

SSM + Template Matching Results(Shape Predicted Landmarks)

Manually selected landmarks and skin surface

Shape predicted landmarks

Local Maxima Search ● Crop the correlation map around the shape

predicted landmark (120 mm x 120 mm) ● Move the shape predicted landmark to the

local maximum of the correlation map

Shape predicted landmark Cropped correlation map Shape predicted landmark + local maxima

search

SSM + Template Matching Results+ Local Maxima Search

Move the shape predicted landmark to a local maximum using correlation maps for individual landmarks

Manually selected landmarks and skin surface

Shape predicted landmarks + local maxima search

ResultsSeries of leave-one-out experiments performed on thirty

2D MRI images

Conclusion

● SSM + Template Matching + Local Maxima search provides a robust detection of landmark points on skin surface

● Average error = 3.4mm 2.1 mm

Future Work● Extend the algorithm to 3D● Incorporate active appearance models

Questions !!!