csci 6971: image registration lecture 16: view-based registration march 16, 2004 prof. chuck...

CSci 6971: Image Registration Lecture 16: View-Based Registration

March 16, 2004

CSci 6971: Image Registration Lecture 16: View-Based Registration

March 16, 2004

Prof. Chuck Stewart, RPIDr. Luis Ibanez, KitwareProf. Chuck Stewart, RPIDr. Luis Ibanez, Kitware

Image Registration Lecture 16 2

OverviewOverview

Retinal image registration The Dual-Bootstrap ICP algorithm

Covariance matrix Covariance propagation Model selection

View-based registration Software design

Warning: mathematically, this lecture is a little “rich.” You will not be responsible for knowing the details

Retinal image registration The Dual-Bootstrap ICP algorithm

Covariance matrix Covariance propagation Model selection

View-based registration Software design

Warning: mathematically, this lecture is a little “rich.” You will not be responsible for knowing the details


Retinal Image Registration: ApplicationsRetinal Image Registration: Applications

Mosaics Multimodal integration Blood flow animation Change detection

Mosaics Multimodal integration Blood flow animation Change detection


MosaicsMosaics


Multimodal IntegrationMultimodal Integration


Fluorescein Angiogram AnimationFluorescein Angiogram Animation


Change VisualizationChange Visualization

QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.


Retinal Image Registration - PreliminariesRetinal Image Registration - Preliminaries

Features Transformation models Initialization

Features Transformation models Initialization


landmarks vascular centerlines

FeaturesFeatures

Vascular centerline points Discrete locations along

the vessel contours Described in terms of

pixel locations, orientations, and widths

Vascular landmarks Pixel locations,

orientations and width of vessels that meet to form landmarks

Vascular centerline points Discrete locations along

the vessel contours Described in terms of

pixel locations, orientations, and widths

Vascular landmarks Pixel locations,

orientations and width of vessels that meet to form landmarks


Transformation ModelsTransformation Models

)(pXq ⋅Θ= [ ]Tyxyxyx 221)( =pX

Model Parameter Matrix DoF Accuracy (pixels)

Similarity 4 5.05

Affine 6 4.58

Reduced quadratic

6 2.41

Full quadratic 12 0.64

⎥⎦

⎤⎢⎣

⎡ −000

000

abt

bat

y

x

⎥⎦

⎤⎢⎣

⎡

262524232221

161514131211

θθθθθθ

θθθθθθ

⎥⎦

⎤⎢⎣

⎡000

000

dbt

cat

y

x

⎥⎦

⎤⎢⎣

⎡ −ffabt

eebat

y

x

0

0


Initializing RegistrationInitializing Registration

Form list of landmarks in each image Form matches of one landmark from

each image The selection of these matches

will be discussed in Lectures 18 and 19

Choose matches, one at a time For each match:

Compute an initial similarity transformation in the small image region surrounding the landmarks

Apply Dual-Bootstrap ICP procedure to see if the initial alignment can be successfully grown into an accurate, image-wide alignment

End when one match leads to success, or all matches are exhausted

Form list of landmarks in each image Form matches of one landmark from

each image The selection of these matches

will be discussed in Lectures 18 and 19

Choose matches, one at a time For each match:

Compute an initial similarity transformation in the small image region surrounding the landmarks

Apply Dual-Bootstrap ICP procedure to see if the initial alignment can be successfully grown into an accurate, image-wide alignment

End when one match leads to success, or all matches are exhausted


Dual-Bootstrap - OverviewDual-Bootstrap - Overview

Match and refine estimate in each region

Bootstrap the model: Low-order for small

regions; High-order for large Automatic selection

Bootstrap the region: Covariance propagation

gives uncertainty

Iterate until convergence


Matching and Estimation in Each RegionMatching and Estimation in Each Region

Matching - standard stuff: Vascular centerline points from within current region of

moving image Mapped using current transform estimate Find closest point using Borgefors digital distance map

Estimation: Fix scale estimate Run IRLS

Matching - standard stuff: Vascular centerline points from within current region of

moving image Mapped using current transform estimate Find closest point using Borgefors digital distance map

Estimation: Fix scale estimate Run IRLS


Covariance Matrix of EstimateCovariance Matrix of Estimate

Measures uncertainty in estimate of transformation parameters

Basis for region growth and model selection The next few slides will give an overview of

computing an approximate covariance matrix We’ll start with linear regression

Measures uncertainty in estimate of transformation parameters

Basis for region growth and model selection The next few slides will give an overview of

computing an approximate covariance matrix We’ll start with linear regression


Problem Formulation in Linear RegressionProblem Formulation in Linear Regression

Independent (non-random) variable values:

Dependent (random) variable values

Linear relationship based on k+1 dimensional parameter vector a:

Independent (non-random) variable values:

Dependent (random) variable values

Linear relationship based on k+1 dimensional parameter vector a:


Least-Squares FormulationLeast-Squares Formulation

Least-squares error term:

Here:

Least-squares error term:

Here:


Estimate and Covariance MatrixEstimate and Covariance Matrix

Estimate:

Residual error variance (square of “scale”):

Parameter estimate covariance

Estimate:

Residual error variance (square of “scale”):

Parameter estimate covariance


Aside: Line Fitting in 2DAside: Line Fitting in 2D

Form of the equation:

If the xi values are centered:

Then the parameters are independent with variances

for the linear and constant terms, respectively

Form of the equation:

If the xi values are centered:

Then the parameters are independent with variances

for the linear and constant terms, respectively


Hessians and CovariancesHessians and Covariances

Back to k dimensions, re-consider the objective function:

Compute the Hessian matrix:

Observe the relationship

Back to k dimensions, re-consider the objective function:

Compute the Hessian matrix:

Observe the relationship


Hessians and CovariancesHessians and Covariances

This is exact for linear regression, but serves as a good approximation for non-linear least-squares

In general, the Hessian will depend on the estimate (in regression it doesn’t because the problem is quadratic), so the approximate relationship is

This is exact for linear regression, but serves as a good approximation for non-linear least-squares

In general, the Hessian will depend on the estimate (in regression it doesn’t because the problem is quadratic), so the approximate relationship is


Hessian in RegistrationHessian in Registration

Recall the weighted least-squares objective function:

Keeping the correspondences and the weights fixed,

where Dk gives the error of the k-th correspondence

Inverting this gives the covariance approximation. This approximation is only good when the estimate is

fairly accurate

Recall the weighted least-squares objective function:

Keeping the correspondences and the weights fixed,

where Dk gives the error of the k-th correspondence

Inverting this gives the covariance approximation. This approximation is only good when the estimate is

fairly accurate


Back to Dual-Bootstrap ICPBack to Dual-Bootstrap ICP

Covariance is used in two ways in each DB-ICP iteration: Determining the region incorporates enough

constraints to switch to a more complex model Similarity => Affine => Reduced Quadratic => Quadratic

Determining the growth of the dual-bootstrap region: More stable transformation estimates lead to faster

growth

Covariance is used in two ways in each DB-ICP iteration: Determining the region incorporates enough

constraints to switch to a more complex model Similarity => Affine => Reduced Quadratic => Quadratic

Determining the growth of the dual-bootstrap region: More stable transformation estimates lead to faster

growth


Model SelectionModel Selection

What model should be used to describe a given set of data?

Classic problem in statistics, and many methods have been proposed

Most trade-off the fitting accuracy of higher-order models with the stability (or lower complexity) of lower-order models

What model should be used to describe a given set of data?

Classic problem in statistics, and many methods have been proposed

Most trade-off the fitting accuracy of higher-order models with the stability (or lower complexity) of lower-order models


Model Selection in DB-ICPModel Selection in DB-ICP

Use correspondence set Estimate the IRLS parameters and covariance matrices for each

model in current set

For each model (with dm parameters) this generates a set of weights and errors and a covariance matrix:

Choose the model maximizing the model selection equation (derived from Bayesian modeling):

The first two terms increase with increasingly complex models; the last term decreases

Use correspondence set Estimate the IRLS parameters and covariance matrices for each

model in current set

For each model (with dm parameters) this generates a set of weights and errors and a covariance matrix:

Choose the model maximizing the model selection equation (derived from Bayesian modeling):

The first two terms increase with increasingly complex models; the last term decreases


Region Growth in DB-ICPRegion Growth in DB-ICP

Grow each side independently

Grow is inversely proportional to uncertainty in mapping of boundary point on the center of each side

New rectangular region found from the new positions of each of the boundary points

Grow each side independently

Grow is inversely proportional to uncertainty in mapping of boundary point on the center of each side

New rectangular region found from the new positions of each of the boundary points


Aside: Covariance Propagation and Transfer Error

Aside: Covariance Propagation and Transfer Error

Given mapping function:

We will treat Θ as a random variable, but not gk

Uncertainty in Θmakes gk’ a random variable.

What then is the covariance of gk’?

We solve this using standard covariance propagation techniques:

Compute the Jacobian of the transformation, evaluated at gk:

Pre- and post-multiply to obtain the covariance of gk’

In computer vision, this is called the “transfer error”

Given mapping function:

We will treat Θ as a random variable, but not gk

Uncertainty in Θmakes gk’ a random variable.

What then is the covariance of gk’?

We solve this using standard covariance propagation techniques:

Compute the Jacobian of the transformation, evaluated at gk:

Pre- and post-multiply to obtain the covariance of gk’

In computer vision, this is called the “transfer error”


Outward Growth of a SideOutward Growth of a Side

Let k be the outward normal of the side, and let rk be the distance of the side from the center of the region

Project the transfer error covariance onto k to obtain a scalar variance k

The outward growth (along normal k) is

where controls the maximum growth rate, which occurs when k < 1

Let k be the outward normal of the side, and let rk be the distance of the side from the center of the region

Project the transfer error covariance onto k to obtain a scalar variance k

The outward growth (along normal k) is

where controls the maximum growth rate, which occurs when k < 1


Putting It All Together - The Example, RevisitedPutting It All Together - The Example, Revisited


Turning to the SoftwareTurning to the Software

A “view” is a definition or snapshot of the registration problem.

A “view” contains: An image region (current region, plus goal region) A current transformation estimate and estimator A current stage (resolution) of registration

Views work in conjunction with multistage registration

A “view” is a definition or snapshot of the registration problem.

A “view” contains: An image region (current region, plus goal region) A current transformation estimate and estimator A current stage (resolution) of registration

Views work in conjunction with multistage registration


View-Based Registration - ProceduralView-Based Registration - Procedural

The following is repeated for each initial estimate For each stage:

Do Match Compute weights Estimate scale For each model

Run IRLS to estimate parameters and covariances Re-estimate scale Generate next view

For DB-ICP view generator this choose the best model and grows the region

Until region has converged and highest order model used Prepare for next stage

The following is repeated for each initial estimate For each stage:

Do Match Compute weights Estimate scale For each model

Run IRLS to estimate parameters and covariances Re-estimate scale Generate next view

For DB-ICP view generator this choose the best model and grows the region

Until region has converged and highest order model used Prepare for next stage


ImplementationImplementation

rgrl_view Store the information about the view

rgrl_view_generator Generate the next view

rgrl_view_based_registration Mirrors rgrl_feature_based_registration

with modifications based on the outline on previous slide

Example rgrl/example/registration_retina.cxx

rgrl_view Store the information about the view

rgrl_view_generator Generate the next view

rgrl_view_based_registration Mirrors rgrl_feature_based_registration

with modifications based on the outline on previous slide

Example rgrl/example/registration_retina.cxx


SummarySummary

Retina registration: Models, features and initialization

DB-ICP: Matching, estimation and covariances Model selection Region growing

Generalization to view-based registration and its implementation in the toolkit.

Retina registration: Models, features and initialization

DB-ICP: Matching, estimation and covariances Model selection Region growing

Generalization to view-based registration and its implementation in the toolkit.


Looking Ahead to Lecture 17Looking Ahead to Lecture 17

Discussion of toolkits: What’s easy, what’s hard What’s missing

Project discussion Requirements Topics and initial steps

Discussion of toolkits: What’s easy, what’s hard What’s missing

Project discussion Requirements Topics and initial steps

csci 6971: image registration lecture 16: view-based registration march 16, 2004 prof. chuck...

Documents

image registrationlecture

image form

image registration lecture

small image region

image wide alignment

kitware slide

exhausted slide

multimodal integration