consistent image registration christensen
TRANSCRIPT
-
8/13/2019 Consistent Image Registration Christensen
1/15
568 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 20, NO. 7, JULY 2001
Consistent Image RegistrationG. E. Christensen* and H. J. Johnson
AbstractThis paper presents a newmethod forimageregistra-tion based on jointly estimating the forward and reverse transfor-mations between two images while constraining these transformsto be inverses of one another. This approach produces a consistentset of transformations that have less pairwise registration error,i.e., better correspondence, than traditional methods that estimatethe forward and reversetransformations independently. The trans-formations are estimated iteratively and are restricted to preservetopology by constraining them to obey the laws of continuum me-chanics. The transformations are parameterized by a Fourier se-ries to diagonalize the covariance structure imposed by the con-tinuum mechanics constraints and to provide a computationallyefficient numerical implementation. Results using a linear elasticmaterial constraint are presented using both magnetic resonanceand X-ray computed tomography image data. The results showthat the joint estimation of a consistent set of forward and reverse
transformations constrained by linear-elasticity give better regis-tration results than using either constraint alone or none at all.
Index TermsCorrespondence, deformable templates, imageregistration, inverse transformation.
I. INTRODUCTION
IMAGE registration has many uses in medicine such as
multimodality fusion, image segmentation, deformable
atlas registration, functional brain mapping, image-guided
surgery, and characterization of normal versus abnormal
anatomical shape and variation. The fundamental assumption
in each of these applications is that image registration can be
used to define a meaningful correspondence mapping betweenanatomical images collected from imaging devices such as
computed tomography (CT), magnetic resonance imaging
(MRI), cryosectioning, etc. It is often assumed that this corre-
spondence mapping or transformation is one-to-one, i.e., each
point in image is mapped to only one point in image and
vice versa. A fundamental problem with a large class of image
registration techniques is that the estimated transformation
from image to does not equal the inverse of the estimated
transform from to . This inconsistency is a result of the
matching criterias inability to uniquely describe the correspon-
dences between two images. This paper seeks to overcome this
limitation by jointly estimating the transformation from to
Manuscript received October 14, 1999; revised April 26, 2001. This workwas supported in part by the National Institutes of Health (NIH) under GrantNS35368, Grant CA75371, and Grant DC03590, and in part by a grant fromthe Whitaker Foundation. The Associate Editor responsible for coordinating thereview of this paper and recommending its publication was M. W. Vannier. As-terisk indicates corresponding author.
*G. E. Chrisensen is with the Department of Electrical and Computer Engi-neering, University of Iowa, 205 CC, Iowa City, IA 52242 USA (e-mail: [email protected]).
H. J. Johnson is with theDepartment ofElectricaland ComputerEngineering,University of Iowa, Iowa City, IA 52242 USA.
Publisher Item Identifier S 0278-0062(01)05523-9.
and from to while enforcing the consistency constraint
that these transforms are inverses of one another.
The forward transformation from image to and the re-
verse transformation from to are shown in Fig. 1. Ideally,
the transformations and should be uniquely determined and
should be inverses of one another. Estimating and indepen-
dently very rarely results in a consistent set of transformations
due to a large number of local minima. To overcome this defi-
ciencyin current registration systems, we propose to jointly esti-
mate and while constraining these transforms to be inverses
of one another. Jointly estimating the forward and reverse trans-
formations provides additional correspondence information and
helps ensure that these transformations define a consistent cor-
respondence between the images. Although uniqueness is verydifficult to achieve in medical image registration, the joint es-
timation should lead to more consistent and biologically mean-
ingful results.
Image registration algorithms use landmarks [1][4],
contours [5][7], surfaces [8][11], volumes [12][18], [6],
[19][21], or a combination of these features [22] to manually,
semi-automatically or automatically define correspondences
between two images. The need to impose the invertibility
consistency constraint depends on the particular application
and on the correspondence model used for registration. In
general, registration techniques that do not uniquely determine
the correspondence between image volumes should benefit
from the consistency constraint. This is because such tech-niques often rely on minimize/maximize a similarity measure
which has a large number of local minima/maxima due to the
correspondence ambiguity. Examples include methods that
minimizing/maximizing similarity measures between features
in the source and target images such as image intensities, object
boundaries/surfaces, etc. In theory, the higher the dimension
of the transformation, the more local minima these similarity
measures have. Methods that use specified correspondences
for registration will benefit less or not at all from the invert-
ibility consistency constraint. For example, landmark-based
registration methods implicitly impose an invertibility con-
straint at the landmarks because the correspondence defined
between landmarks is the same for estimating the forward andreverse transformations. However, the drawbacks of specifying
correspondences include requiring user interaction to specify
landmarks, unique correspondences cannot always be specified,
and such methods usually only provide coarse registration due
to the small number of correspondences specified.
In this paper, we will restrict our analysis to the class of ap-
plications that can be solved using diffeomorphic transforma-
tions. A diffeomorphic transformation is defined to be contin-
uous, one-to-one, onto, and differentiable. The diffeomorphic
restriction is valid for a large number of problems in which the
02780062/01$10.00 2001 IEEE
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
2/15
http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
3/15
http://-/?-http://-/?-http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
4/15
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
5/15
http://-/?- -
8/13/2019 Consistent Image Registration Christensen
6/15
CHRISTENSEN AND JOHNSON: CONSISTENT IMAGE REGISTRATION 573
Fig. 2. Example of elements contained in an 8 2 8 lattice. (a) contains all of the points while contains the points within the doted lines. (b) Elementsof .
and are used to enforce/balance the constraints. The effects
of varying these parameters are studied later in the paper.
The basis coefficients
are determined by gradient descent using
(13)
(14)
where is the step size. The superscript corresponds to thevalue of and at the th iteration. Each equation above repre-
sents a 3 1 vector equation,3 i.e., one equation for each com-
ponent of the vectors , , , and . The partial derivatives4 of
used in (13) and (14) are
3The notation where is a scalar-valued function and is a 3 2 1vector is defined as the 3 2 1 vector .
4
The coefficients
and
are assumed to be constant when computingthe partial derivatives.
where
The terms and are 3 1 vectors and are efficiently com-
puted for all using 3-D FFTs; each component of and
requires a separate 3-D FFT for a total of six FFTs. The nota-
tion represents the gradient of evaluated at the
point . The gradient iscomputed usingsymmetric dif-
(12)
-
8/13/2019 Consistent Image Registration Christensen
7/15
574 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 20, NO. 7, JULY 2001
ference equations5 and trilinear interpolation is used to evaluate
the gradient at the point . The notation
is defined similarly. The gradient descent insures that and
have complex conjugate symmetry. Notice that and are
the Fourier transforms of a real signal implying that and
have complex conjugate symmetry. This in turn implies that the
gradients in (13) and (14) have conjugate symmetry and, there-
fore, thebasis coefficients and have complex conjugate sym-metry.
The steps involved in estimating the basis coefficients and
are summarized in the following algorithm.
Algorithm:
1. Set for and set
.
2. Compute using the procedure
described in Section II-C and set
.
3. Compute new forward basis coefficients
using (13).
4. Compute the forward displacement field
using (11).
5. Compute using the procedure
described in Section II-C and set
.
6. Compute new reverse basis coefficients
using (14).
7. Compute the reverse displacement field
using (11).
8. If the criteria is met to increase
the number of basis functions then set
, and set the new coefficients
in (11) to zero.9. If the algorithm has not converged or
reached the maximum number of
iterations goto step 2.
10. Use forward displacement field
to transform the template image
and reverse displacement field
to transform the target image .
III. RESULTS
Two sets of experiments were performed to demonstrate the
linear-elastic consistent image registration algorithm. The first
set of experiments were designed to show the effect of varyingthe transformation parameters and the second setof experiments
demonstrate typical results that can be expected using a mul-
tiresolution approach to transform full resolution MRI and CT
data.
A. Parameter Evaluation
Two MRI and two CT image volumes were used to investi-
gate the effect of varying the parameters used in the consistent
image registration algorithm. The data sets were collected from
5The gradient of evaluated at is computed as 0 0 0
0 , 0 0
different individuals using the same MR and CT machines and
the same scan parameters. The MRI data sets correspond to two
normal adults and the CT data sets correspond to two 3-mo.-old
infants, one normal and one abnormal (bilateral coronal synos-
tosis). The MRI and CT data sets were chosen to test registration
algorithm when matching anatomies with similar and dissimilar
shapes, respectively.
The MRI data were preprocessed by normalizing theimage intensities, correcting for translation and rotation, and
segmenting the brain from the head using Analyze (Mayo
Clinic, Rochester, MN). The translation aligned the anterior
commissure points, and the rotation aligned the corresponding
axial and sagittal planes containing the anterior and posterior
commissure points, respectively. The MRI data sets were
down-sampled and zero padded to form a 64 64 80 voxel
lattice. The CT data sets were corrected for translation and
rotation and down-sampled to form a 64 64 48 voxel
lattice. The translation aligned the basion skull landmarks, and
the rotation aligned the corresponding Frankfort horizontal and
midsagittal planes, respectively.
Tables I and II show the results of 32 experiments for toMRI-to-MRI and CT-to-CT registration, respectively, as the
weighting values and were varied. The weight for the
similarity cost was set to one for all of the experiments. The
values of and ranged from 0.0 to 0.0125 and 0 to 5000
for the MRI-to-MRI experiments, respectively. The values of
and ranged from 0.0 to 0.001 25 and 0 to 1275 for the
CT-to-CT experiments, respectively. The gradient descent step
size was set to 0.000 04 for the MRI experiments and 0.0001
for the CT experiments. The difference in the parameters is
due to the different intensity characteristics of each modality.
These ranges can be used as a guide for determining parameter
settings for registration of other modalities. We have found that
there is no need to adjust the parameters for additional data sets
of the same modality.
The data sets were registered initiallywith zero andfirst-order
harmonics. Each experiment was run for 1000 iterations unless
the algorithm failed to converge. After every 100th iteration, the
maximum harmonic wasincreased by one. Each experiment that
ran for 1000 iterations took approximately 1.5 h to run on an Al-
phaPC clone using a single 667-MHz, alpha 21 264 processor. It
is expected that this time can be significantly decreased by opti-
mizing the code and using a better optimization technique than
gradient descent. In some of the experiments the Jacobian of the
transformation went negative due to insufficient regularization
or due to a bad choice of parameters. In these cases, the experi-ments were stopped before the Jacobian went negative to report
the results. The numbers reported for the Similarity cost ,
the linear elasticity cost , and the inverse consistency cost
, were scaled by 10 000 for presentation.
Experiments MRI01 and CT01 correspond to unconstrained
estimation in which the forward and reverse transformations
were estimated independently. These experiments produced the
worst registration results as evident by the largest values of
, , and in the respective tables. These experi-
ments were stopped before the 1000th iteration because the Ja-
cobian went negative during the gradient descent. This was ex-
pected since the regularization terms help prevent the Jacobian
-
8/13/2019 Consistent Image Registration Christensen
8/15
CHRISTENSEN AND JOHNSON: CONSISTENT IMAGE REGISTRATION 575
TABLE IMRI-TO-MRI CONSISTENTLINEAR-ELASTICREGISTRATION EXPERIMENTAL PARAMETERS ANDFINALVALUES
REG: Linear-elastic regularization; ICC: Inverse consistency constraint; : REG weight; : ICC weight; 1: Similarity cost weight. The valuesof , , and have been scaled by 10 000
TABLE IICT-TO-CT CONSISTENTLINEAR-ELASTICREGISTRATION EXPERIMENTAL PARAMETERS ANDFINALVALUES
REG: Linear-elastic regularization; ICC: Inverse consistency constraint; : REG weight; : ICC weight; 1: Similarity cost weight. The valuesof , , and have been scaled by 10 000
from going negative. The similarity cost is the lowest for these
experiments since the algorithm finds the best match between
the images without any constraint preventing the Jacobian from
going negative.
Experiments MRI05, MRI09, MRI13, CT05, CT09, andCT13 demonstrate the effect of estimating the forward and
reverse transformations independently while varying the
weight of the linear elastic cost. As before, the large difference
between the forward and reverse displacement fields as reported
by confirms that linear elasticity alone is not sufficient
to guarantee that the forward and reverse transformations are
inverses of one another. However, the linear elasticity constraint
did improve the transformation over the unconstrained case
because the minimum Jacobian and the inverse of the maximum
Jacobian is far from being singular.
Experiments MRI02, MRI03, MRI04, CT02, CT03, and
CT04 demonstrate the effect of jointly estimating the forward
and reverse transformations without enforcing the linear elas-
ticity constraint. The values for these experiments are
much lower than the previous cases since they are being min-
imized. The forward and reverse transformations are inverses
of each other when are zero so that the smaller the costs, the closer the transformations are to being inverses of
each other.
The remaining experiments show the effect of jointly esti-
mating the forward and reverse transformations while varying
the weights on both the linear elasticity constraint and the in-
verse consistency constraint. These experiments show that it is
possible to find a set of parameters that produce better results
using both constraints than only using one or none. Notice that
increasing the constraint weights causes the similarity cost to in-
crease indicating a worse intensity match between the images.
At the same time, the worst case values of the Jacobian increase
as the constraint weights increase indicating less spatial distor-
-
8/13/2019 Consistent Image Registration Christensen
9/15
576 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 20, NO. 7, JULY 2001
Fig. 3. Statistics associated with the MRI11 experiment.
Fig. 4. Statistics associated with the CT15 experiment.
tion. The optimal set of parameters should be chosen to provide
a good intensity match while producing the least amount of spa-
tial distortion as measured by the Jacobian and an acceptable
level of inverse consistency error.
The time series statistics for experiments MRI11 and CT15
are shown in Figs. 3 and 4, respectively. These figures show that
the gradient descent algorithm converged for each set of trans-
formation harmonics. In both cases, the similarity cost
decreased at each iteration while the prior terms increased be-
fore decreasing. Notice that the inverse consistency constraint
increased as the images deformed for each particular harmonic
resolution. Then when the number of harmonics were increased,
the inverse constraint decreased before increasing again. This
is due to the fact that a low-dimensional Fourier series does
not have the degrees of freedom (DOFs) to faithfully represent
the inverse of a low-dimensional Fourier series. This is seen by
looking at the high dimensionality of a Taylor series representa-
tion of the inverse transformation. Finally, notice that the inverse
consistency constraint caused the extremal Jacobian values of
the forward and reverse transformations to track together. The
extremal Jacobian values correspond to the worst case distor-
tions produced by the transformations.
Fig. 5 shows the effect of varying and on the inverse con-
sistency cost as a function of iteration. Fig. 5(a) shows
-
8/13/2019 Consistent Image Registration Christensen
10/15
CHRISTENSEN AND JOHNSON: CONSISTENT IMAGE REGISTRATION 577
(a)
(b)
Fig. 5. Plots demonstrating the effect of (a) modifying the inverse consistencyconstraint weight while 0 and (b) modifying the linear-elasticityregularization weight while 0.
that increases with iteration and then drops every 100 it-erations when additional parameters (DOFs) are added to the
transformation. The curves decrease in amplitude as is in-
creased until becomes to large and the algorithm fails to con-
verge. Fig. 5(b) shows that increases as the linearelasticity
weight is increased. This makes sense because the two regu-
larization terms fight one another. The inverse consistency cost
increases as the linear elasticity cost is penalized more.
B. Multiresolution Registration
A spatial and frequency multiresoltion procedure was used
to estimate the full resolution registration of the data sets used
in the previous section. In this approach, global structures are
matched before local to reduce the likelihood of incorrect localregistration errors and to increase convergence. Table III shows
the number of iterations, the harmonic increment iteration, theinitial number of harmonics, and the final number of harmonics
at each resolution. This schedule proceeds from low resolution
to high resolution starting at one eight the spatial resolution and
increases to full resolution.
A set of parameters were chosen from Tables I and II that gave
a good tradeoff between the image intensity match, the inverse
consistency of the transformations, and low spatial distortion.
The parameters used were time step 0.000 04, 1.0,
0.001 25, and 2500 for the MRI-to-MRI registration and
time step 0.0001, 1.0, 0.001 25, and 600 for
the CT-to-CT registration. These parameters were used at all
resolutions.
TABLE IIIMULTIRESOLUTION ITERATIONSCHEDULE USED TOGENERATE THEFULL
RESOLUTIONMRI-TO-MRIAND CT-TO-CT REGISTRATION RESULTS
Fig. 6 shows three transverse sections from the 3-D result
from the template , deformed target , target , deformed
template MRI data sets. The first two columns and the last
two columns should look a like for a good registration. These
pairs of columns look similar with respect to the global struc-
tures but have small local differences as seen by the difference
images shown in the first two columns of Fig. 7. Notice that
the outer contour of the deformed images match their respec-
tive target data sets and that there is good correspondence in the
region of the ventricles. The local mismatch is mostly due to
differences in the topology of the gray matter folds and due to
the low-frequency Fourier series parameterization of the trans-
formations.
The last two columns of Fig. 7 shows the normed differ-
ence between the forward and reverse transformations for the
MRI-to-MRI experiment. These figures show the spatial loca-
tions of where the forward and reverse transformations have the
largest inverse consistency errors. The range on the difference
for the entire 3-D volume is from 0 to 0.002 234. This maximum
difference corresponds to a registration error between 0.571 and
0.749 voxel units.6 Notice that most of the error is internal to
the brain and that most of the error appears in the cortex re-
gions. The similarities between the absolute difference intensity
images to the normed transformation difference images suggestthat most of the inverse consistency error occurs were the trans-
formed images are still mismatched. A further description and
additional figures showing the effect of using or not using the
inverse consistency cost as it relates to the spatial inverse con-
sistency error can be found in [31].
Three transverse slices from the 3-D full resolution CT-to-CT
experiment are shown in Fig. 8. Notice the good global registra-
tion of the corresponding CT data sets. The first two columns
of Fig. 9 shows the absolute intensity difference between these
slices. As before, the errors show up along the boundaries of the
objects. The last two columns of this figure show thenormed dif-
ference between the forward and reverse transformations. Again
we see similarities between the intensity differences and thetransformation differences. The maximum inverse consistency
error for this experiment is between 0.871 and 1.16 voxel units.7
The MRI-to-MRI registration following the schedule in
Table III took approximately 4, 40, 60, and 55 min to compute
at the 32 32 40, 64 64 80, 128 128 160, and 256
256 320 voxel resolutions, respectively. The CT-to-CT
registration took approximately 2, 21, 33, and 30 min to
6The minimum was computed as 256 2 0.002 234 and the maximum as 3202 0.002 234.
7The minimum was computed as 192 2 0.004 538 and the maximum as 2562 0.004 538.
http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
11/15
578 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 20, NO. 7, JULY 2001
Fig. 6. Transverse slices 109, 135, and 165 (rows top to bottom) from the full resolution MRI-to-MRI registration experiment. The columns from left to rightcorrespond to the template , the deformed target , the target and the deformed template . The intensities are on a range from 0 to 1.0.
compute at the 32 32 24, 64 64 48, 128 128
96, and 256 256 192 voxel resolutions, respectively. All
times are for an AlphaPC clone using a single 667-MHz, alpha
21 264 processor.
IV. DISCUSSION
The experiments presented in this paper were designed to testthe validity of the new inverse transformation consistency con-
straint as applied to a linear-elastic transformation algorithm. As
such, there was no effort made to optimize the rate of conver-
gence of the algorithm. The convergence rate of the algorithm
can be greatly improved by using a more efficient optimization
technique than gradient descent such as conjugate gradient at
each parameterization resolution. In addition, a convergence cri-
teria can be used to determine when to increment the number of
parameters in the model. The CT data used in the experiments
was selected to stress the registration algorithm. The conver-
gence of the algorithm would have been much faster if the data
sets were adjusted for global scale initially.
A. Measurement of Transformation Distortion
It is important to track both the minimum and maximum
values of the Jacobian during the estimation procedure. The
Jacobian measures the differential volume change of a point
being mapped through the transformation. At the start of the
estimation, the transformation is the identity mapping and,
therefore, has a Jacobian of one. If the minimum Jacobiangoes negative, the transformation is no longer a one-to-one
mapping and as a result folds the domain inside out [30].
Conversely, the reciprocal of the maximum value of the
Jacobian corresponds to the minimum value of the Jacobian
of the inverse mapping. Thus, as the maximum value of the
Jacobian goes to infinity, the minimum value of the Jacobian
of the inverse mapping goes to zero. In the present approach,
the inverse transformation consistency constraint was used
to penalize transformations that deviated from their inverse
transformation. A limitation of this approach is that cost
function in (3) is an average metric and cannot enforce the
pointwise constraints that and
http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
12/15
CHRISTENSEN AND JOHNSON: CONSISTENT IMAGE REGISTRATION 579
Fig. 7. The first two columns correspond to the absolute intensity difference between the template and the deformed target 0 (column one) and betweenthe target and the deformed template 0 (column two) in Fig. 6. The intensity range for the absolute difference is on a range from 0 to 0.892. The last twocolumns correspond to the normed difference between the forward and inverse of the reverse transformation 0 (column three), and between the reversetransformation and the inverse of the forward transformation 0 (column four). The intensity range for the normed transformation differences is from 0to 0.002234.
. This point is illustrated by
Tables I and II by the fact that the minimum values of
and differ from the reciprocal of the maximum values of
and , respectively, However, these extremal Jacobian
values do give an upper bound on the worst case distortions
produced by the transformations demonstrating the consistencybetween the forward and reverse transformations.
B. Spatial Multiresolution
The minimization problem is discretized so it can be imple-
mented on a digital computer. The higher the sampling rate the
more accurate the discrete approximation is to the continuous
case. An advantage of discretizing a continuous formulation
is that the problem can be solved at different spatial sampling
rates. The approach that is taken is to solve the minimization
at a course resolution initially to approximate the solution. The
advantage of solving the problem on a course grid is that the al-
gorithm requires fewer computations/iteration that a finer grid.
This results in reduced computation time at low resolution. Each
time the resolution of the grid is increase by a factor of two in
each dimension, the computation time increases by a factor of
eight. The drawback of solving the problem at low resolution isthat there can be significant registration errors due to the loss
of detail in the down sampling procedure. The tradeoff between
quicker execution times at low resolution and more accurate reg-
istration at higher resolution can exploited by solving the regis-
tration problem at low spatial resolution during the initial iter-
ations to approximate the result and then increasing the spatial
resolution to get a more accurate result at the later iterations.
The spatial multiresolution approach works well with the
frequency multiresolution approach provided by increasing the
number of harmonics used to represent the displacement fields.
The number of harmonics used to represent the displacement
fields is initially set small and then increased as the number
-
8/13/2019 Consistent Image Registration Christensen
13/15
580 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 20, NO. 7, JULY 2001
Fig. 8. Transverse slices 068, 116, and 124 (rows top to bottom) from the full resolution CT-to-CT registration experiment. The columns from left to rightcorrespond to the template , the deformed target , the target and the deformed template . The intensities are on a range from 0 to 1.0.
of iterations are increased. A low-frequency registration result
is an approximation of the desired high-frequency registrationresult. Computing the gradient descent for a low-frequency
basis coefficient at low spatial resolution gives approximately
the same answer as using high spatial resolution but the
computational burden is much less.
C. Comparison to Other Methods
Other investigators have proposed methods for enforcing
pairwise consistent transformations. For example, Woods et
al. [27] computes all pairwise registrations of a population of
image volumes using a linear transformation model, i.e., a 33 matrix transformation. They then average the transformation
from to with all the transformations from to to . Theoriginal transformation from to is replaced with average
transformation. The procedure is repeated for all the image
pairs until convergence. This technique is limited by the fact
that it cannot be applied to two data sets. Also, there is no
guarantee that the generated set of consistent transformations
are valid. For example, a poorly registered pair of images can
adversely effect all of the pairwise transformations.
The method proposed in this paper is most similar to the
heuristic approach described by Thirion [6]. Thirions idea was
to iteratively estimate the forward , reverse , and residual
transformations in order to register the images and
. At each iteration, half of the residual is added to and half
of the residual is mapped through and added to . After
performing this operation, is close to the identity transfor-mation. The advantage of Thirions method is that it enforces
the inverse consistency constraint without having to explicitly
compute the inverse transformations as in (3). The residual
method is an approximation to the inverse consistency method
in that the residual method approximates the correspondences
between the forward and reverse transformations while the
inverse consistency method computes those correspondences.
Thus, the residual approach only works under a small deforma-
tion assumption since the residual is computed between points
that do not correspond to one another. This drawback limits
the residual approach to small deformations and it, therefore,
cannot be extended to nonlinear transformation models. On
the other hand, the approach presented in this paper can be
extended to the nonlinear case by modifying the procedure used
to calculate the inverse transformation to include nonlinear
transformations.
D. Limitations of Diffeomorphic Transformations
Diffeomorphictransformationsarevalidforregisteringimages
collected from the same individual imaged by two different
modalities such as MRI and CT, but it is not necessarily valid
when registering images before and after surgery. Likewise, a
diffeomorphic mapping assumption may be valid for registering
MRI data from two different normal individuals if the goal
http://-/?-http://-/?-http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
14/15
CHRISTENSEN AND JOHNSON: CONSISTENT IMAGE REGISTRATION 581
Fig. 9. The first two columns correspond to the absolute intensity difference between the template and the deformed target 0 (column one) and betweenthe target and the deformed template 0 (column two) in Fig. 6. The intensity range for the absolute difference is on a range from 0 to 0.776. The last twocolumns correspond to the normed difference between the forward and inverse of the reverse transformation 0 (column three), and between the reversetransformation and the inverse of the forward transformation 0 (column four). The intensity range for the normed transformation differences is from 0to 0.004538.
is to match the deep nuclei of the brain, but it may not be
valid for the same data sets if the goal is to match the sulcal
patterns.
Alternatively, diffeomorphic transformations may be used to
identify areas where two image volumes differ topologically by
analyzing the properties of the resulting transformation. For ex-
ample, consider the problem of matching an MRI image with
a tumor to one without a tumor. A possibly valid diffeomor-
phic transformation would be one that registers all of the cor-
responding brain structures by shrinking the tumor to a small
point. Such a transformation would have an unusually small Ja-
cobian which could be used to detect or identify the location ofthe tumor. Conversely, consider the inverse problem of matching
the image without the tumor to the one with the tumor. A valid
registration in this case is to register all of the corresponding
brain structures by allowing the transformation to tear (i.e.,
not be diffeomorphic) at the site of the tumor [29]. Just as valid
could be a diffeomorphic transformation that registers all of the
corresponding brain structures by allowing the transformation
to stretch at the site of the tumor.
As in the previous examples, we assume that a valid transfor-
mation is diffeomorphic everywhere except possibly in regions
where the source and target images differ topologically, e.g., in
the neighborhood of the tumor. These ideas can be extended to
nondiffeomorphic mappings by including the proper boundary
conditions around regions that differ topologically.
V. SUMMARY ANDCONCLUSION
This paper presented a new algorithm for jointly estimating a
consistent set of transformations that map one image to another
and vice versa. A new parameterization based on the Fourier
series was presented and was used to simplify the discretized
linear-elasticity constraint. The Fourier series parameterization
is simpler than our previous parameterizations and each basis
coefficient can be interpreted as the weight of a harmonic com-ponentin a singlecoordinate direction. The algorithmwas tested
on both MRI and CT data. It was found that the unconstrained
estimation leads to singular or near singular transformations. It
was also shown that the linear-elastic constraint alone is not suf-
ficient to guarantee that the forward and reverse transformations
are inverses of one another. Results were presented that sug-
gest that even though the inverse consistency constraint is not
guaranteed to generate nonsingular transformations, in practice
it may be possible to use the inverse consistency as the only
constraint. Finally, it was shown that the most consistent trans-
formations were generated using both the inverse consistency
and the linear-elastic constraints.
http://-/?-http://-/?- -
8/13/2019 Consistent Image Registration Christensen
15/15
582 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 20, NO. 7, JULY 2001
ACKNOWLEDGMENT
The authors would like to thank J. Haller and M. W. Vannier
of the Department of Radiology, The University of Iowa for pro-
viding the MRI data. They would also like to thank J. L. Marsh
of the Department of Surgery, Washington University School of
Medicine for providing the CT data. Finally, they would like to
thank M. I. Miller and S. C. Joshi for their helpful insights overthe last eight plus years.
REFERENCES
[1] F. L. Bookstein, The Measurement of Biological Shape and ShapeChange. New York: Springer-Verlag, 1978, vol. 24, Lecture Notes inBiomathematics.
[2] ,Morphometric Tools for Landmark Data. New York: CambridgeUniv. Press, 1991.
[3] A. C. Evans, C. Beil, S. Marret, C. J. Thompson, and A. Hakim,Anatomical-functional correlation using an adjustable MRI-basedregion of interest atlas with positron emission tomography, J. Cereb.
Blood Flow Metab., vol. 8, pp. 513530, 1988.[4] K. Rohr, M. Fornefett, and H. S. Stiehl, Approximating thin-plate
splines for elastic registration: Integration of landmark errors andorientation attributes, in Information Processing in Medical Imaging,A. Kuba and M. Samal, Eds. Berlin, Germany: Springer-Verlag, June1999, LCNS 1613, pp. 252265.
[5] C. A. Davatzikos, J. L. Prince, and R. N. Bryan, Image registrationbased on boundary mapping, IEEE Trans. Med. Imag., vol. 15, pp.112115, Feb. 1996.
[6] J. P. Thirion, Image matching as a diffusion process: An analogy withMaxwells demons,Med. Image Anal., vol. 2, pp. 243260, 1998.
[7] G. Subsol, Crest lines for curve-based warping, in BrainWarping. San Diego, CA: Academic, 1999, pp. 241262.
[8] C. A. Pelizzari, G. T. Y. Chen, D. R. Spelbring, R. R. Weichselbaum,and C. T. Chen, Accurate three-dimensional registration of CT, PET,and/or MR images of the brain, J. Comput. Assist. Tomogr., vol. 13, no.1, pp. 2026, 1989.
[9] P. M. Thompson and A. W. Toga, A surface-based technique forwarping three-dimensional images of the brain, IEEE Trans. Med.
Imag., vol. 15, pp. 116, Aug. 1996.[10] N. Krahnstover and C. Lorenz, Development of a point-based shaperepresentation of arbitrary three-dimensional medical objects suitablefor statistical shape modeling, in Proc. SPIEMedical Imaging1999: Image Processing, vol. 3661, K. M. Hanson, Ed., Feb. 1999, pp.620631.
[11] J. H. Downs, J. L. Lancaster, and P. T. Fox, Surface-based spatial nor-malization using convex hulls, in Brain Warping. San Diego, CA:Academic, 1999, pp. 263282.
[12] R. Bajcsy and S. Kovacic, Multiresolution elastic matching,Comput.Vis. Graph. Image Processing, vol. 46, pp. 121, 1989.
[13] R. P. Woods, J. C. Mazziotta, and S. R. Cherry, MRIPET registrationwith automated algorithm, J. Comput. Assist. Tomogr., vol. 17, no. 4,pp. 536546, July/August 1993.
[14] M. I. Miller, G. E. Christensen, Y. Amit, and U. Grenander, Mathemat-ical textbook of deformable neuroanatomies, Proc. Nat.Acad. Sci., vol.90, no. 24, pp. 11 94411 948, Dec. 1993.
[15] G. E. Christensen, R. D. Rabbitt, and M. I. Miller, Deformable tem-plates using large deformation kinematics, IEEE Trans. Image Pro-cessing, vol. 5, pp. 14351447, Oct. 1996.
[16] D. L. Collins, P. Neelin, T. M. Peters, and A. C. Evans, Automatic3-D intersubject registration on MR volumetric data in standardized ta-lairach space,J. Comput. Assist. Tomogr., vol. 18, no. 2, pp. 192205,
March/April 1994.[17] Y. Amit, A nonlinear variational problem for image matching,SIAMJ. Scientif. Computat., vol. 15, no. 1, Jan. 1994.
[18] J.G. Gee, L.Le Briquer,and C.Barillot,Probabilisticmatchingof brainimages, in Information Processing in Medical Imaging, Y. Bizais, C.Braillot, and R. Di Paola, Eds. Boston, MA: Kluwer Academic, June1995, pp. 113125.
[19] R. P. Woods, S. T. Grafton, C. J. Holmes, S. R. Cherry, and J. C. Mazz-iotta, Automated image registration: I. General methods and intrasub-ject, intramodality validation, J. Comput. Assist. Tomogr., vol. 22, no.1, pp. 139152, 1998.
[20] J. Ashburner and K. J. Friston, Spatial normalization, in BrainWarping. San Diego, CA: Academic, 1999, pp. 2744.
[21] J. Ashburner, J. L. R. Andersson, and K. J. Friston, High-dimensionalimage registration using symmetric priors, NeuroImage, vol. 9, pp.619628, 1999.
[22] G. E. Christensen, S. C. Joshi, and M. I. Miller, Volumetric transforma-
tion of brain anatomy,IEEE Trans. Med. Imag., vol. 16, pp. 864877,Dec. 1997.[23] J. Talairach and P. Tournoux, Co-Planar Stereotactic Atlas of the Human
Brain. Stuttgart, Germany: Beorg Thieme Verlag, 1988.[24] L. A. Segel, Mathematics Applied to Continuum Mechanics. New
York: Dover, 1987.[25] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens,
Multimodality image registration by maximization of mutual informa-tion,IEEE Trans. Med. Imag., vol. 16, pp. 187198, Apr. 1997.
[26] W. M. Wells III, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis,Multi-modal volume registration by maximization of mutual informa-tion,Med. Image Anal., vol. 1, no. 1, pp. 3551, 1996.
[27] R. P. Woods, S. T. Grafton, J. D. Watson, N. L. Sicotte, and J. C. Mazz-iotta, Automated image registration: II. Intersubject validation of linearand nonlinear models, J. Comput. Assist. Tomogr., vol. 22, no. 1, pp.153165, 1998.
[28] G. E. Christensen, R. D. Rabbitt, and M. I. Miller, 3-D brain map-
ping using a deformable neuroanatomy,Phys. Med. Biol., vol. 39, pp.609618, 1994.[29] M. I. Miller, A. Banerjee, G. E. Christensen, S. C. Joshi, N. Khaneja,
U. Grenander, and L. Matejic, Statistical methods in computationalanatomy, Stat. Meth. Med. Res., vol. 6, pp. 267299, 1997.
[30] G. E. Christensen, R. D. Rabbitt, M. I. Miller, S. C. Joshi, U. Grenander,T. A. Coogan, and D. C. Van Essen, Topological properties of smoothanatomic maps, in Information Processing in Medical Imaging, Y.Bizais, C. Braillot, and R. Di Paola, Eds. Boston, MA: KluwerAcademic, June 1995, vol. 3, pp. 101112.
[31] H. J. Johnson, Method for consistent linear-elastic medical image reg-istration, masters thesis, Dept. Elect. Comput. Eng., Univ. Iowa, IowaCity, May 2000.
[32] A. Toga, Ed.,Brain Warping. San Diego, CA: Academic, 1999.