a similarity measure for nonrigid volume registration

Medical Image Analysis 7 (2003) 553–564www.elsevier.com/ locate/media

A similarity measure for nonrigid volume registration using knownjoint distribution of targeted tissue: Application to dynamic CT data of

the liver

a , a b b*Jun Masumoto , Yoshinobu Sato , Masatoshi Hori , Takamichi Murakami ,c b aTakeshi Johkoh , Hironobu Nakamura , Shinichi Tamura

aDivision of Interdisciplinary Image Analysis, Osaka University Graduate School of Medicine, Suita, Osaka 565-0871, JapanbDepartment of Radiology, Osaka University Graduate School of Medicine, Suita, Osaka 565-0871, JapancSchool of Allied Health Sciences, Faculty of Medicine, Osaka University, Suita, Osaka 565-0871, Japan

Abstract

A similarity measure for nonrigid volume registration with known joint distribution of a targeted tissue is developed to process tissueslide at the boundaries between the targeted and non-targeted tissues. Pre-segmentation of the targeted tissue is unnecessary. This measureis applied to registering volumes acquired at different time-phases in dynamic CT scans of the liver using contrast materials and can bederived for the case where only the joint distribution of the targeted tissue is known. The similarity measure is formulated as a likelihoodby introducing a concept termed ‘exclusivity condition’ and embedded into a cost function for nonrigid registration to be combined withthe smoothness term. In addition, a practical method for estimating the joint distribution of the liver from unregistered clinical CT data isdescribed. We demonstrate experimentally that tissue slide is effectively processed by this proposed measure using simulated dynamic CTdata generated from a software phantom and clinical CT data of eight patients. 2003 Elsevier B.V. All rights reserved.

1 . Introduction registration is tissue slide that occurs along the boundariesbetween the liver and other tissues. Tissue slide results in

Dynamic contrast-enhanced CT scans are effective discontinuities in the 3D vector field, designated non-rigidmeans for hepatic disease diagnosis and surgical planning. deformation(Rueckert et al., 1999; Lester and Aridge,In a dynamic CT scan operation, CT volumes are typically 1999).Previous attempts to deal with this problem requireacquired at different time-phases but not necessarily within either pre-segmentation of the hepatic region(Chen et al.,a single breath-hold. Hence, these volumes are not always1999)or specification for possible locations of tissue slideregistered between different time-phases due to the respira- before registration(Wang and Staib, 2000).However,tory movements. Their registration by post-processing is segmentation of the liver from CT data is a ratherhowever highly desirable to (1) perform accurate correla- complicated task(Schenk et al., 2000),direct registrationtion between different time-phases, (2) register more between raw CT volumes without segmentation is there-accurately in 3D rendering of the liver, the portal /hepatic fore desirable in clinical practice.veins and tumors enhanced at different phases, and (3) To process tissue slide without pre-segmentation, weestimate time–density curves at every voxel, which should have developed a similarity measure for volume registra-eventually permit automatic cancer characterization(Car- tion using known joint distribution. In the dynamic CT, therillo et al., 2000). tissue contrast during scans at different time-phases varies

The present paper addresses non-rigid registration be- with the particular tissue involved. Thus, unlike cross-tween volumes acquired at different time-phases of dy- correlation measure(Lemieux et al., 1998),a similaritynamic CT scans of the liver. An important issue in hepatic measure should also be able to cope with differences in

contrast between volumes to be registered. Althoughmutual information(Bell and Sejnowski, 1995; Collignon*Corresponding author.

E-mail address: [email protected](J. Masumoto). et al., 1995; Maes et al., 1997; Wells et al., 1996)(or the

1361-8415/03/$ – see front matter 2003 Elsevier B.V. All rights reserved.doi:10.1016/S1361-8415(03)00036-7

mailto:[email protected]

554 J. Masumoto et al. / Medical Image Analysis 7 (2003) 553–564

entropy correlation coefficient: ECC(Pluim et al., 2000))is known to be useful as a similarity measure in such acase(Holden et al., 2000; Roche et al., 1999),we employknown joint distribution of a targeted tissue, originallysuggested byLeventon et al. (1998)and extended byChung et al. (2002).In our present study, we utilize theknown joint distribution of the targeted tissue only ratherthan that of the entire volume in order to register solely thetargeted tissue, for example the liver, while ignore non-targeted tissues. And we effectively overcome the tissueslide issue that is inevitable in registration of the abdomi-nal domain. A notable feature of our measure is thesimultaneous execution of both rough segmentation andregistration of the targeted tissue using its known joint Fig. 1. Example of joint distribution.distribution, therefore any pre-segmentation or manualinteraction is not needed to deal with tissue slide, animprovement from the reported methods(Chen et al., distribution P(i, j) estimated from training data sets, the1999; Schenk et al., 2000). maximum log likelihood transformationT is defined asML

The following of the paper describes the formulation ofT 5argmaxO log P I(x), J T(x) . (2)s s s dddMLour similarity measure as a likelihood and its embedding

xTinto a nonrigid registration procedure to be combined with

In previous work byLeventon et al. (1998),aimed at rigidthe smoothness constraint (Section 2). The effectiveness ofregistration of the brain in two different MR scans, thethe similarity measure for tissue slide is experimentallywhole joint distribution with various anatomical structuresdemonstrated by using simulated dynamic CT data gener-in the brain is modeled by mixture of Gaussian and Parzenated from a software phantom and clinical CT data ofwindow models and estimated from correctly alignedpatients (Section 3). The significance, advantages andtraining data sets.limitations of the similarity measure are discussed (Section

4), followed by future directions (Section 5).2 .2. Similarity measure as likelihood

The present study is aimed at the application of nonrigid2 . Methods registration of the liver at two different time-phases of

dynamic contrast-enhanced abdominal CT data. The dif-2 .1. Registration based on joint intensity model ficulties of dealing with the liver in CT data as compared

to the brain in MR data are summarized as follows.We assume that a pair of images represented asI(x) and • In formulating a procedure of nonrigid registration,TJ(x) is aligned correctly, wherex 5 [x, y, z] . We define maximization of image similarity, which is defined as a

the set of anatomical structuresG5 g , g , . . . ,g , andh j1 2 n likelihood in the present study, acts as image constraint,assume that anatomical structureg (g [G) appears with while the so-called ‘smoothness constraint’ is typicallythe intensity valuei in the imageI and j in the imageJ combined with it to stabilize the estimation of thewith a joint probabilityP(i, jug ). We also defineP(g ) be deformation field. Although the spatial variations of thethe prior probability ofg. Fig. 1 illustrates an example of deformation vector field within the liver can be as-the joint distribution. sumed to be smooth, those at the boundaries between

In the case of an unregistered pair of imagesI(x) and the liver and other tissues are often discontinuous due toJ(x), which is assumed to be modeled by the joint tissue slide. The smoothness constraint shows unwanteddistribution described above if aligned correctly, given a behaviors when it is applied across the boundarieshypothesis of registration transformationT(x) to align J along which discontinuities occur.with I, the likelihood can be calculated by • There are large variations of the joint distribution

among different patients in contrast-enhanced CT dataP(I, JuT )5P P(I(x), J(T(x)))(Fig. 2) and thus it is difficult to construct a generalx[I

model for the whole joint distribution with various5P O P(g ) ?P(I(x), J(T(x))ug ), (1)

anatomical structures in the abdomen from training datax[I g[G

sets. In addition, correctly aligned training data sets arewhereo P(g )5 1, and we make the assumption that not easy to prepare due to the difficulty of manualg[G

voxels are independent samples from the distribution (and nonrigid registration.relative positions of voxels are ignored). Given the joint In order to overcome the above difficulties, we model the

J. Masumoto et al. / Medical Image Analysis 7 (2003) 553–564 555

Fig. 2. Joint histograms of two different phases of dynamic contrast-enhanced CT data from eight different patients. The joint histograms were calculatedfrom the volume of interest described in Section 2.2.2. Significant variations among patients were observed. The horizontal and vertical axes correspond tothe intensity values of pre- and post-contrast images, respectively. Brightness represents frequency.

joint distributions and formulate a similarity measure as a in one image to identified boundaries in the other imagelikelihood in the following ways: will become extremely difficult. To overcome the difficul-• The patient-dependent joint distribution of the targeted ty, we make an assumption described below.

tissue, i.e. the liver, is estimated from the unregistered We defineP i, jug as being exclusive ifP iug ands d s ds s

data sets of the patient. P jug , which are projections ofP i, jug onto the i- ands d s ds s

• The joint distribution of other tissues is basically j-axes, respectively, have only negligible overlap withmodeled as a uniform distribution in order not to affect P iug andP jug for all t (wheret ± s), respectively. Fig.3s d s dt t

likelihood (i.e. similarity) computation. illustrates the exclusivity condition. Tissueg is exclusive3

Based on the joint distribution modeling described above, if the distributions of all other tissuesg (where t ± 3) dot

the likelihood maximization process tries to register within not overlap with the shaded area shown in Fig.3. Tissueg3

the targeted tissue regions to make the joint histogram is not regarded as exclusive in Fig.3(a) because theresemble the distribution model of the targeted tissue distributions of other tissues (g , g , g g ) are overlapped4 5 6 8

estimated beforehand while it does not force to register for with the shaded area in Fig.3(a), while tissueg is3

other non-targeted regions since any image constraint for regarded as exclusive in Fig.3(b). Mathematically,them is not provided. As a result, the unwanted behavior of P i, jug is regarded as exclusive ifP i, jug satisfies thes d s ds s

the smoothness constraint should be avoided because following conditions:contradicting image constraints in the non-targeted tissue

;(i, j) (i, j)uP i, jug .e ,h s d jsside of the boundaries are not forced.We assume that the set of anatomical structuresG O P i, tugs dsconsists of two tissues, the targeted tissue, i.e. the liver (L) t

]]]]].r andand non-hepatic tissues (O), where O represents all the O O P i, tugs dk

ttissues except the targeted tissue. In this case, Eq. (1) is g [Gk

rewritten as O P t, jugs dstP(I, J)5P (P(L) ?P(I(x), J(T(x))uL) ]]]]].r, (4)

x[I O O P t, jugs dktg [G1P(O) ?P(I(x), J(T(x))uO)). (3) k

wheree andr are sufficiently close to 0 and 1, respective-The similarity measure is derived by definingP(i, juL),ly.P(i, juO), P(L) and P(O) in the above likelihood.

We assume that the joint distribution of the targetedtissue, i.e. the liver (L), satisfies the exclusivity condition.2 .2.1. Exclusivity conditionAlthough this condition may appear to be too rigorous toWe introduce the exclusivity condition in order to ensurebe satisfied in real term, when nonrigid registration isthat boundaries between the targeted tissue and non-considered, it is unnecessary to satisfy the exclusivitytargeted tissues in local regions are identified in bothcondition in the whole image since the likelihood calcula-images to be registered. If the boundaries in either of thesetion (joint histogram evaluation) for nonrigid registration isimages are not identified, finding the corresponding parts


performed in each local area rather than in the wholeimage. Thus, the condition just needs to be satisfied in alocal area, which is practically reasonable. Fig.4 illustratesthe local exclusivity condition. Assuming the joint dis-tribution of the whole image shown in Fig.4(b), theexpected joint histograms of local areas specified in Fig.4(a) are shown in Fig.4(c)–(g). Although g is not3

exclusive in the joint distribution of the whole image,g3

can be regarded as exclusive in each local area except Fig.4(d). The exclusivity condition guarantees the intensitydistribution of the targeted tissue (liver) regions be well-separated from those of non-targeted tissues in each singleimage ofI(x) and J(x). This means that image boundariesbetween the targeted and non-targeted tissue regions aredetectable in bothI(x) and J(x). The robustness against theviolation of the exclusivity exclusive condition is ex-perimentally examined in Section 3.

The methods for estimatingP(i, juL), P(i, juO), P(L) andP(O) in Eq. (3) are as follows. Firstly,P(i, juL) is estimatedfrom an unregistered pair of images for which nonrigidregistration is performed. Secondly,P(i, juO) is derived sothat P(i, juL) satisfies the exclusivity condition. Finally,P(L) and P(O) are estimated based onP(i, juL) andP(i, juO).

2 .2.2. Estimation of P(i,juL)The joint distribution of a targeted tissueP(i, juL) needs

to be obtained before the maximum likelihood registration.Fig. 3. Exclusivity condition. (a) Tissueg is not regarded as exclusive. As described above, there are large variations inP(i, juL)3

(b) Tissueg is regarded as exclusive.3 among patient data sets. Thus, a practical method is

Fig. 4. Local exclusivity condition. (a) Original image. (b) Joint distribution in the whole image. The horizontal and vertical axes correspond to theintensity values of imageI and imageJ, respectively. (c)–(g) Joint distribution in local areas: Area 1, Area 2, Area 3, Area 4 and Area 5.


necessary for estimating the joint distribution from two area shown in Fig.3 based on the projectionsP(iuL) andunregistered images to be registered. We assume that theP( juL) of P(i, juL) onto thei-axis andj-axis, respectively.joint distribution P(i, juL) is well approximated by the This is realized by subtracting back-projections ofP(iuL)Gaussian function given by andP( juL) from the uniform distribution, after normaliza-

tion of P(iuL) and P( juL).] ] ]1 T 21 22(1 / 2)(v2v) S (v2v) Let G iuL and G juL be exp2 (i 2i9) /2s andh js d s d]]]P(vuL)5 e , (5) i 9]] ] ] ]22p uSu exph2 ( j 2j9) /2s j, where i9 and j9 are average valuesœ j 9

of the projections ofP i, juL onto the i-axis and j-axis,s d] ] ] ]]where v 5 (i, j), v 5 (i, j), (i, j) are the average values, respectively, ands ands are their standard deviations,i 9 j 9andS is covariance matrix. respectively.P i, juO is modeled ass dA method for estimating the joint distribution of the

liver region from unregistered two images at different 1]P i, juO 5 12max G iuL , G juL , (6)s d h h s d s d j jtime-phases in dynamic contrast-enhanced CT scans of the S

abdomen is established as follows. The field of viewwhere(FOV) for abdominal CT scans is usually set based on the

position of the spine. We set the volume of interest (VOI)S 5O 12max G iuL , G juL . (7)h h s d s d j jso that it would be mostly occupied by hepatic tissue (Fig.

i, j5(a)). In abdominal CT data acquired in our hospital(Osaka University Hospital, Japan), the position of the VOI 2 .2.4. Estimation of P(L) and P(O)can be fixed for each patient because the position of the The prior probabilities of tissueL and O basicallyliver relative to the spine is not greatly different in each depend on the ratio of their volumes. In nonrigid registra-] ]case. We estimate the averages (i, j) and covariance matrix tion, the prior probabilities should depend on the ratio ofS of the joint probability distributionP i, juL of Eq. (3) bys d their volumes in the local area where the likelihood isanalyzing the joint histogram obtained from the VOI of the calculated. Because it is not easy to precisely estimate this] ]two volumes. (i, j) andS are estimated from the histogram ratio, we use the following empirical method to determineregion whose center is the mode of the joint histogram and the prior probabilities. LetP(L) andP(O) bea (0<a < 1)whose horizontal and vertical widths are three times the and 12a, respectively. The value ofa is not so sensitivefull width half maximum (FWHM) values of 1D histo- to the registration results. WhenP(L) ?P(i, juL)4P(O) ?grams projected onto thei- and j-axes, respectively. P(i, juO), however, the registration processes tend to regis-Although the two volumes are not registered at this point, ter the target tissue in one image to the peripheral tissue init still provide a good approximation of the joint dis- the other image if the peripheral tissue have voxel valuestribution. Figs. 5(b) and (c) show the joint histogram somewhat similar to the target tissue in one image. Whenobtained from the VOI and the estimated joint distribution P(L) ?P(i, juL)<P(O) ?P(i, juO), the image constraint de-of the liver, respectively. rived from the known distribution of the target tissue does

not become effective. In order to avoid the these situations,we assume that the maximum probability ofP(L) ?P(i, juL)2 .2.3. Modeling of P(i,juO)is equal to that ofP(O) ?P(i, juO). Based on the assump-The joint distribution of non-targeted tissuesP(i, juO) istion, we obtain the constraint ona given bybasically modeled as a uniform distribution in order not to

provide any image constraint on the condition thatmaxP(L) ?P(i, juL)5maxP(O) ?P(i, juO). (8)P(i, juL) being exclusive should be satisfied.P(i, juO)

i, j i, jshould be modeled as uniform except for the shaded areashown in Fig.3. We model the distribution of the shaded From the above constraint,a is derived as

Fig. 5. Estimation of liver distribution. (a) Volume of interest (VOI). (b) Joint histogram in VOI. (c) EstimatedP(i, juL) (in Eq. (3)). (d) EstimatedP(i, juO)(in Eq. (3)).


1 =C]]]]] ]]a 5 . (9) F5F1m

maxP(i, juL) i=Cii, j]]]]11maxP(i, juO) ? Update the gradient vector=C

i, j , ,11? Move up to finer hierarchical grid (F →F )2 .3. Embedding similarity measure into nonrigid ? until Finest hierarchical gridregistration procedure

,whereF denotes the B-spline control point grid at, thWe embed the similarity measure as a likelihood de- hierarchical level. The method proposed by (Forsey and

, ,11scribed above into a cost function defined as Bartels, 1988)is employed for the operationF →F .In our experiments, the hierarchical grid consisted of three

C(F)5O 2C (f)1lC (f) , (10)h jsimilarity smooth levels, and the grid intervals were 42, 21 and 10.5 mm.f[F

where C (f) denotes the similarity measure termsimilarity

defined as the likelihood described above,C (f)smooth 3 . Experimentsdenotes the smoothness term, andl is a weight parameterbalancing the two terms.F denotes the parameters describ- 3 .1. Simulation experimentsing the registration transform given by

T(x)5 x 1d(x; F), (11) 3 .1.1. Synthesized data sets and evaluation methodsWe evaluated the above described method using syn-in which d(x; F) represents Free-form deformation (FFD)

thesized CT images generated from a software phantom(Rueckert et al., 1999)described by B-splines,F denotesshown in Fig.6(a). The software phantom was designed tothe whole set of the B-spline control points, andf (f [simulate the dynamic contrast-enhanced CT scans of the

F) denotes a subset of the control points involved in eachabdomen. The CT values of organs and tissues in thelocal area.phantom were set to be similar to those in real dynamic CTC (f) and C (f) are the similarity andsimilarity smooth image. Gaussian noise was added to the phantom data tosmoothness terms(Wahba, 1990)in each local area, whichsimulate noise in CT image.are given by

Parameters related to image intensities such as the CTvalues and the standard deviation (S.D.) of Gaussian noiseC (f)5O log P I(x), J(x 1d(x; f)) , (12)s s ddsimilarity

x[V at two difference time-phases (pre- and post-contrast) weresummarized in Table1. It should be noted that the CTandvalues of the liver and the vessel were different between

2 2 2 2 2 2≠ f ≠ f ≠ f two time-phases to simulate dynamic contrast enhance-]] ]] ]]C (f)5 1 1S D S DS Dsmooth 2 2 2 ment. The experiments were specially aimed at evaluating≠x ≠y ≠z

effectiveness for tissue slide and robustness against the2 2 2 2 2 2≠ f ≠ f ≠ f violation of the exclusivity condition. The CT valuesS D]] ]] ]]1 2 1 2 12 , (13)S D S D≠xy ≠xz ≠yz shown in D9 of Table 1 were used in the experiments for

evaluating the robustness against the violation of therespectively, whereV denotes a local area in whichfexclusivity condition (shown in Fig.10) only. The CTinvolves.value shown in D was used for other experiments (shownIn order to minimize Eq. (10), we use a steepest descendin Figs. 7–9). Synthesized CT images used for thealgorithm(Maes et al., 1999)using a hierarchical grid. Weexperiments are shown in Fig.6(b). Geometric parameterssuccessively refine the deformation field by using theused to generate the phantom are summarized in Table2.deformation field estimated on a coarse grid as an initial

We compared the proposed similarity measure with thefield of the estimation procedure on its next finer grid. Thisentropy correlation coefficient (ECC), which is equivalentcoarse–fine procedure is given byto normalized mutual information (NMI)(Pluim et al.,

, 2000) and Pearson product–moment cross correlation? Initialize the control pointsF (, 5 0)(NCC). For the comparison, the similarity term,? repeatC (f), in Eq. (10) was replaced by ECC and NCC.similarity? Calculate the gradient vector of Eq. (10):In each of our measure, ECC and NCC, registration

, accuracy was evaluated for various values ofl in Eq. (10).≠C(F )]]]=C 5 The following two criteria for the evaluations were used:,≠F

(1) Difference of region shape(Dice, 1945) (the? while i=Ciu.e do difference between the liver region shape inI(x) and that in? Update the control points: J(T(x))): Let V be the liver region pre-contrast imageI(x),I


Fig. 6. Software phantom and synthesized CT images used in simulation experiments. (a) Software phantom (A: air, B: rib, C: fat, D: liver, E: vessel). Itsxy-plane (left),xz-plane (middle) and 3D visualization (right) are shown. (b) Synthesized (pre-contrast) CT images generated from software phantom(xy-plane). CT values of the liver region (D) was 80 (left), 60 (middle) and 50 (right). The middle and right images were used for evaluating robustnessagainst violation of the exclusivity condition.

andV be that in registered post-contrast imageJ(T(x)). We registration processes and the true vectors. LetV be theJ

defined the difference of region shapeE as liver region, andN be the number of voxels in regionV.region V

The difference of vectorE was written asvectorV <V 2V >VI J I J 1]]]]]E 5 3100 (%). (14)region ]V <V E 5 O ud (x)2d (x)u, (15)I J vector e tNV x[V

(2) Difference of vector (the Euclidean distance be- whered (x) and d (x) are estimated and true vectors ate t

tween true and estimated deformation vector fields inside voxelx.the liver region): We defined the difference of vector as theaverage difference between the vectors obtained by the3 .1.2. Results

Fig. 7 shows the results of the simulation experiments.Our method was superior than ECC and NCC irrespectiveof l values in both evaluation criteria. In ECC and NCC,improved results were achieved when using smalll. In

T able 1these measures, the smoothness constraint was combinedCT values of organs/ tissues in software phantom and standard deviationwith two adjacent contradicting image constraints, in the(S.D.) of Gaussian noisepresent study, the rib and the liver. Thus, increasingOrgan, tissue Image 1 (S.D.) Image 2 (S.D.)smoothness (i.e. increasingl) resulted in unwanted in-

A (Air) 21000 (5) 21000 (5) fluence. Our method, however, allowed image constraintB (Rib) 200 (5) 200 (5)

be forced only for the liver region (whose joint distributionC (Fat) 50 (5) 50 (5)is given byP(i, juL)) but not for the rib region.D (Liver) 80 (5) 100 (5)

D9 (Liver) 50, 60, 70 (5) 100 (5) Fig. 8 shows color-coded vector fields inside the liverE (Vessel) 70 (5) 150 (5) region obtained using ECC, NCC, and our method. In

Image 1: pre-contrast image; image 2: post-contrast image. subfigures (b), (c) and (d) of Fig.8, the direction of the


color should be uniformly bright blue. While the middle ofthe liver region was estimated to be not moving (appearedgray) in ECC and large variation in color was observed inNCC. By contrast, the result using our measure showeduniformly bright blue, which was a highly accurate reflec-tion of the actual movement.

Fig. 9 shows overlaid displays of the estimated vectorfield and the cross-sectional images of bothI(x) andJ(T(x)). In the xz-plane of Fig.9(a), the estimated vectorfield was not correct in the middle of the liver in ECC. Inthe xy-plane of Fig.9(b), an unwanted rotation componentwas observed in NCC. Both ECC and NCC did notestimate the vector field correctly. In Fig.9(c), theestimated vector field was approximately truthful inside theliver region using our method. However, it should be notedthat registration was not performed in non-targeted tissuessince our method did not force the image constraint tonon-targeted tissues such as the ribs.

Fig. 10 shows the results of evaluation for robustnessagainst the violation of the exclusivity condition. Wesimulated several grades of the violation of the exclusivitycondition for the targeted tissue (D9 in Table1). By usingthe CT value shown in D9 of Table 1, the exclusivitycondition for the liver was violated substantially when CTvalue of D9 was set to 50 (which is the same CT value asthe fat (C)) and the CT value was set to 70 (which is thesame CT value as the vessel (E)). When the CT value wasFig. 7. Results of simulation experiments. Comparison results amongset to 50, boundaries between the liver and fat regionsECC, NCC and our method are shown. (a) Difference of region shape

with variablel. (b) Difference of vector with variablel. could barely be identified in Image 1 (pre-contrast) asshown in the right panel of Fig.6(b) because the dis-

vector was color-coded. Thex direction was coded as red, tribution of both the liver and fat is the same. They direction as green, andz direction as blue. boundaries between the liver and fat regions were the most

The magnitude of the vector was coded as brightness. important indication for registration, and thus the resultsBright color represents movement toward positive direc- were poor. In addition, when the CT value was set to 70,tion, and dark color toward negative direction. If tissue the results was inferior as compared with the other valuesdoes not move, tissue possessed the same color as the (80 and 60). Nevertheless, our method showed betterbackground. In the software phantom, the liver region performance than ECC and NCC did except when the CTshifted along the positivez-direction between the two value of D9 was set to 50.time-phases. The vector field should therefore be uniform To perform nonrigid registration, approximately 40 minand had only positivez-direction component, and thus its were required for a pair of 25632563128 volumes

Fig. 8. Visual comparison of estimated vector field. In subfigures (b), (c) and (d), the direction of the vector was color-coded. Thex direction was coded asred, y direction as green, andz direction as blue. The magnitude of the vector was coded as brightness. Bright color represents movement toward positivedirection, and dark color toward negative direction. (a) Spatial relationship between the liver and the ribs in software phantom. (b) Color-coded vector fieldobtained using ECC. (c) Color-coded vector field obtained using NCC. (d) Color-coded vector field obtained using our measure.


Fig. 9. Visual comparison of vector field obtained in simulation experiments. Upper:xy-plane. Lower:xz-plane. (a) Vector field obtained using ECC. (b)Vector field obtained using NCC. (c) Vector field obtained using our measure.

irrespective to selection of similarity measures usingPentium IV (2.2 GHz).

3 .2. Experiments using clinical data

3 .2.1. CT data sets and evaluation methodEight data sets of dynamic CT scans of the liver

acquired at Osaka University Hospital (Suita, Osaka,Japan) and National Cancer Center (Tokyo, Japan) wereused for performance evaluation. The imaging conditionsare summarized in Table3. Each CT data set originallyconsisted of volumes at three or four different time-phases,of which two phases were registered. One was before theinjection of the contrast material (pre-contrast) or the earlyarterial phase (when the effect of the contrast material is

T able 2Geometric parameters of software phantom

Parameter Value

Matrix size 25632563128Voxel size 13132 mmDiameter of body 200 mmDiameter of ribs 20 mmNumber of ribs 8Diameter of vessels 10 mmNumber of vessels 8Diameter of liver 150 mm

Fig. 10. Results of experiments for evaluating robustness against viola-Height of liver 100 mm

tion of exclusivity condition. (a) Difference of region shape with variablez-Position of liver 80 mm (Pre), 100 mm (Post)

l. (b) Difference of vector with variablel.


T able 3Imaging conditions of clinical CT data

Case[ Thickness (interval) FOV Phase 1 Phase 221 2.5 mm (1.25 mm) 34334 cm Early arterial Portal22 2.5 mm (1.25 mm) 34334 cm Early arterial Portal23 2.5 mm (1.25 mm) 34334 cm Early arterial Portal24 2.5 mm (1.25 mm) 34334 cm Early arterial Portal25 2.0 mm (1.0 mm) 28328 cm Pre-contrast Portal26 2.0 mm (1.0 mm) 32332 cm Pre-contrast Portal27 2.0 mm (1.0 mm) 32332 cm Pre-contrast Portal28 2.0 mm (1.0 mm) 32332 cm Pre-contrast Portal

pre-contrast imageI(x) and those in the registered post-small); the other was the portal phase (when the effect ofcontrast imageJ(T(x)). Let p(x) be the contour inJ(T(x)),contrast enhancement is large, i.e. post-contrast). Becauseand q(x) be the contour inI(x)). E was defined asthe volumes at these two phases were not acquired in a contour

single breath-hold, there was considerable deformation1

between them due to respiratory movement. The original ]]E 5 O min ua 2 bu, (16)contour N b[q(x)p(x) a[p(x)matrix size was 512351231502200 (voxels).To assess the quality of the registration, we used the

whereN is the number of voxels inp(x). In addition top(x)contours of the liver in both phase images extracted by anthe evaluation of the registration error, we visually evalu-

expert, and we defined the registration errorE as thecontour ated the acceptability of the estimated deformation field.average distance between the contours of the liver in

3 .2.2. Results Fig. 11 shows the evaluation results for the eight data

sets. The registration error was smaller in our measure thanin ECC and NCC for all eight cases. While considerableimprovement was observed in the NCC measure, little wasobserved in the ECC measure.

Fig. 12 shows the deformation field. In this case, tissueslide between the ribs and the liver occurred. The livershifted downward while the ribs on the left side shiftedupward due to respiratory movement and the ribs on theright side remained steady. Since NCC forced to registerboth the ribs and the liver, the left part of the liver wasaffected by upward movement of the left ribs and the

Fig. 11. Registration error for eight dynamic CT data sets. Registrationlower right part by the right stable rib in Fig.13(a). Ourerror was defined as the average distance between the contours of themethod was not affected by the movement of non-targetedliver in pre-contrast imageI(x) and those in the registered post-contrast

image J(T(x)). tissue in Fig.13(b) because the joint distribution of the ribs

Fig. 12. Example of CT image used in our experiment. (a) Pre-contrast image (phase 1). (b) Post-contrast image (phase 2). (c) Movements of tissues. Pre-and post-contrast images are superimposed. Arrows denote the directions of tissue movement between pre- and post-contrast phases.


targeted tissue in the FOV was roughly determined. Weperformed experiments in order to examine the sensitivityto inaccuracy of VOI setting using nine CT data sets. Weadded Gaussian noise (the standard deviation was tenpixels) to the center position and the size of VOI. The

] ]standard deviations of the estimated averages (i, j) de-scribed in Section 2.2.2 were less than three (HU), and weconsidered that these differences did not considerablyaffect the final result. We confirmed that the liver regioninside the VOI was more than 50% of the entire VOI andthe estimation was successful with all the data sets used inour experiments.

The method developed in the present study assumes thatthe targeted tissue satisfies the exclusivity condition. Innonrigid registration, this condition can only be satisfied ineach local area, and considered that tissue boundaries areable to be identified in each of the two image to beregistered. In the simulation experiments, we evaluatedrobustness against the violation of the exclusivity con-dition. Except the case in which the exclusivity conditionwas entirely violated, our method was reasonably effectivein general even if the exclusivity condition was partiallyviolated.

In our experiments, we used pre-contrast and post-contrast images (Fig. 12). The intensities inside the liverwere mostly constant in the pre-contrast image, whilevessels inside the liver were enhanced in the post-contrastimage. However, the intensity patterns caused by vesselenhancement in the post-contrast image did not provideFig. 13. Estimated deformation vector field obtained from clinicalsufficient information for registering the inner part of thedynamic CT data. (a) Vector field obtained using NCC. (b) Vector fieldliver because these patterns did not exist in the pre-contrastobtained using our measure.

image. Therefore, the boundaries of the liver were regis-tered appropriately based on the similarity measure de-

(bone tissue) was different from the known distribution scribed in the present study while the deformation field inand these tissues were not registered. the inner part of the liver was considered to be estimated

mostly based on B-spline interpolation. One approach toaddress this problem would be the use of a biomechanical

4 . Discussion interpolation method rather than that of B-splines. Withrespect to this issue,Schnabel et al. (2001)suggested the

For application to dynamic CT data of the liver, our evaluation using finite element method.measure showed superior results than ECC and NCC. We Potentially, the method established in the present studybelieve the reason is that the method can more effectively is equally useful for the lung. In movement analysis of thedeal with tissue slide. Using our measure, the registration lung, tissue slide can occur between the lung and the ribsprocess does not try to register the entire volume but only as well due to the known difference between the ribthose regions with the known joint distribution. It simply movement and the lung movement.ignores non-targeted tissues. Consequently, it is not affect-ed by discontinuities in the deformation field occurring atthe boundaries of two tissues. Although the rib boundary 5 . Conclusionwas not well registered using our method, this should notbe considered disadvantageous because the aim is to We have developed a novel similarity measure forregister the targeted (i.e. liver) tissue only. volume registration when the joint distribution of a

Our similarity measure assumes that the joint distribu- targeted tissue is known. Application of this measure totion of the targeted tissue is known. One related issue is dynamic CT data sets of the liver confirms its effectivenesshow this should be estimated. The method using histogram in dealing with tissue slide without the need for anyanalysis of the fixed VOI, explained in Section 2.2.1, was pre-segmentation or manual interaction. We have estab-quite effective so long as the relative position of the lished additionally a method for estimating a good approxi-


D ice, L.R., 1945. Measures of the amount of ecologic associationmation of the joint distribution of the targeted tissue frombetween species. Ecology 26, 297–302.two unregistered volumes. Our measure works well for

F orsey, D.R., Bartels, R.H., 1988. Hierarchical B-spline refinement. ACMregistering the boundaries of the targeted tissue, while theTrans. Comput. Graphics 22 (4), 205–212.

registration of the inner part of the tissue is estimated H olden, M., Hill, D.L.G., Denton, E.R.E., Jarosz, J.M., Cox, T.C.S.,mostly based on B-spline interpolation. Future directions Rohlfing, T., Goodey, J., Hawkes, D.J., 2000. Voxel similarity mea-

sures for 3-D serial MR brain image registration. IEEE Trans. Med.may include the development of a post-processing methodImag. 19 (2), 94–102.that is able to register the inner part of a tissue by taking

L emieux, L., Wieshmann, U.C., Moran, N.F., Fish, D.R., Shoovon, S.D.,intensity patterns into account and the demonstration of the1998. The detection and significance of subtle changes in mixed-signal

improvement for accuracy of segmentation of the liverbrain lesions by serial MRI scan matching and spatial normalisation.

region and tissue classification including tumor detection Medical Image Analysis 2 (3), 227–242.inside the liver region by using registered multi-phase L ester, H., Aridge, S.R., 1999. A survey of hierarchical non-linear

medical image registration. Pattern Recogn. 32, 71–86.dynamic CT data.L eventon, M.E., Grimson, W.E.L., 1998. Multi-modal volume registration

using joint intensity distributions. Lecture Notes in Computer Science,Vol. 1496, pp. 1057–1066, MICCAI’98.

A cknowledgements M aes, F., Vandermeulen, D., Suetens, P., 1999. Comparative evaluation ofmultiresolution optimization strategies for multimodality image regis-

This work was partly supported by JSPS Research for tration by maximization of mutual information. Medical ImageAnalysis 3 (4), 373–386.the Future Program JSPS-RFTF99I00903 and JSPS Grant-

M aes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.,in-Aid for Scientific Research (B)(2) 12558033. The1997. Multimodality image registration by maximization of mutualauthors would like to thank Dr Shigeru Nawano atinformation. IEEE Trans. Med. Imag. 16, 187–198.

Radiology Division, National Cancer Center Hospital East, P luim, J.P.W., Maintz, J.B.A., Viergever, M.A., 2000. InterpolationJapan, for providing clinical CT data sets of case numbers artifacts in mutual information-based image registration. Comput. Vis.5, 6, 7 and 8. Image Understand. 77, 211–232.

R oche, A., Malandain, G., Ayache, N., Prima, S., 1999. Toward a bettercomprehension of similarity measures used in medical image registra-tion. Lecture Notes in Computer Science, Vol. 1679, pp. 555–566,

R eferences MICCAI’99.R ueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes,

B ell, A.J., Sejnowski, T.J., 1995. An information-maximisation approach D.J., 1999. Nonrigid registration using free-form deformations: appli-to blind separation and blind deconvolution. Neural Comput. 7, 1129– cation to breast MR images. IEEE Trans. Med. Imag. 18 (8), 712–721.1159. S chenk, A., Prause, G., Peitgen, H.O., 2000. Efficient semiautomatic

C arrillo, A., Duerk, J.L., Lewin, J.S., Wilson, D.L., 2000. Semiautomatic segmentation of 3D objects in medical images. Lecture Notes in3-D image registration as applied to interventional MRI liver cancer Computer Science, Vol. 1935, pp. 186–195, MICCAI2000.treatment. IEEE Trans. Med. Imag. 19 (3), 175–185. S chnabel, J.A., Tanner, C., Smith, A.C., Leach, M.O., Hayes, C.,

C hen, M., Kanade, T., Pomerleau, D., Schneider, J., 1999. 3D deformable Degenhard, A., Hosem, R., Hill, D.L.G., Hawkes, D.J., 2001. Valida-registration of medical images using a statistical atlas. Lecture Notes tion of non-rigid registration using finite element methods. Lecturein Computer Science, Vol. 1679, pp. 621–630, MICCAI’99. Notes in Computer Science, Vol. 2082, pp. 344–357, IPMI’01.

C hung, A.C.S., Wells, W.M., Norbash, A., Grimson, W.E.L., 2002. Multi- W ahba, G., 11990. Spline models for observational data. Series inmodal image registration by minimising Kullback–Leibler distance. Applied Mathematics, Vol. 59. SIAM, Philadelphia.Lecture Notes in Computer Science, Vol. 2489, pp. 525–532, W ang, Y., Staib, L.H., 2000. Physical model-based non-rigid registrationMICCAI2002. incorporation statistical shape information. Medical Image Analysis 4,

C ollignon, A., Verndermeulen, D., Suetens, P., Marchal, G., 1995. 3D 7–20.multi-modality medical image registration using feature space cluster- W ells III, W.M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R., 1996.ing. Lecture Notes in Computer Science, Vol. 905, pp. 195–204, Multi-modal volume registration by maximization of mutual infor-CVRMed’95. mation. Medical Image Analysis 1 (1), 35–51.

a similarity measure for nonrigid volume registration

Documents