GEODESIC REGRESSION OF IMAGE AND SHAPE DATA FORIMPROVED MODELING OF 4D TRAJECTORIES

James Fishbaugh1, Marcel Prastawa1, Guido Gerig1* , Stanley Durrleman2

1Scientific Computing and Imaging Institute, University of Utah2INRIA/ICM, Pitie Salpetriere Hospital, Paris, France

ABSTRACT

A variety of regression schemes have been proposed on im-ages or shapes, although available methods do not handlethem jointly. In this paper, we present a framework for jointimage and shape regression which incorporates images aswell as anatomical shape information in a consistent manner.Evolution is described by a generative model that is the analogof linear regression, which is fully characterized by baselineimages and shapes (intercept) and initial momenta vectors(slope). Further, our framework adopts a control point pa-rameterization of deformations, where the dimensionality ofthe deformation is determined by the complexity of anatom-ical changes in time rather than the sampling of the imageand/or the geometric data. We derive a gradient descent al-gorithm which simultaneously estimates baseline images andshapes, location of control points, and momenta. Experi-ments on real medical data demonstrate that our frameworkeffectively combines image and shape information, resultingin improved modeling of 4D (3D space + time) trajectories.

1. INTRODUCTION

Analysis of longitudinal data incorporating both spatial andtemporal information is essential for various clinical taskssuch as predicting patient outcome and measuring efficacyof different therapeutic strategies. A crucial tool for longi-tudinal analysis is regression of observed data, which enablesinterpolation to generate continuous evolution models as wellas extrapolation to predict future observations. Regressionmodels are also necessary for conducting population studiescomparing the change trajectories of different subjects.

In medical imaging, it is important to consider image datain anatomical context, which motivates regression on imageand shape data in different combinations (a multi-object com-plex). A variety of regression schemes have been proposed onimages or shapes, although available methods do not handlethem jointly. For example, the extension of kernel regression

*Supported by grants: RO1 HD055741 (ACE, project IBIS), U54EB005149 (NA-MIC), and U01 NS082086 (HD) and the Utah Science Tech-nology and Research (USTAR) initiative at the University of Utah.

for image data [1] or piecewise linear regression for time se-ries of images [2] and shapes [3]. Combining intensity andgeometric information has been explored for registration [4].

To conduct statistical analysis on 4D (3D space + time)data, it is particularly useful to consider compact generativeregression models which have low number of parameters sit-uated at only one chosen time point. Geodesic regression issuch a model and is fully characterized by baseline imagesand shapes (the intercept) and the tangent vector defining thegeodesic at the baseline objects (the slope). Geodesic regres-sion frameworks for images [5, 6] and for shapes [7] havebeen proposed using the LDDMM setting. However, no no-tion of how to combine images and shape data is provided.

We propose a novel geodesic regression framework thatleverages image and shape data together to estimate a sin-gle deformation of the ambient space. We use the currentsrepresentation for geometric data that allows flexible repre-sentation of a wide variety of shape objects such as pointsets, curves, or surface meshes, without the need for pointcorrespondence between shapes. Compared to image regres-sion alone, shape data provides anatomical information thatconstrains the regression, especially in cases where imageshave low contrast, by placing larger weights on regions withanatomical importance. Compared to shape regression alone,image information provides data in areas where segmenta-tions are not available, as well as providing context to re-gions surrounding anatomical objects. Our framework usesthe control-point parameterization of geodesic flows intro-duced in [8], which makes the parameterization of the defor-mation independent from the data. This allows us to keep areasonable dimension of the parameterization, which is deter-mined by the complexity of anatomical changes in time, andnot the sampling of the data. We therefore combine imageand shape data without introducing a complexity overhead.

2. METHODOLOGY

We perform regression on observed time-series data of imagesIi and shapes Xi, each acquired at time-point ti. Shape datamay consist of a mix of point sets, curves, or surface mesheswhere all vertices are concatenated into one vector Xi.

978-1-4673-1961-4/14/$31.00 ©2014 IEEE 385

We use the control point formulation of [8] to generategeodesic flows of diffeomorphisms. Let S0 = {c0,k, α0,k}be a set of momentum vectors α0,k attached to control pointsc0,k distributed in the image domain. Geodesic flows are com-puted by evolving control points and momenta by integratingthe following Hamiltonian equations over the time interval ofinterest:

ck(t) =N∑

i=1

K(ck(t), ci(t))αi(t)

αk(t) = −

N∑

i=1

αk(t)Tαi(t)∇1K(ck(t), ci(t))

(1)

with initial conditions ck(0) = c0,k and αk(0) = α0,k (as-suming starting time-point to be 0), and K is a Gaussian ker-nel with variance σ2

V which controls the spatial scale of de-formation. For simplicity, we write these equations as S(t) =F (S(t)) with S(0) = S0. The convolution of the momentadefines the following time-varying velocity field: v(t, x) =∑N

i=1 K(x, ci(t))αi(t) for any point x in the domain. Thevelocity is used to deform the domain: a particle at point xat time 0 moves to φ(t, x) at later time t, where φ(t, x) fol-lows the integral curve of ∂φ(t,x)

∂t= v(t, φ(t, x)) starting with

φ(0, x) = x. In this formulation, the velocity of the particleis given by the field v(t, .) at its current location. It has beenshown in [9] that for all t, φ(t, .) is a 3D diffeomorphism.

Following this model, the vertices of a given baselineshape complex concatenated into a vector X0 move at time t

to X(t) = φ(t,X0), which satisfies the ordinary differentialequation (ODE): X(t) = v(t,X(t)) with X(0) = X0. Tomake explicit the dependency of the equation of motion onS(t), we write it as: X(t) = G(X(t),S(t)).

A given baseline image I0 is also deformed by the flow

Fig. 1. Conceptual overview of geodesic regression on multi-object complexes containing both image and shape data. Theframework estimates parameters at t = 0 which consist of thebaseline image I0 and shape X0 along with the deformationmodel parameterized by control points c0 and initial momentaα0 such that overall distance between the deformed objectsand the observations are minimal.

of diffeomorphisms and its trajectory is given as I(t) = I0 ◦

φ(t, .)−1. The inverse flow satisfies the equation ∂φ(t,.)−1

∂t=

−dφ(t, .)−1v(t, .). For the sake of simplicity, we denoteY (t, .) = φ(t, .)−1, a L2 function that maps the point x toits position at time t under the inverse flow φ−1(t, x). Thismaps satisfies Y (t, .) = −dY (t, .)v(t, .) = H(Y (t, .),S(t)),where we make explicit the dependency on S(t). At time t,the intensity of the warped baseline image at voxel position x

is given by I(t, x) = I0(Y (t, x)) using 3D interpolation.A conceptual overview of our framework is shown in

Fig. 1 where regression is performed by minimizing theoverall distance between the observations and the deformedbaseline objects (shapes and/or images). Let d(X(ti), Xi)be a metric between the deformed baseline shape complexX0 at time ti and the data shape complex Xi. This metricmay be a weighted sum over each component of the shapecomplex of the currents metric between sets of curves or sur-face meshes. This term essentially depends on X(ti) and isdenoted A(X(ti)). Similarly, we have a metric d(I(ti), Ii)denoted as B(Y (ti, .)) that is the sum of squared differencesbetween the deformed baseline image I0 ◦ Y (ti, .) and theobserved image Ii.

The geodesic regression problem amounts to finding thedeformation parameters S0 and baseline anatomical configu-ration (I0, X0) such that the following criterion is minimized:

E(S0, I0, X0) =∑

ti

(

λItiA(X(ti)) + λSti

B(Y (ti, .)))

+ L(S0)(2)

subject to

S(t) = F (S(t)) S(0) = S0

X(t) = G(X(t),S(t)) X(0) = X0

Y (t, .) = H(Y (t, .),S(t)) Y (0, .) = Id

(3)

where the regularizer L(S0) =∑N

i,j=1 αT0,iK(c0,i, c0,j)α0,j

is the squared norm of initial velocity and weights on imageand shape matching λIti

and λSti.

As shown in the supplemental material (www.cs.utah.edu/˜jfishbau/docs/isbi2014_eqns.pdf), thegradient is computed by integrating 3 linear ODEs withsource terms from final time-point Tf back to time-point 0:

∇S0E = ξ(0) +∇S0

L ∇X0E = η(0)

1

2∇I0E =

∑

tj

SplatY (tj ,.)(I0 ◦ Y (tj , .)− Ii)

with

η(t) = −∂1G(t)T η(t)−∑

ti

∇X(ti)Aδ(t− ti)

θ(t) = −∂1H(t)†θ(t)−∑

ti

∇Y (ti,.)B δ(t− ti)

ξ(t) = −∂2G(t)T η(t)− ∂2H(t)†θ(t)− dS(t)FT ξ(t)

386

with final conditions η(Tf ) = θ(Tf ) = ξ(Tf ) = 0.The vector η is same size as X0, which brings back to time

t = 0 the gradients of the data matching terms, and is used toupdate the position of the vertices of the baseline shape com-plex. Similarly θ is of the same size as Y (0, .) (an image ofvectors in practice) which integrates the successive gradientsof the image matching terms that acts as jumps in the differ-ential equation. Finally, ξ is a variable of the same size as S0

which is used at time t = 0 to update the deformation param-eters (the position of the control points and their momentumvectors). The gradient with respect to the baseline image in-volves the splatting of the current residual images at positionsY (ti, .) as done in [8].

3. RESULTS AND DISCUSSION

Pediatric Brain Development: We explore the impact ofjoint image and shape regression in modeling pediatric braindevelopment. The data consists of T1W images of the samehealthy child observed at 6, 12, and 25 months of age. Re-gression on images alone is difficult in this case due to thevery low contrast in the 6 month old image. Despite the lowcontrast, tissue segmentations can still be reliably and consis-tently estimated [10]. We estimate a geodesic model usingonly T1W images and a model jointly on images and whitematter surfaces to emphasize the development of the tissueinterface. We initialize 120 control points on a regular gridwith the deformation kernel σV = 20 mm. Finally, due tolimited contrast at 6 months, we estimate the baseline at 25months and follow the evolution backwards in time.

The results of geodesic regression are shown for severalsnapshots in time in Fig. 2. The model estimated using onlyimages mostly captures the scale change, but does not cap-

Fig. 2. Images and deformations estimated by geodesic re-gression using images alone (top) and jointly on images andwhite matter surfaces (bottom). Regression jointly on imageand shape results in a more realistic evolution which capturesdetailed changes in brain tissue in addition to the increase inbrain size. In both cases, geodesic regression was estimatedbackwards in time.

0 5 10 15 20 253200

3300

3400

3500

3600Model estimated from images alone

Cau

date

Vol

ume

(mm

3 )

Time from baseline (months)

Left

Right

0 5 10 15 20 253200

3300

3400

3500

3600Model estimated jointly on image and shapes

Cau

date

Vol

ume

(mm

3 )

Time from baseline (months)

Left

Right

Fig. 3. Caudate volume extracted continuously after regres-sion compared to observed caudate volumes (circles and x’s).Volume is measured continuously from the modeled shapetrajectories, not fitted to discrete volume measurements. Themodel estimated on images alone fails to capture the volumeloss. Evolution of caudates for the image only model is notestimated, but instead we shoot the baseline caudate shapesalong the geodesic estimated from images alone. Note: mea-surements extracted continuously from non-linearly deform-ing shapes can produce either linear or non-linear trends withno prior assumption of linearity.

ture much deformation in the interior of the brain. The modelestimated jointly on image and shape captures more detaileddevelopment as white matter stretches and expands.

Neurodegeneration in Huntington’s Disease: Next, we in-vestigate the application of joint image and shape regressionto Huntington’s disease (HD) where accurate 4D models areneeded to measure the effectiveness of therapies or drug treat-ments. In HD, degeneration of the caudate has been shown tobe significant [11]. Here we explore T1W image data from asingle patient diagnosed with HD scanned at 58, 59, and 60years of age. Sub-cortical structures are segmented, manuallyverified, and cleaned. Models are estimated using only T1Wimages as well as T1W images plus caudate surfaces. Controlpoints are initialized on a regular grid with 10 mm spacingwith kernel σV = 10 mm.

The trajectory of caudate volume extracted after regres-sion is shown in Fig. 3. The model estimated from imagesalone fails to capture the volume loss observed in both cau-dates, and rather, shows an increase in right caudate volume.By incorporating caudate shape data in model estimation, we

387

Fig. 4. Top) Evolution estimated on images alone. Evolution of caudates are not estimated, but instead we shoot the baselinecaudate shapes along the estimated geodesic. Bottom) Evolution estimated jointly using images and caudate shapes. Regressionon images alone results in a slight expansion of ventricles, but does not capture the shrinking of caudates. Our method is ableto capture both the expansion of ventricles and the shrinking of caudates.

are able to capture the shrinking of the caudates. The corre-sponding expansion of the ventricles is also captured, shownin Fig 4, due to the inclusion of imaging data. By incorporat-ing shape and image information jointly, we are able to modelboth the expansion of the ventricles and the degeneration ofthe caudates. Accurate models of change are essential whenextrapolating beyond the observation time interval, which canprovide insight into disease progression.

Conclusions: We presented a novel geodesic regressionframework that jointly considers image and shape informationin the LDDMM framework, where dense diffeomorphismsare built using a control point formulation. This formulationdecouples deformation parameters from input object parame-ters (e.g., voxels, surface points) providing greater flexibilityand consistency in mapping different object types across time.Our regression model seamlessly handles images and multi-object complexes consisting of points, curves, and/or surfacesin different combinations. Experiments show that our frame-work effectively combines image and shape information toestimate a single deformation of the ambient space, resultingin improved modeling of 4D trajectories.

4. REFERENCES

[1] B.C. Davis, P.T. Fletcher, E. Bullitt, and S. Joshi, “Pop-ulation shape regression from random design data,” inICCV. 2007, pp. 1–7, IEEE.

[2] A.R. Khan and M.F. Beg, “Representation of time-varying shapes in the large deformation diffeomorphicframework,” in ISBI. 2008, pp. 1521–1524, IEEE.

[3] S. Durrleman, X. Pennec, A. Trouve, J. Braga, G. Gerig,and N. Ayache, “Toward a comprehensive frameworkfor the spatiotemporal statistical analysis of longitudinalshape data,” IJCV, pp. 1–38, 2012.

[4] P. Cachier, E. Bardinet, D. Dormont, X. Pennec, andN. Ayache, “Iconic feature based nonrigid registration:the pasha algorithm.,” Computer Vision and Image Un-derstanding, vol. 89, no. 2-3, pp. 272–298, 2003.

[5] M. Niethammer, Y. Huang, and F.X. Vialard, “Geodesicregression for image time-series,” in MICCAI, 2011,vol. 6892 of LNCS, pp. 655–662.

[6] N.P. Singh, J. Hinkle, S. Joshi, and P.T. Fletcher, “Avector momenta formulation of diffeomorphisms for im-proved geodesic regression and atlas construction,” inISBI, 2013, pp. 1219–1222.

[7] J. Fishbaugh, M. Prastawa, G. Gerig, and S. Durrle-man, “Geodesic shape regression in the framework ofcurrents,” in IPMI, 2013, vol. 23, pp. 718–729.

[8] S. Durrleman, M. Prastawa, G. Gerig, and S. Joshi, “Op-timal data-driven sparse parameterization of diffeomor-phisms for population analysis,” in IPMI, 2011, vol.6801 of LNCS, pp. 123–134.

[9] S. Joshi and M. Miller, “Landmark matching via largedeformation diffeomorphisms,” IEEE Trans. Image Pro-cessing, vol. 9, no. 8, pp. 1357–1370, 2000.

[10] A. Vardhan, M. Prastawa, S. Gouttard, J. Piven, andG. Gerig, “Quantifying regional growth patternsthrough longitudinal analysis of distances between mul-timodal MR intensity distributions,” in ISBI, 2012, pp.1156–1159.

[11] EH. Aylward, AM. Codori, A. Rosenblatt, M. Sherr,J. Brandt, OC. Stine, PE. Barta, GD. Pearlson, and CA.Ross, “Rate of caudate atrophy in presymptomatic andsymptomatic stages of huntington’s disease,” MovementDisorders, vol. 15, no. 3, pp. 552–560, 2000.

388