fast mesh-based medical image registration
TRANSCRIPT
Fast Mesh-Based Medical Image Registration
Ahmadreza Baghaie, Zeyun Yu, Roshan M. D’souza
College Of Engineering And Applied Science, University Of Wisconsin - Milwaukee
December 10, 2014
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 1 / 25
Content
1 Introduction and Literature Review
2 Image Registration: General Framework
3 Motivation
4 Proposed Method
5 Results and Discussion
6 Conclusion
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 2 / 25
Introduction and Literature Review
Introduction
Medical Image Registration is an active area in the field of imageprocessing with applications ranging from image mosaicing in retinalimages to slice interpolation [1] etc.
Image registration problems [2]:
Multi-View Analysis: image mosaicing, shape recovery from stereo;
Multi-Temporal Analysis: monitoring of the healing therapy andtumor evaluation;
Multi-Modal Analysis: anatomical (MRI)/functional (PET)monitoring in radiotherapy and nuclear medicine;
Scene to Model Registration: patient’s image to anatomical atlases.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 3 / 25
Introduction and Literature Review
Further categorization?
Being more focused on the implementation aspects of image registration[3]:
Parametric Registration, based on a finite set of parameters orimage features:
rigid/affine registration;landmark-based registration;principal axes-based registration;FFT based registration;etc...
Non-Parametric Registration:Diffusion registration;Elastic registration;Curvature registration;etc...
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 4 / 25
Image Registration: General Framework
Image Registration: General Framework
In general, image registration is considered as an ill-posed inverse problem.Therefore the process of solving consists of three components [4]:
a deformation model;
an objective function to be optimized;
an optimization method.
A general objective function can be defined as:
E [u] = D[Re,Te u] + αS [u] (1)
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 5 / 25
Image Registration: General Framework
Similarity or distance measures D:
Sum of Squared Differences (SSD);
Mutual Information (MI);
Cross-Correlation (CC);
etc...
Regularization term S :
diffusion operator;
elastic operator;
curvature operator;
etc...
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 6 / 25
Motivation
Motivation
In case of non-rigid image registration methods, the deformation is localrather than global. Therefore the problem has a BIG number of degrees offreedom (DOF) in the optimization process.
MORE DOF ⇒ MORE COMPUTATIONAL COMPLEXITY
How can we solve it?
GPU [5];
Multi-resolution [6];
Octree based [7];
Triangular mesh based !!!
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 7 / 25
Proposed Method
Proposed Method
Assume Te and Re as input images, and a set of triangles defined on thetemplate image (V ,T ), where V is a nV × 2 matrix containing thecoordinates of nV nodes and T is a nT × 3 matrix, each line containingthe indexes of nodes creating each one of the nT triangles.
Re represents a continuous domain of X ∈ Ω, hence Re(V ) = Re(X )|X=V .
Here a slightly different approach will be considered in which instead ofapplying the smoothing term at the same time as update of thedisplacement filed, this will be done after each iteration using a diffusionprocess.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 8 / 25
Proposed Method
The energy functional is therefore defined as follows:
E [u(V )] = D[Re(X )|X=V+u,Te(V ) u(V )] (2)
where E and D represent the energy functional and the distance measurerespectively. Also, the operator is defined as:
Te(V ) u(V ) = Te(V + u(V )) (3)
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 9 / 25
Proposed Method
The Sum of Squared Differences (SSD) is used which can be defined asfollows:
D(Re(X )|X=V+u,Te u) =1
2||Re(X )|X=V+u − Te u||2
=1
2
∑i=1:nV
(Te(Vi ) u(Vi )− Re(Xi )|Xi=Vi+ui )2
(4)
where the last summation is computed over all of the mesh nodes.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 10 / 25
Proposed Method
Gradient Descent (GD) for minimization:
uk+10 = uk1 − τ 5uk1
E [uk1 ] (5)
where τ is the step size (here 0.005) and 5uk1is the gradient operator with
respect to variable uk1 .
Gateaux derivative of the distance measure results in:
5uk1E [uk1 ] = 5uk1
D
= (Te(V + uk1 )− Re(X )|X=V+u).5uk1Te(V + uk1 )
(6)
where 5uk1Te(V + uk1 ) needs to be computed on mesh nodes.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 11 / 25
Proposed Method
For smoothing the displacements on the mesh, a diffusion process needs tobe solved on the mesh nodes. This diffusion process can be modeled asfollows:
∂uk+10
∂t= λ4 uk+1
0 (7)
where 4 represents the Laplacian operator on mesh nodes. This diffusionprocess is solved using a forward difference time-stepping approach.Assuming the time step as 1, we have:
uk+11 = uk+1
0 + λ4 uk+10 (8)
where 0 < λ < 1 is the smoothing parameter defined by the user (here0.8).
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 12 / 25
Proposed Method
Gradient Discretization
Consider node Vi and its 1-ring (N1) neighbor nodes. Assuming triangleTj created by nodes [ViVjVk ] as one of the triangles surrounding Vi , theapproximation of the gradient of the function f on Tj will be:
5fTj=
1
4A2j
(fi [(−→Vij ,−→Vjk)(Vk − Vi ) + (
−→Vik ,−→Vkj)(Vj − Vi )]
+fj [(−→Vji ,−→Vik)(Vk − Vj) + (
−→Vjk ,−→Vki )(Vi − Vj)]
+fk [(−→Vkj ,−→Vji )(Vi − Vk) + (
−→Vki ,−→Vij)(Vj − Vk)]
) (9)
where fi is the function value on node Vi , Aj is the area of the triangle Tj ,−→Vij is the vector connecting nodes i and j and (−→a ,
−→b ) gives the dot
product of vectors −→a and−→b .
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 13 / 25
Proposed Method
Having the approximation of the gradient on surrounding triangles, theapproximate gradient for node Vi can be computed as follows:
5f (Vi ) =1
A(Vi )
∑j∈N1(i)
Aj 5 fTj(10)
where A(Vi ) =∑
j∈N1(i)Aj . For a complete analysis on the approximation
error the reader is referred to [8]. The areas of triangles should becomputed at the beginning of each iteration.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 14 / 25
Proposed Method
Diffusion-Based Smoothing
Taking the same approach as [9], the Laplacian operator on a mesh can beapproximated by the so-called umbrella operator on each node as follows:
4u(Vi ) =1
mi
∑j∈N1(i)
u(Vj)− u(Vi ) (11)
where mi is the valence (number of 1-ring neighbors) of node Vi . Thisoperator can be defined in a matrix form as follows:
4u = (ALap − I )u (12)
where I is the identity matrix and ALap is a sparse nV × nV matrix whichits non-zero elements are defined as follows:
ALapij =
1
mi, for all j ∈ N1(Vi) (13)
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 15 / 25
Proposed Method
Considering (8) and (13) together with a few manipulations, the diffusionprocess can be simplified as a weighted average of the displacements ofthe 1-ring neighborhood of each node:
uk+11 =
((1− λ)I + λALap
)uk+10 (14)
The above equation can be applied iteratively for further smoothness ofthe displacement field on the mesh nodes. Here, only one iteration ofsmoothing is applied.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 16 / 25
Proposed Method
Proposed Algorithm
Inputs: Re, Te, (V ,T ) defined on the template image, λ, τ ;Pre-Computation: N1 and neighbor triangles for each mesh node, ALap;
For k = 1→ convergence
Update
E [u] = D(Re(X )|X=V+u,Te u)5uk
1E [uk1 ]
uk+10 = uk1 − τ 5uk
1E [uk1 ]
Smoothing :
uk+11 =
((1− λ)I + λALap
)uk+10
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 17 / 25
Results and Discussion
Content Adaptive Mesh Generation
For generating the content adaptive mesh, the method proposed by Minget al. [10] is used:
1 Node generation:
Canny sample points;Halftoning sample points;Uniform sample points.
2 Mesh generation via Delaunay triangulation;3 Image-based mesh smoothing:
Image-based Centroid Voronoi Tessellations (CVT) mesh smoothing;Image-based Optimal Delaunay Triangulations (ODT) mesh smoothing;Edge flipping.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 18 / 25
Results and Discussion
(a) (b)
Figure 1: Example of content adaptive mesh generation
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 19 / 25
Results and Discussion
Example 1- Brain CT Images
Mesh has 5406 nodes and 10744 triangles. The average time for eachiteration is about 156 ms for these images. The MSDs before and afterregistration are 271.8 and 77.3 respectively.
(a) (b) (c)
Figure 2: (a) Template image, (b) Reference image, (c) Difference image
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 20 / 25
Results and Discussion
(a)
(b) (c)
Figure 3: (a) Displacement fields in horizontal and vertical directions, (b)Registered image, (c) Difference image after registration
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 21 / 25
Results and Discussion
Example 2- Brain CT Database
80 images, each of the size of 512× 512 pixels. Each mesh containsapproximately 3300 nodes and 6700 triangles.
An implementation of the curvature-based registration method [11] hasbeen used for comparison. This implementation takes advantage of a fastDiscrete Cosine Transform (DCT) solver. The DCT solver is implementedusing the embedded DCT function in MATLAB which uses a Cimplementation.
Table 1 summarizes the computational time of these two methods,implemented on a desktop computer with an Intel Core i7 3.5 GHz CPUand 6 GB of RAM, as well as the mean MSD error of the methods.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 22 / 25
Results and Discussion
Table 1: Computational time and mean MSD error for pixel-based andmesh-based registration methods
Pixel-based Method Mesh-based Method
Mean MSD 116.66 108.91
CPU Time 1534 sec 1320 sec
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 23 / 25
Conclusion
Conclusion
Multi-resolution techniques do not distinguish between regions thathave significant feature content and regions that are featureless.
In octree based methods, the rectangular boundaries do no suitfeature boundaries that tend to be curvilinear.
A new efficient triangular mesh-based image registration technique isintroduced.
Higher speeds can be achieved with C or GPU implementations.
Furthermore, images at any desired resolution can be considered forregistration since we only need to deal with the mesh nodes and notimage pixels.
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 24 / 25
Conclusion
References
Baghaie, A., Yu, Z.: Curvature-Based Registration for Slice Interpolation of Medical Images. In: Zhang, Y.J., Tavares,J.M.R.S. (eds.) CompIMAGE 2014. LNCS, vol. 8641, pp. 69–80. Springer, Heidelberg (2014)
Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vision Comput. 21(11), 977–1000 (2003)
Modersitzki, J.: Numerical methods for image registration. OUP, Oxford (2003)
Sotiras, A., Davatzikos, C., Paragios, N.: Deformable medical image registration: A survey. IEEE T. Med. Imaging 32(7),1153–1190 (2013)
Fluck, O., Vetter, C., Wein, W., Kamen, A., Preim, B., Westermann, R.: A survey of medical image registration ongraphics hardware. Comput Meth Prog Bio 104, no. 3, e45-e57 (2011)
Corvi, M., Nicchiotti, G.: Multiresolution image registration. In: IEEE International Conference on Image Processing1995, Vol.3, 224-227, IEEE Press, (1995)
Haber, E., Heldmann, S., Modersitzki, J.: Adaptive mesh refinement for nonparametric image registration. SIAM J SciComput 30.6, 3012-3027 (2008)
Xu, G.: Convergent discrete Laplace-Beltrami operators over triangular surfaces. In: Geometric Modeling and Processing2004, IEEE Press, (2004)
Desbrun, M., Meyer, M., Schroder, P., Barr, A.H.: Implicit fairing of irregular meshes using diffusion and curvature flow.In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pp. 317-324. ACMPress/Addison-Wesley Publishing Co., (1999)
Xu, M., Gao, Z., Yu, Z.: Feature-Sensitive and Adaptive Mesh Generation of Grayscale Images, In: Y.J. Zhang, J.M.R.S.Tavares (Eds.): CompIMAGE 2014, LNCS 8641, pp. 204-215, Springer (2014)
Fischer, B., Modersitzki, J.: A unified approach to fast image registration and a new curvature based registrationtechnique. Linear Algebra Appl 380, 107-124 (2004)
Ahmadreza, Zeyun, Roshan (CEAS-UWM) Mesh Based Registration December 10, 2014 25 / 25