intensity-based deformable registration of 2d fluoroscopic x- ray images to a 3d ct model aviv...
TRANSCRIPT
Intensity-based deformable registration of 2D fluoroscopic X-
ray images to a 3D CT model
Aviv Hurvitz
Advisor: Prof. Leo Joskowicz
2
Navigation in orthopedic surgery
Applications
• Determine position of surgical tools relative to anatomy
• Position surgical robots
• Match pre-operative model to anatomy
[medtronic.com][hss.edu]
3
Navigation in orthopedic surgery
Related registration methods
• Registration to fiducials– Implanted fiducials– On skin
• Contact based registration (Point-cloud to surface registration)
• Registration of fluoroscopic X-ray to CT
4
Registration of fluoroscopic X-ray to CT
[LaRose 2001; Tomazevic 2003; Knaan 2003; Livyatan 2003; and others]
Preoperatively:1. Acquire CT of anatomy of interestIntraoperatively:1. Fix a tracking marker to the bone2. Acquire 2-5 fluoroscopic X-ray images from various camera
poses3. For each image, record:
• T(trackercamera) - camera position at acquisition time• T(trackerbone) - bone marker position at acquisition time
4. Obtain an initial transformation estimate T(boneCT)5. Estimate transformation T(boneCT) with the algorithm we
describe next
Tracker updates T(trackerbone) continuously, enabling real-time navigation.
5
Registration of fluoroscopic X-ray to CT
Input:1. For 2-5 X-ray images:
a. T(trackercamera) at acquisition timeb. T(trackerbone) at acquisition time
2. Initial T(boneCT) estimateRepeat until convergence:
1. For each camera position:a. Define a virtual camera positioned relative to CTb. Create DRR
2. Rate similarity of DRRs to X-ray images by a similarity metric
3. Determine the next T(boneCT) by an optimization algorithm
6
DRR creation
[Knaan et al., 03]
7
Is it possible to do the registration without a CT?
Advantages:
• Save time before operation
• Save costs
• Decrease radiation exposure
Disadvantages:
• A patient-specific CT provides more information
8
Is it possible to create a patient-specific model without a CT?
Approach• Use prior information – everyone’s bones are
similar• Deduce exact shapes of bones from the
fluoroscopic images
In absence of CT, need to search for both bone pose (6 d.o.f.) and the patient-specific bone shape
9
Image analysis-by-synthesisDefine a model of the bone which is parameterized by
pose and by shape.Repeat until convergence:
1. Simulate X-ray imaging process to create DRRs2. Rate similarity of DRRs to fluoroscopic images by similarity
metric3. Modify pose and shape parameters by optimization
algorithm
When the DRRs match the fluoroscopic images We found the pose and the shape of the bone We have registration: a transformation from every
point in the model to its corresponding point in the patient anatomy.
We can map data from the model to the patient anatomy. E.g., map the surface of the bone.
10
Active Appearance Model (AAM)[Cootes et al., 98; Matthews and Baker, 04]
• Model shape variations and appearance variations compactly (with few parameters)
• Can be trained from a dataset of samples • A generative model – can generate new images which
are similar to the training images• Image analysis-by-synthesis
[Edwards, Taylor and Cootes, 98]
11
• Each shape is represented by a vector of landmark coordinates
Shape• Manually label landmark points on training
images• Define a triangular mesh between the landmarks
[Matthews and Baker, 04]
12
• Procrustes analysis - align meshes to cancel global variations in translation, rotation and scale
• PCA on shape vectors
13
Appearance
• Warp each training image to the base mesh so. This creates a set of shape-normalized images.
• Represent pixels inside the base mesh so as vectors
• PCA on shape-normalized image vectors:
14
The resulting model
[Matthews and Baker, 04]
15
Instantiating the model
• Input:– Shape parameters – Appearance parameters
• Computation:1. Instance’s shape-normalized appearance
2. Instance’s shape mesh
16
Instantiating the model
[Matthews and Baker, 04]
17
Fitting AAMs to images
• Input:– Image I(x)– AAM
• Goal: find shape p and appearance λ that yield an AAM image similar to I(x).
• Minimize SSD over λ and p:
Sum differences over all pixels x in base mesh
18
• The optimization is nonlinear in p and linear in λ.• The parameters are found by iterative gradient
descent.• Standard gradient descent is slow because it
requires calculating the partial derivatives at every iteration.
• There are tricks to estimate gradient quickly [Cootes, 98] , or use a pre-computed gradient.[Matthews and Baker, 04].
• After improvements, algorithm is fast enough for real-time face tracking
19
Using AAMs for 2D-3D registration
• The training dataset is CTs (3D images). ROIs of femoral head approximately 100x100x100 pixels.
• The shapes are defined by tetrahedral meshes (approx. 3,000 nodes and 20,000 tetras).
• It is too difficult to find landmarks manually.
20
Finding landmarks automatically
• Select one CT arbitrarily as a template.• Define a tetrahedral mesh on the template.• Perform 3D-3D deformable registration of
each CT to the template. Defines a transformation
Ti:TemplateCTi
• Apply transformation to template mesh transfer mesh to the corresponding landmarks in each CT.
21
Finding landmarks automatically• 3D-3D registration performed with Elastix software (an
ITK wrapper) [Klein and Staring, 07]
• Registration procedure1. Preparation: match histogram to template2. Rigid registration:
• Initialize by aligning centerpoints of ROIs• Metric: Normalized Cross Correlation• Optimizer: Standard gradient descent
3. Deformable registration• Transform: 3D B-spline. (Grid spacing 8x8x8 voxels)• Metric: Sum of squared differences (SSD)• Optimizer: Conjugate gradient• Use image pyramids (3 levels)
• Unsupervised• Each registration runs approx. 10 mins.• Compute all registrations in parallel on bmos cluster
(mosix)
22
Building AAM of femur CTs• Input:
– 14 CTs and 14 corresponding meshes
• Output:– Mean shape mesh and 6 significant basis
vectors– Mean appearance image and 10 significant
basis vectors
23
Mean volume image - A0
24
Shape model
s0 + 2 std s1 s0 + 2 std s2 S0
25
Using the AAM in 2D-3D registration
Repeat until convergence:• Generate 2D DRRs out of AAM using current
pose, shape, and appearance parameter estimates.
• Rate similarity of DRRs to fluoroscopic images by similarity metric
• Modify AAM shape parameters and the pose parameters by optimization algorithm. (We ignore appearance variations)
26
Generating DRRs from AAM – Algorithm 1
1. Generate a 3D image of the AAM instancefor each pixel in 3D image:
– find tetrahedron containing pixel center (use KD-tree)
– find corresponding point in base mesh (an affine transform)
– read value from appearance image
2. Proceed with standard ray casting of 3D image
[Knaan et al., 03]
27
Generating DRRs from AAM – Algorithm 2
• Cast rays directly through tetrahedral mesh of AAM instance
• Sample points at uniform intervals on ray• Find corresponding points in base mesh• Read values from appearance image• Requires identifying all ray-tetrahedron intersection
points. We implemented algorithm of [Marmitt and Slusallek, 06]
Instance mesh
Base mesh Shape-normalized
volume image
28
Parameter search issues
• Accuracy and Robustness– Search from various initial locations– Genetic algorithm– Downhill simplex optimizer (ITK amoeba)
• Speed– Multi-resolution– Limit DRR generation to ROIs
29
Current status
• The naïve search converges to unsatisfactory result (local minimum)
• Need to improve both accuracy and speed– Search from multiple initialization points– ROIs
• Think about a smarter optimization algorithm
Future:– Experiment on real fluoroscopic images
30
Intensity-based registration of 2D X-ray images to a 3D deformable
model
31
Principal Component Analysis (PCA)