robot registration - lirmm.fr · robot registration jocelyne troccaz ... summer school on surgical...
Post on 27-Aug-2018
215 Views
Preview:
TRANSCRIPT
1
Robot registration
Jocelyne TROCCAZTIMC Lab.jocelyne.troccaz@imag.frhttp://www-timc.imag.fr/gmcaohttp://www-timc.imag.fr/Jocelyne.Troccaz
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Grenoble GMCAO (CAMI) teamCreated in 1985 by Philippe CinquinHeaded from 1996 by Jocelyne TroccazStrong connection to Grenoble HospitalAbout 35 people
Grenoble HospitalWe are thereGrenoble
2
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Research topics
Imaging
Modelling
Planning and simulation
Data fusionRobotics
Navigation
Sensors
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
What is it?Registration consists in determining geometric relationships between two reference framesRobot registration essentially consists in transferring the planning to the robot coordinate system
Pre-operative data
Intra-operative data
Robot
Image registration
?
?
3
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example
C
C
R
R
T
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Tools
CalibrationTrackingData registration
Using:Patients’ dataExternal objects
RequiresIntrinsic robot calibration
4
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Contents
IntroductionMethodsExamples: four main situationsConclusion
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Possible reference frames of interest
Rpatient
planningRimg_sensor(i)
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rlocalizer
5
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Hardware examplesLocalizers:
Optical, US, magnetic, mechanical arm
Imaging sensors:Fluoroscopy, digital X-Ray, ultrasound
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Examples A: Robodoc, ACRobot, (CAD-Implant)
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient
Rrobot
6
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Examples B: CASPAR
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rlocalizer
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Examples C:Speedy, Cyberknife, MARS
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rimg-sensor2
7
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example D: Cyberknife+Synchrony
Rpatient
planningRimg_sensor1
Intra-operativelyPre-operatively
RpatientRrobot
Rlocalizer
Rimg_sensor2
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Registration basics
Two reference frames RA and RB and a transform TA
B to be determinedSelection of features FA in RA and FB in RB
Definition of a similarity measure (or distance) between FA and FB
Determination of TAB such that the
similarity is maximum (or distance minimum)
TAB= arg min d(FA,TA
B(FB))
8
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Typical 3D/3D rigid registration methods
Point to point (Procrustes)
External fiducialsAnatomical landmarks
Surface registrationAnatomical surface (i.e. ICP, chamfer matching)Template [Radermacher]
Intensity-basedregistration (for images only)
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Examples APre-op: planning on CT dataIntra-op: a robotDeveloped methods:
Robodoc: robot palpation of implantedfiducialsACRobot: robot palpation of anatomicalsurfaceCAD-Implant: fiducials+template (robot is pre-operative)
9
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Robodoc (for hip surgery)
Define precisely the prosthesis position (geometrical or biomechanicalcriteria)Improve the preparation of the hip cavity
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Robodoc [Taylor et al.]
1. Planning: Orthodoc
3. Intra-operative bone milling procedureusing Robodoc (based on the IBM scara robot)
2. Pre-op to intra-op registrationusing implanted titanium pins (anatomical registration in the last version)
6D force sensor
10
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example A.1: Robodoc
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rpatient
planning
3D/3D point-to-pointregistration
Palpation
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
ACRobot [Davies et al.]
« Hands on » robotKnee arthroplasty3 DOFsBone surface palpated with the robotIEEE TRA 03: registration accuracy evaluation
11
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
ACRobot registration tests
Initial estimate from 4 anatomical landmarksICP surface matchingIntra-op criterion: rms distanceDoes a small rms mean a good registration?Experiments with phantom and artificialdata:
Generated palpated points (10 to 100) with or withoutrandom noise added (max up to 1.5mm)Initial estimates in the range of +/-10mm and 2°Known translational and rotational errors
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
ACRobot: results
Nb pts <70 makesICP more problematicResults highlydepend on the data setsMay have a rms=0.6 witherrors of 0.8mm and 2° From [IEEE-TRA03]
12
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example A.2: ACRobot
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rpatient
planning
ICP 3D/3D surfaceregistration
Palpation
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
CAD-Implant [Champleboux et al.]
A system for dental implant assistanceA template associated to fiducials visible on CTA pre-operative robotIntra-operatively: no robot, no computersSurface registration withoutcomputers
13
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example A.3: CAD-Implant
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient
Rrobot
Line-to-line registration
calibration
Template matching
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example B
Close to RobodocKnee application Pre-op: planning on CT dataIntra-op: a robot, a tracking sensorDeveloped method:
implanted fiducials S for registrationfiducials S’ for motion detection
14
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example B: CASPAR
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient Rrobot
3D/3D point-to-pointregistration
Palpation
Rlocalizer
Motion detection
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Examples C
Pre-op: planning on CT dataIntra-op: a robot, X-Ray sensorsDeveloped methods:
Speedy V1 [Lavallée89]: Direct X-ray/robot calibration and manual image registrationCyberknife V1 [Schweikard98]: Indirect X-ray/robot calibration and intensity-based registration
15
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Speedy V1
Stereotactic neurosurgeryPre-operative MR or CTIntra-operative X-Ray (AP and lateral) – several examsDirect X-Ray/robot calibrationManual image registration (anatomical for pre-op/intra-op and markers for intra-op/intra-op)
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
SpeedyX-Ray extrinsiccalibration: bi-plane model
2D/2D image registration fiducials
source
(P1) (P2)
MM2M1
f1: M (u,v) M1(x1,y1)f2: M (u,v) M2(x2,y2)
image
16
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example C.1: Speedy V1
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rimg-sensor2
X-ray/robotcalibration
X-Ray calibration
imageregistration
intra-operativeimage registration
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Cyberknife V1 [Schweikard et al.]
RadiotherapyapplicationComplextrajectories for improved tumordestruction (multiple radiation ports)6 DOFs requiredVery heavy tools
Traditional linearaccelerator set-up
17
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Cyberknife V1
Planning on CT dataIntra-operatively:
Indirect X-Ray/robot calibration (via isocenter)« X-Ray/pre-computedDRRs » intensity-basedregistration (before eachbeam activation) Small motion compensation when necessary / interruption of the procedureand replanning for large motion
A Digitally ReconstructedRadiograph (DRR)
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example C.2: Cyberknife V1
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rimg-sensor2
Robot/Isocalibration
X-Ray calibration
iterated imageregistration
Risocenter
X-Ray/Iso calibration
Set-up
18
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example D: Cyberknife+synchronyPre-op: planning on CT dataIntra-op: a robot, two X-ray sensors, a localizerDeveloped methods:
X-Ray/robot calibrationX-Ray/DRR registration for head motion compensationOr fiducial-basedregistration plus real-time tracking for targetsmoving with respiration
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Real-time registrationLarge motion tracking[Schweikard05]
Internal fiducials(gold seeds) for initial registration External fiducials(IR diodes) for respiration trackingLearning internal/externalfiducialsrelationship
19
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example D: Cyberknife+Synchrony
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient=Riso
Rrobot
Rlocalizer
Rimg_sensor2
Pre-learning
tracking
imagecalibration
X-Ray/robot calib.
initialregistration
indirect
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Another type of solution (E)Target defined in the intra-operative imaging data*Examples
Indirect visual servoing: computing the robot position from the images
PAKY+RCM [Stoianovici et al.], LPR [Cinquin et al.], etc.
Direct visual servoing: modeling variations of the robot position to variations of the target in the images
GABIE [Morel], ZEUS [deMathelin], etc.
*if pre-operative planning: need for pre-op/intra-op registration
20
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example E.1
planning
Rimg_sensor=Rpatient
Intra-operatively
Rrobot
Image analysisand robot model
PAKY+RCM
LPR
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Example E.2
planning
Rimg_sensor=Rpatient
Intra-operatively
Rrobot
Image analysis andImage/robot differential model
toolheart
US plane(ω1,ω2,ω3,d)=f(d(T1,P1), d(T2,P2))
T1
T2
P1
P2
GABIE
21
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
DiscussionModus operandi:many solutions
Palpation (fiducial, anatomy): easy, invasiveImaging (anatomy): more difficult, less or non invasiveTemplate: easy, limited to few applicationsNeed for updated or real-time registration?
No motionMotion detectionDiscrete motion detection and compensationContinuous motion detection and tracking
Summer School on Surgical Robotics Montpellier, Sept. 09
Robot Registration – J. Troccaz
Discussion (cont’d)No universal recipies: depends on the applicationSome important issues
Intra-operative evaluation of registration accuracySafety of real-time registration
top related