robotics autonomous systems - peoplesastry/pubs/... · 2006-04-05 · elsevier robotics and...

ELSEVIER Robotics and Autonomous Systems 21 (1997) 305-316

Robotics and

Autonomous Systems

Milli-robotics for remote, minimally invasive surgery *

S.S. Sastry a,,, M. Cohn b, F. Tendick c a Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94720-1770, USA

b Endoroboties Corporation, P.O. Box 4408, 5, Deerwood Trail, Warren, NJ 07059, USA c Department of Surgery, S-550 Box 0475, University of California, San Francisco, CA 94143, USA

Abstract

In this paper, we describe an ongoing collaborative research project between the Universities of California at Berkeley and San Francisco with Endorobotics Corporation to develop milli-robotic tools for remote, minimally invasive surgery. We describe the limitations of current surgical practice and then describe the technological and scientific issues involved in building a telesurgical workstation. We describe the novel techniques that we have adapted from MEMS for the design of the milli-robots, their actuators, tactile sensors and displays. We also discuss the need for modeling compliant tissue for telepresent manipulation and training. We, then, describe a test bed telesurgical workstation that has been set up at Berkeley. Animal trials are ready to commence on this surgical workstation. Finally, we do a brief review of related projects.

In this paper, we describe our research program for developing tools for minimally invasive remote surgery. Key to this paper is the use of minimally invasive surgery. For urgent remote care, it is usually advisable not to cause additional trauma by an invasive operating procedure and also to keep low the possibility of infection in an incompletely sterile environment. Thus, the need for milli-robotics for this surgery and the need for new kinds of robots, tactile and visual sensors, and human-machine interfaces. While we talk primarily about general surgery in this white paper, we have the goal of eventually being able to do coronary procedures, through minimally invasive thoracoscopes.

Keywords: Minimally invasive surgery; Endoscopy; Milli-robotics; MEMS (micro-electro mechanical systems)

* Research of SSS supported in part by ARO under grant DAAL 03-91-G0191, of MC, FT and SSS by NASA Langley under STTR grant NAS1-20288, and NSF under SBIR grant DMI 94-61299. This paper describes an ongoing joint research project involving the University of California, Berkeley, Endorobotics Corporation and the University of California, San Francisco. The key investigators involved in this project include, in addition to us, R.S. Fearing and L. Stark at Berkeley, W. Tang at Endorobotics Corporation, L. Way at University of California, San Francisco. We encourage all to visit our home page on the WWW with address "http://robotics.eecs.berkeley.edu/~lara/medical.html" for more pictures and movies of some of our systems in operation.

* Corresponding anthor. E-mail: sastry @ robotics.eecs. berkeley.edu.

I. Opportunities in minimally invasive surgery

Minimally invasive surgical techniques, including endoscopy (gastrointestinal surgery), laparoscopy (ab-

dominal surgery), arthroscopy (orthopedic surgery) and thoracoscopy (lung surgery), are revolutionizing surgery. The surgery includes a number of techniques that access internal anatomy via small incisions or orifices. Trauma to surrounding tissue is minimized, thereby reducing recovery time, risk, and costs. The advent and improvement of active optics, including fiber optics, CCD imagers, and CRT displays, has advanced the state of the art in minimally invasive surgery for procedures such as gall bladder removal

0921-8890/97/$17.00 © 1997 Elsevier Science B.V. All rights reserved PII S0921-8890(96)00082-6

S.S. Sastry et al. /Robotics and Autonomous Systems 21 (1997) 305-316 307

Laparoscopic

/ I \ _

Surgeon

Fig. 1. Surgical teleoperation system concept. Not shown are the details of the surgical master and helmet mounted "tiled" displays for the surgeon.

intervention, to treat and stabilize wounded soldiers close to the front lines and also during the course of evacuation from the battlefield. The truck is needed to provide a sterile environment and the minimally invasive procedures are critical to prevent operative trauma to the soldier. We add that there are variations possible on this scenario, but it is clear that the development of this technology is critical to enhance the survivability of war injuries. Further, it is clear that the technology and basic science that will be developed has dual use,

with big wins possible for civilian minimally invasive health care, and also for astronauts in space, miners, fire fighters, and ot]her people working in hazardous environments.

In the last three 3,ears, we have been involved in a joint project between the Department of Electrical En- gineering and Computer Sciences at Berkeley (Profs. Sastry, Fearing, and Stark), Endorobotics Corpora- tion (Cohn, Hatley, and Tang), and the Department of Surgery at UCSF (Profs. Tendick and Way) in developing telerobotic tools for minimally invasive surgery. In this project, we have been developing milli-robotic

tools with actuation, sensing and manipulation at the

millimeter scale for use in minimally invasive surgery. The technological challenges in developing millimeter scale robots, which are capable of developing several Newtons of force, along with force feedback and tactile sensing and display are quite considerable. In Section 4, we will review the status of our progress thus far in the area of technology and the projects for the future.

While technology has been proceeding, the usage of the milli-robotic minimally invasive device is still in its infancy. Surgical manipulations such as excision, ab- lation, cauterization, cutting, grasping, clamping and suturing need precise movements, force control, var- ious levels of power and high dexterity. In addition, we have only a rudimentary understanding of grasping and manipulation of compliant and slippery tissue. We discuss these scientific issues in Section 3. In Section 5 we discuss our ideas of a telesurgical workstation. We have constructed such a telesurgical workstation and are currently in the process of testing it. Photographs and video tape can be made available. We finally describe some other related projects in Section 6.

308 S.S. Sastry et al./Robotics and Autonomous Systems 21 (1997) 305-316

2. Limitations of current technology

In this section, we discuss the limitations of current technology and give a preview of how our project ad- dresses these limitations: (1) Current needle holders, graspers, and other tools

transmit a surgeon's hand motions by passive mechanics. The most serious limitation is caused by the reduced number of degrees of freedom. As the instruments slide, twist and pivot through the point at which they enter the body wall, they are four-degree-of-freedom manipulators. Conse- quently, the surgeon can reach points within a three-dimensional volume but cannot fully control orientation. For simple tasks this is not a major hindrance, but it makes complex tasks such as suturing and knot tying extrem, ely difficult [31 ]. An- other drawback is that the instrument handles are anchored with respect to the patient and as a con- sequence it is difficult for the surgeon to align the video display to the camera and instrument axes. This also results in misleading perspective cues. These difficulties would be addressed by multi- degree-of-freedom end effectors with an appropri- ate surgeon:machine interface.

(2) Another difficulty is that surgery requires fine motion control. Current minimally invasive instruments require a mixture of gross and fine motions, because the entry portal acts as a fulcrum, scaling down the pivoting motions but leaving others un- affected. In open surgery, surgeons can brace their wrists so that arm tremor is not transmitted to the instrument. The need for gross motions precludes bracing in the use of laparoscopic instruments, so that the angular tremors of the shoulder and elbow multiplied by their moment arms make a contribu- tion to the position error. Because the surgeon cannot insert his or her hands directly into the surgical site in minimally invasive surgery, tactile feedback is lost. Feedback of forces and torques at the instrument tip is greatly reduced due to friction and stiffness in the cannulas through which the instruments pass. These difficulties would be alleviated by the use of effective force and tactile feedback, multi-degree-of-freedom end effectors, and filter- ing of hand tremor in the master controller.

(3) Techniques for minimally invasive surgery are characterized by a slow learning curve. Many

surgeons who try to learn advanced techniques fail owing to the complexity of the user interface and the unavailability of animal training facili- ties. This drawback will be addressed by good surgical simulators.

To summarize, some simple minimally invasive procedures can be performed effectively by all surgeons. However, only the most skilled surgeons can currently perform advanced procedures. Also the prevalence of poor technology necessitates the use of unusual or sub- optimal procedures with the attendant enhanced risk of complications. Furthermore, some procedures, such as liver and colon resection to remove a tumor, are so technically formidable that their widespread use is controversial.

3. Basic scientific challenges

Minimally invasive surgery demands extensive training, even for surgeons already experienced in open surgery. Practicing in the animal lab is currently the best way to learn advanced techniques, but this is very expensive, and animals differ from humans in anatomy and pathology. Several groups have produced prototype computer graphics simulators that create virtual environments for surgical training [26]. While the graphics in some of these simulators are sophisticated, tissue behaviors and user interaction are poor because the simulators do not have adequate mechanical models of tissues. Also, the vast majority of the literature on grasping and manipulation of objects using multi-fingered robotic hands (including a recent text book co-authored by one of us (SSS) [22]) deals primarily with rigid objects. In the context of minimally invasive surgery, it is important to hold and manipulate material which is compliant and slippery. In fact, in the course of blunt dissection and suturing, it is important to lose not folds of tissue and to design instrument kinematics and control strategies for their manipulation, which account specifically for the properties of the ambient material.

3.1. Modeling of compliant tissue

The mechanical properties of collagen, the protein responsible for the behavior of most passive tissues, have been well studied, but collagen's properties are

S.S. Sastry et al./Robotics and Autonomous Systems 21 (1997) 305-316 309

complex. Even in a finite element model, the responses of organs depend on the structure of the collagenous tissue, the zero-stress state and loading of the tissue, the shape of the organ, and other factors. Con- sequently, the best way to model tissue behaviors is with data from experimental measurements. We will develop models of tissues during deformation, division (by blunt dissection, cutting, or electrocautery), and suturing from force and displacement measurements taken from the milli-robotic end effector while surgeons use the master controller and robot to perform these tasks. These measurements, with off-line finite element modeling, will allow us to compute key parameters, such as zero-stress state, elasticity, and ul- timate stress, for these tasks performed on key tissues and organs, including arteries, veins, connective tissue, stomach, and colon. This work is best done using ex vivo tissues taken from pigs except where it is necessary to use in vi~vo measurements, as for arteries with blood pressure.

on manipulation of rigid objects. An additional feature to be studied in the manipulation of compliant material is the development of models allowing for the piercing of the compliant surfaces.

3.4. Tactile-based grasping~clamping

While most of the problems involved in manipulation of compliant material will need some form of tactile sensing, tactile sensing will be especially useful to feel changes in texture, compliance and other charac- teristics in the tissue being manipulated to sense the presence of solid objects (metal, shrapnel, tumors) to grasp and to be able to sense injured vessels, veins, fascia to clamp and or suture the wounds.

4. Technological challenges

4.1. Prototyping of millimeter scale robotic "hands"

3.2. Grasping of compliant material

Compliant material cannot be grasped as easily as rigid material. Simple experiments with holding a piece of Jello show that it is difficult to squeeze compliant material, without changing the position and nature of the grasp contact. Traditional methods of analyzing the grasp stability and manipulability of compliant materials, as elucidated for example in [22], fail. Using models of compliant tissue, one needs to develop new concepts of manipulability and to study the design of kinematic configurations of fingers and finger tips to allow for tasks such as blunt dissection, which involves spreading apart connective tissue surfaces, for example to isolate vessels.

3.3. Dual (multi-) hand manipulation of compliant material

As pointed out in Section 3.2, the derivation of grasp maps for compliant tissue needs to be reformulated. However, problems in the manipulation of compliant tissue are far more substantial. In tasks such as suturing, two pieces of compliant tissue need to be attached using two or more robotic hands. This involves not only grasping, but also manipulating, piercing and mating surfaces. Traditional studies [22] have relied

To achieve teleoperation in minimally invasive surgical procedures as well as in remote surgery, the most promising approach is to use a robotic multi-degree- of-freedom hand in conjunction with computer control and a user interface such as a force reflecting master. To meet the demands of complexity, small size, and low cost we have developed a milli-robot design technology by adapting microfabrication to the millimeter scale. Specifically, we have fabricated these millimeter scale mechanisms using a combination of lithography, etching, and injection molding for planar layered structures. Our methods are an adaptation of those from micro-electro mechanical systems (MEMS) to the millimeter scale and involve an integrated design of actuators, sensors, and mechanisms.

We have developed a milli-robot design technology by adapting microfabrication to the millimeter scale. A planar layered structure will facilitate batch fabrication and miniaturization. Motors, bearings and other mechanical components have been fabricated by lithography, etching, injection molding, and related techniques [8,20], but these actuators have insuffi- cient force output. We have done work in designing and operating tendon-driven multi-fingered robotic hands [6,23,35]. However, tendon driven actuators also do not produce enough force. Consequently, we have deviated from traditional electrostatic, piezo and

310

Fig. 2. A prototype two-degree-of-freedom hydraulic "hand" fabricated using the moulding techniques described in the text. The "hand" is shown inside a 10mm trocar of the kind most commonly used in gastrointestinal surgery.

S.S. Sastry et aL/Robotics and Autonomous Systems 21 (1997) 305-316

Fig. 4. Detail of a laparoscopic manipulator, showing the positioning of the milli-robot inside the body of the patient.

10mm (as required by currently used cannulas), the amount of force needed for fine motion tasks such as suturing is roughly 10 N at the gripper. For the joints to achieve a speed comparable to human fingers we need about 3-5 Hz. Calculations show that this is feasible, and we have produced actuator designs to achieve this.

We have prototyped multi-degree-of-freedom "hands" of the kind shown in Fig. 4, and are mak- ing several variants on the kinematics to reflect the functionality required in several tasks.

Fig. 3. Another laparoscopic manipulator, with some novel actuation mechanisms also designed for a 10mm trocar. Figure: Courtesy of the Endorobotics Corporation.

magnetic designs with hydraulic actuators. Dextrous low cost manipulators meeting the size and force requirements of minimally invasive surgery can be built using these hydraulic actuators. The details of their construction and performance are under patent review. Examples of the designs that we have been working on are shown in Figs. 2 and 3.

4.2. Kinematic design

The application needs an approximately anthropo- morphic arm as shown for example in Fig. 4. Assum- ing an arm length of roughly 20 cm and width of about

4.2.1. Actuator fabrication methods We have prepared many different technologies for

fabrication: (1) Molds using a bench top CNC (computer numeri-

cally controlled) mill and cast parts using resin. (2) Table top CNC mills to machine metallic parts. (3) Assorted techniques for fabricating bladders and

other hydraulic actuators. Lithography has been used to deposit sensors, and laser cutting was used for some high precision ma- chining. Initial research has comprised of two tasks: prototyping actuators, sometimes as an integral part of the hand mechanism, and testing them. Several prototypes have been created in order to experiment with laminating adhesives and sealants and to study leakage and catastrophic failures. The velocity of the actuators and joints is measured using a Hall effect integrated position sensor and the force by strain gauges.


4.2.2. Multiple degree-of-freedom hand designs Key issues to be studied in addition to the func-

tional design of the kinematics of the manipulator [22] include: (1) the integrated design of the mechanism with our

actuators; (2) leakage issues around the shaft of the piston (our

design minimizes this); (3) static friction, which may impede the controllabil-

ity of the actuate,r; to this end the actuators will be built with integrated position sensors;

(4) sterility issues to prevent contaminants from en- tering the body of the patient.

4.3. Milli-scale teletaction

In surgery, the sense of touch is particularly valu- able in preventing tissue injury, diagnosing tissue condition, and guiding instrument motions when the camera's view is obscured. The surgeon needs to be provided with complete haptic information, including local contact shape and pressure information, as well as net force feedback. We will develop a millimeter scale teletaction system which will include a tactile sensor array mounted on the remote instrument, a processing system, and an analog force display giving the operator a realistic and comfortable sensation of grasped tissue or the contact at the end of a catheter. Such a system will also help in the perception of structures hidden from view, such as vessels embedded in connective tissue or ,:umors within the liver or colon.

4.3.1. Tactile sensor design In earlier work, Fearing developed technology for

capacitive-based tactile sensing arrays in planar and cylindrical geometry [9]. The normal strain sensitive elements have a raw sensitivity (dependent on contact area) of less than 1000Pa, bandwidth of 100Hz, and typically an 8 x 8 array of elements. Using silicon surface micromachining techniques, we (Fearing, Gray, Cohn, and others) are currently developing a tactile sensor for the end of a catheter, with 125 micron spac- ing. Sensors will be developed both for the gripper fingers, and initially, as a simple 1 cm diameter probe for exploratory touching. The main research issues to be resolved for the tactile sensor design are packaging to protect tissue and the sensor, and cabling to bring sig- nals out of the body without interfering with the range

of motion of the milli-manipulator. We plan to use a standard Berkeley CMOS/sensor process to integrate analog and digital circuitry with our sensors and explore the use of fiber-optic based tactile sensors which are inherently safe and have high noise immunity.

4.3.2. Tactile processing algorithms A direct map from surface pressure on tactile sen-

sor to surface pressure on the tactile display should provide the most realistic sensations to the surgeon (albeit with a low-pass filtered pressure distribution due to anti-aliasing measures). However, to enhance perception of internal tissue structure, we will explore pre-processing of the tactile strain data to extract quan- titative information about the shape of inclusions or to amplify pulse sensations. Another example of a particularly useful sensation that is lost without tactile feedback is the detection of separation as tissues are pulled apart. We will attempt to detect separation from the relative tissue motion on the tactile probe surface, and enhance this motion, as necessary, on the display.

4.3.3. Tactile display design The tactile display device requires a true analog

force display to be able to present shape and hardness features to the surgeon. We have fabricated a stim- ulator prototype with a 5 x 5 array of pneumatically driven pistons [3], with 3 bits of force display and 8 Hz bandwidth. The prototype uses PWM driven solenoid valves which are too noisy and bulky for practical use, so we will explore using miniature proportional valves to control the actuators. To overcome problems in the prototype with friction and leakage, we will design a new 6x6 tactile display (about 1.5 cm 2) molded in silicone rubber using miniature bellows. This display will comply to the surgeon's finger, and be compact enough to attach to the manipulandum as in Fig. 5.

4.4. Image/video acquisition and registration

While all types of acquisition are relevant for medical imaging, we will focus on optical and ultrasonic acquisition methods because both permit faster re- construction than back projection methods and can be used on small probes. For example, small ultrasonic sensors have been developed for use inside the body. We will also study the enhancement of images

312 S.S. Sastr3' e t a L / R o b o t i c s a n d A u t o n o m o u s S y s t e m s 21 ( 1 9 9 7 ) 3 0 5 - 3 1 6

!i~,i !i ~ =

i i!ii!i !iiiii! i ii i i!i ii i i !ii !i /iiiii !i ii giiii!i !! iii ! ! i i~i!ili~ii~il ili

Fig. 5. A sample master manipulandum with the integration of force feedback. Not shown is the integration of tactile sensors into the master. The master has seven degrees of freedom.

captured from small sensors. These images tend to be plagued by noise and lack resolution. However, multiple views are available (in CCDs, for example) and enhancement methods from digital television, based on motion compensation, can be used to both improve resolution and reduce noise. Enhancement methods range from very simple linear and nonlinear filters to sophisticated iterative procedures. The trade-offs between different methods, given the near real-time constraint of the monitoring task, are being investigated. Bounds on possible resolution enhancements (e.g. given an oversampling due to multiple images of the same object) will be studied. An important topic of investigation related to image acquisition, using vari- ous methods and acquisitions at different time instants (as is typical in a dynamic environment like surgery), is image registration. That is, different views (possibly using different sensors) of possibly deformable objects

have to be fused to provide a consistent picture. This task involves both traditional signal processing methods (e.g. interpolation, smooth deformation) and computer vision techniques (e.g. automatic labeling and tracking of objects).

4.4.1. Image~video transmission and compression Because of the large amounts of real-time visual

data to be transferred, as well as the need for storage and retrieval, image/video compression techniques represent an enabling technology for remote medical procedures. Original requirements of medical imaging will be specifically considered, such as the possibility of joint lossy and lossless compression. Also, specific modifications of standard algorithms (like JPEG and MPEG I/II) in order to meet requirements for medical applications will need to be investigated. We believe that efficient compression will be at the heart of the use of high speed networking for remote medical work, be it diagnosis or intervention. However, high speed networks have some peculiarities (for example, the quality of service) which are difficult to recon- cile with the high demands of medical information exchange. Therefore, the problems of service guaran- tees on high speed networks needs to be considered for high quality video services.

5. Telepresent surgical workstation: Systems integration issues

The devices that we have built for laparoscopic manipulation and feedback can restore, and in some ways surpass, some of the dexterity and sensation of open surgery. It is important that they be engineered to present the surgeon with a unified, intuitive interface. An example of a master controller (manipulandum) with which the surgeon interacts with the robot inside the body is shown in Fig. 5. Initially, we have used a commercial four-degree-of-freedom joystick, developed by Immersion Corp., customized with three extra degrees of freedom. The interface will include finger masters with tactile display units. The system, including the master controller, computer, and robot, can be thought of as a tool acting as an interface between the mechanical impedance of the surgeon's hand and tissue inside the patient. The system has poten- tial advantages not possible with conventional surgical

S.S. Sastry et at./Robotics and Autonomous Systems 21 (1997) 305-316 313

instruments, such as the ability to scale motions and forces independently or to filter hand tremor.

Good visual display of the surgical environment is essential to the surge, on. We will evaluate the use of stereoscopic laparoscopes and displays. As fiat panel displays with resolution and brightness comparable to CRT displays become available, it will become possible to place multiple displays at ergonomic viewing lo- cations for the primary surgeon and assistants. Images and data from multiple modalities can be integrated using computer graphics and digital video, including: live enhanced ultrasound, force, or tactile information; previously obtained X-ray, CT or MR images; loca- tions of instruments out of the camera view; patient data and monitoring; and video images from multiple scopes. Optical tracking of markers on the patient, laparoscope, and instruments will be used to deter- mine relative positions for registration of data from multiple modalities. The surgical simulation will be developed on the same platform (Silicon Graphics workstations), so it will be possible to integrate simulation with live video or three-dimensional imaging data, allowing the surgeon to practice before perform- ing the operation. Simulation will be an essential element in remote surgery in which communication time delays are significant.

Technology can reduce health costs by improving efficiency in the ope, rating room, decreasing complications, and shortening hospital stays and recovery times. Consequently, an essential part of this research project is the development of testbeds for evaluation of the technologies and modeling of surgeon-machine performance. We h~we established two testbeds for system integration. The first, located at Berkeley, is used for development and prototyping. Devices that show promise will then be incorporated into the evaluation testbed at UCSF, where surgeons test the system in representative surgical tasks. We use the surgeons as subjects in performance of tasks used in advanced procedures, including: traction and retrac- tion, blunt dissection, division, detection of embedded structures, catheterization, ligation, suturing and knot tying, and anastomosis. Fig. 6 shows a current testbed at Berkeley with the four-degree-of-freedom laparoscopic robot attached to a three-degree-of-freedom positioning robot (to hold the trocar) connected to a master with seven degrees of freedom, and force feedback. The tactile sensors are not yet shown in-

Fig. 6. Showing our first prototype telesurgical workstation, featuring a four-degree-of-freedom laparoscopic robot, three- degree-of-freedom positioner, and the seven-degree-of-freedom surgical master.

corporated into the master. We are investigating the effect of key system parameters, including sensory display bandwidth and update rates, transmission time delays, and correspondence between display and control coordinate frames. Initially, we will perform these experiments using ex-vivo tissues and organs. Later, we will test system prototypes during performance of these tasks in realistic situations in anesthetized pigs, including dissection of peritoneum near the esophagus, catheterization of the common bile duct, suturing on the stomach surface, and tactile location of hidden structures. Finally, surgeons will perform Nissen fundoplications, a complex operation to control stomach reflux that entails performance of all of the component tasks.

Safety is an essential system design factor. Range of motion, including velocities, accelerations, and forces, will be recorded during system evaluation. These data will be used to place software and hardware limits on robot motion to limit damage in case of failure. Re- dundant sensors will be used, and redundancy, consis- tency, and system "heartbeat" checks will be included in system software and hardware design.

5.1. Enhanced teleoperation

Head mounted displays with head motion tracking will allow the surgeon to explore the real and virtual display by looking around. We will evaluate slaving the laparoscope motion to the surgeon's head

314 s.s. Sastry et al./Robotics and Autonomous Systems 21 (1997) 305-316

movements. This would eliminate the need for a camera assistant and perhaps enhance depth perception by motion parallax. For remote surgery, the surgeon will require not only the laparoscope view, but images of the patient, operating room environment, and patient monitoring data as well. These real and synthetic images can be tiled onto a "virtual dome" [13] so that as the surgeon looks around he or she sees a cohesive environment.

5.2. Semi-autonomous operation

While teleoperation is an attractive way to keep the surgeon in the loop in surgical procedures, it is important to have the system be partially autonomous, so as to allow for its safe operation when communications are temporarily lost to the master and also when time delays between the remote surgeon and the manipulator become large. The strategies, we have in mind for this involve furnishing the slave devices with enough compliant control and on board intelligence to allow for storage of commands from the surgeon and play- ing them forwards. Important questions about the nature of the visual and force and tactile feedback still need to be answered for this mode of operation.

5.3. Supervisory control for low latency operation

An issue related to semi-autonomous operation is the question of supervision of the activities of the robotic system to allow for low latency response, if untoward events occur. In all minimally invasive procedures there is a danger that sudden and uncontrolled bleeding might necessitate more invasive techniques for accessing the affected site by the medical techni- cians on the site. The question of how to shift this control from the remote site to the patient's side in a low latency way needs to be answered.

6. Related other projects

Here we discuss a few additional projects involving related work that we are aware of. The list is not exhaustive. Jacobsen at the University of Utah has demonstrated several teleoperated systems, including the Utah-MIT hand which is a tendon-driven multi-fingered hand with pneumatic actuators [16].

This group has also recently been working on smaller versions of their pneumatic actuators [28]. Salisbury and his group at the Artificial Intelligence Laboratory at MIT have demonstrated the use of tendon-driven three-fingered robotic hands that are roughly twice the human size [ 19]. Anecdotally, we have heard that he is working on muscle-like actuators. Suzumori and co-workers [29] have demonstrated millimeter sized pneumatic manipulators, which unfortunately can only generate forces in the milli-Newton range. A force-reflecting motion scaling wrist for microsurgery has been developed by Hollis et al. [14]. A related project involving the use of robots (but not teleoperation) in surgery is the pioneering work of Paul and coworkers [24] on hip replacement. Another such project is the knee replacement project of the Rizzoli Institute in Bologna [7,18]. Microsurgical systems are being developed by Hunter at MIT [15] and Schenker at JPL [27]. System concepts similar to ours, but different in many important details, have been proposed by Satava [25], Green [12] and Coletta and Marcucci [5]. Also worthy of mention in this regard is the large project underway at the Faculty of Medicine in Grenoble under the guidance of Cin- quin, with many important advances in sensing and registration for brain surgery and peritoneal surgery; see for example [1,2,33]. Another effort is underway at Carnegie Mellon [17]. Another system for robotic endoscopic surgery that is close in concept to ours has been proposed by Taylor and co-workers [11,30]. All these systems involve the use of macro-scale robots with redundant degrees of freedom for positioning laparoscopic instruments or conventional surgical instruments above the body of the patient. Our proposed system will, however, provide several degrees of freedom inside the body of the patient. It will further help develop disposable tools.

References

[1] E. Bainville, R Chaffanjon and E Chinquin, Computer generated visual assistance to a surgical operation: The retroperitoneoscopy, Proc. AAAI (1994).

[2] E. Balnville, E Chaffanjon, S. Lavallee, J. Troccaz and R Cinquin, Virtual medicine, preprint, Faulte de Medecine, Grenoble University Hospital, 1994.

[3] M. Cohn, M. Lain and R. Fearing, Tactile feedback for teleoperation, SIE Conf. 1833 on Telemanipulator Technology (1992).


[4] M.B. Cohn, L.S. Crawford, J.M. Wendlandt and S.S. Sastry, Surgical applications of milli-robots, Journal of Robotic Systems 12 (6) (1995) 401-416.

[5] A. Coletta and L. IVlarcucci, Laparoscopic colectomy and its implications for the use of virual reality in abdominal surgery, in: Medicine Meets Virtual Reality I (1992).

[6] D.C. Deno, J.M. Wendlandt, M.B. Cohn and S.S. Sastry, Actuation and force sensing for cable driven, Teleoperated manipulators, in: Medicie Meets Virtual Reality I (1992).

[7] M. Fadda, T. Wang, M. Marcacci, S. Martelli, P. Dario, A. Visani and M. Nanetti, Matching between virutal image and real robot world in robot assisted surgery, in: Medical Robotics and Computer Assisted Surgery (1994).

[8] L-S. Fan, Y-C. Tai and R.S. Muller, IC-processed electrostatic micromotors, Sensors and Actuators 20 (1989) 41-48.

[9] R.S. Fearing, Tactile sensing mehanisms, International Journal of Robotics Research 9 (1990) 3-23.

[10] M. Freudenheim, Surgery the kindest cut of all is no cut at all, The New York Times (1991) D1.

[11] J. Funda, R. Taylor, K. Gruben and D. LaRose, Optimal motion control for teleoperated surgical robots, SPIE Conf. Record Volume 2057 (1993) 211-223.

[12] ES. Green, Telepresence: Advanced manipulator technology for minimally invasive surgery, in: Medicine Meets Virtual Reality I (1!)92).

[13] M. Hirose et al. A study on synthetic visual sensation through artificial reality, Proc. 7th Syrup. on Human Interface (1991) 675-682.

[14] R.L. Hollis, S.E. Saicudean and A.E Allan, A 6-degree of freedom magnetically levitated variable compliance fine-motion wrist-design, modelling and control, IEEE Transactions on Robotics and Automation 7 (3) (1991) 32-332.

[15] I.W. Hunter, T.D. Doukoglou, S.R. Lafontaine et al. A Teleoperated microsurgical robot and associated virtual environment for eye surgery, Presence 2 (4) (1993) 265- 280.

[16] S. Jacobsen, E. Iversen, D. Knutti, R. Johnson and K. Biggers, Design of the Utah/MIT dextrous hand, IEEE Int. Conf. on Robotics and Automation (1986).

[17] T. Kanada, M.L. Reed and L.E. Weiss, New technologies and applications in robotics, Communications of the ACM 37 (7) (1994) 58-67.

[18] M. Marcacci, M. Fadda, P. Dario, S. Martelli, A. Visani and T. Wang, The total knee arthroplasty project: A demonstration feasability of the computer assited approach. Proc. 1st Europear,~ Conf. on Medical Robotics, ROBOM (1994).

[19] M. Mason and J. Salisbury, Robot Hands and the Mechanics of Manipulation (MIT Press, Cambridge, 1985).

[20] W. Menz, W. Bacher, M. Hermening and A. Michel, The LIGA technique - a novel concept for microstructures and the combination with Si technologies by injection molding, Proc. 4th IEEE Workshop on Micro Electro Mechanical Systems (1991) 69--73.

[21] T. Mori, S. Bhoyrul and L. Way, History of laparoscopic surgery, in: Fundamentals of Laparoscopic Surgery (1995) 1-12.

[22] R. Murray, Z. Li and S.S. Sastry, A Mathematical Introduction to Robotics Manipulation (CRC Press, Boca Raton, 1994).

[23] R.M. Murray, D.C. Deno, K.S.J. Pister and S.S. Sastry, Control primitives for robot systems, IEEE Transactions on Systems, Man and Cybernetics 22 (1) (1992) 183-193.

[24] Paul et al. Medical news and perspectives, for better hip replacement results: A surgeon's best friend may be a robot, Journal of the American Medical Association 267 (1992) 613-614.

[25] R.M. Satava, Nintendo surgery, Journal of the American Medical Association 267 (1992) 297-304.

[26] R.M. Satava, Virtual reality surgical simulator, the first steps, Surgical Endoscopy 7 (3) (1993) 203-5.

[27] P.S. Schenker, H. Das and T.R. Ohm, A new robot for high dexterity microsurgery, Conf. on Computer Vision, Virtual Reality and Robotics in Medicine (1995).

[28] E Smith, S. Jacobsen, D. Potter and C. Davis, Miniature high performance servovalves, Int. Fluid Power Exposition and Technical Conf. (1992).

[29] K. Suzumori, S. Iikura and H. Tanaka, Development of a flexible microactuator and its application to robotics mechanisms, IEEE Int. Conf. on Robotics and Automation (1991) 1622-1627.

[30] R. Taylor, J. Funda, D. LaRose, Y. Kim, N. Bruun, N. Swamp, C. Cutting and M. Treat, A passive/active manipulation system for surgical augmentation, Proc. 1st Int. Workshop on Mechatronics in Medicine and Surgery (1992) 115-122.

[31] E Tendick, R. Jennings, G. Tharp and L. Stark, Sensing and manipulation problems in endoscopic surgery: Experiment, analysis and observation, Presence 2 (1) (1993) 66-81.

[32] E Tendick, T. Mori and L. Way, Future of laparoscopic surgery, in: Fundamentals of Laparoscopic Surgery (1995) 235-252.

[33] J. Troccaz, S. Lavallee and E. Hellion, PADyC: A passive ann with dynamic constraints, Proc. ICAR (1993).

[34] L.W. Way S. Bhoyrul and T. Mori, Fundamentals of Laparoscopic Surgery (1995).

[35] J. Wendlandt, Milli robotics for endoscopy, MS report, University of California, Berkeley (1993).

S. Shankar Sastry got his Ph.D. in electrical engineering from the Univer- sity of California, Berkeley in 1981. He taught at MIT in 1981-1982 and has been at Berkeley since 1983. He is currently a Professor of Electrical Engi- neering and Computer Sciences and the Director of the Electronics Research Laboratory. He was a Gordon McKay Professor of Electrical Engineering and Computer Sciences at the Division of Applied Sciences, Harvard University

316 S.S. Sastry et al./Robotics and Autonomous Systems 21 (1997) 305-316

in 1994, and a visiting Vinton Hayes fellow at MIT in the fall of 1992. Shankar Sastry won the IEEE Region X Best student paper award in 1977, the President of India medal in 1977, the NSF Presidential Young Investigator award in 1985, the Eckman Award of the American Control Council in 1990 and an MA (Honoris causa) from Harvard in 1994. He is a Fellow of the IEEE. He has authored papers in the areas of nonlinear control, the hierarchical organization of the control of complex systems, hybrid systems, adaptive control, robotics, nonholo- nomic mechanics and multi-fngered robot hands. He is the co- author of two books "Adaptive Control: Stability, convergence and Robustness" with Marc Bodson (Prentice-Hall, 1989) and "A Mathematical Introduction to Robotic Manipulation", with Richard Murray and Zexiang Li (CRC Press, 1994) and has co-edited a book titled "Hybrid Systems II", Lecture Notes in Computer Science, Springer Verlag, 1995. Most recently, he has been interested in millimeter scale robotics for surgery and simulation and visualization techniques for training surgeons.

............ Frank Tendick received the S.B. degree in aeronautics and astronau- tics from the Massachusetts Institute of Technology, the M.S. degree in mechanical engineering from the Uni- versity of California, Berkeley, and the Ph.D. degree in bioengineering from the University of California, San Francisco and Berkeley campuses. Since 1994, he has been in the De- partment of Surgery at UCSE His interests include human - machine

systems, telerobotics, and virtual environments, especially as applied to surgery.

Michael Cohn is a Ph.D. candidate in EECS at the University of California, Berkeley. His current research interests include millimeter-scale actuators and assembly of MEMS. He is a graduate of Harvard University (A,B. Physics 1990) and serves now as CEO of Endorobotics Corporation.

robotics autonomous systems - peoplesastry/pubs/... · 2006-04-05 · elsevier robotics and...

Documents